Create a connector
Description
Use the connector create command to add a connector to either a self-managed OpenSearch cluster or Amazon OpenSearch Service. You can create a connector interactively by following prompts or by using a preconfigured YAML file.
To create a connector in self-managed OpenSearch, do the following:
Select from supported connectors and their associated models
Provide necessary configuration parameters (for example, API key)
OpenSearch Service automatically handles AWS-specific setup, including the following components:
IAM role creation and configuration
Backend role mapping
Secret creation
To create a connector in OpenSearch Service, do the following:
Select from supported connectors and their associated models
Provide AWS-specific parameters (for example, AWS Region)
The CLI guides you through providing only the essential model-specific information needed for your chosen environment.
Prerequisites
Before creating a connector, ensure you have completed the setup process or have a setup configuration file. For setup instructions, see setup guide.
Command syntax
opensearch-ml connector create [--path <value>]
Option:
--path <value>: (Optional) The path to an existing connector configuration YAML file
Usage examples
Creating a connector interactively
To create a connector interactively, run the following command:
opensearch-ml connector createSample response:
Starting connector creation... Enter the path to your existing setup configuration file: /Documents/cli/setup_config.yml Setup configuration loaded successfully from /Documents/cli/setup_config.yml Please select a supported connector to create in Amazon OpenSearch Service: 1. Amazon Bedrock 2. Amazon Bedrock Converse 3. Amazon Comprehend 4. Amazon SageMaker 5. Amazon Textract 6. Cohere 7. DeepSeek 8. OpenAI 9. Custom connector Enter your choice (1-9): 8 Please select a model for the connector creation: 1. Embedding model 2. Custom model Enter your choice (1-2): 1 Enter your OpenAI API key: ******** Enter your connector role prefix: test-role Enter a name for the AWS Secrets Manager secret: test-secret Creating OpenAI connector... Step 1: Create Secret test secret exists, skipping creation. ---------- Step 2: Create IAM role configured in connector Role 'test-role-openai-connector-03ae00' does not exist. ---------- Step 3: Configure IAM role in OpenSearch Step 3.1: Create IAM role for Signing create connector request Role 'test-role-openai-connector-create-03ae00' does not exist. ---------- Step 3.2: Map IAM role test-role-openai-connector-create-03ae00 to OpenSearch permission role ---------- Step 4: Create connector in OpenSearch Waiting for resources to be ready... Time remaining: 1 seconds.... Wait completed, creating connector... Connector role arn: test-connector-role-arn ---------- Successfully created OpenAI connector with ID: connector123 Enter the path to save the output information, or press Enter to save it in the current directory [/Documents/cli/output.yml]: Output information saved successfully to /Documents/cli/output.yml
Creating a connector using a configuration file
To create a connector using a configuration file, run the following command:
opensearch-ml connector create --path /Documents/cli/connector_config.ymlThis example assumes that you have a connector_config.yml file at the specified path with the following content:
setup_config_path: /Documents/cli/setup_config.yml connector_name: OpenAI model_name: Embedding model api_key: test-api-key connector_role_prefix: test-role connector_secret_name: test-secretSample response:
Starting connector creation... Connector configuration loaded successfully from /Documents/cli/connector_config.yml Setup configuration loaded successfully from /Documents/cli/setup_config.yml Creating OpenAI connector... Step 1: Create Secret test secret exists, skipping creation. ---------- Step 2: Create IAM role configured in connector Role 'test-role-openai-connector-03ae00' does not exist. ---------- Step 3: Configure IAM role in OpenSearch Step 3.1: Create IAM role for Signing create connector request Role 'test-role-openai-connector-create-03ae00' does not exist. ---------- Step 3.2: Map IAM role test-role-openai-connector-create-03ae00 to OpenSearch permission role ---------- Step 4: Create connector in OpenSearch Waiting for resources to be ready... Time remaining: 1 seconds.... Wait completed, creating connector... Connector role arn: test-connector-role-arn ---------- Successfully created OpenAI connector with ID: connector123 Enter the path to save the output information, or press Enter to save it in the current directory [/Documents/cli/output.yml]: Output information saved successfully to /Documents/cli/output.yml
Connector configuration YAML file
You can use a connector configuration YAML file to automate the connector creation process. This file contains configuration parameters needed to create and configure the connector through the CLI.
Configuration file template
setup_config_path:
connector_name:
model_name:
access_token:
api_key:
aws_access_key:
aws_secret_access_key:
aws_session_token:
connector_body:
connector_role_inline_policy:
connector_role_prefix:
connector_secret_name:
endpoint_arn:
endpoint_url:
model_id:
project_id:
region:
required_policy:
required_secret:
Note: The order of the fields does not matter. This template will only be used when creating a connector using a configuration file.
Field descriptions
Field |
Description |
Required For |
|---|---|---|
setup_config_path |
Path to setup configuration YAML file |
All models |
connector_name |
Name of the connector (must match supported list) |
All models |
model_name |
Name of the model (must match supported list) |
All models except for custom connector in custom model service |
connector_role_prefix |
Connector role prefix |
AOS only |
connector_body |
Connector request body (JSON format) |
Custom models, custom connector in custom model service |
aws_access_key |
AWS access key |
Bedrock, Bedrock Converse, Comprehend, SageMaker, Textract (open-source only) |
aws_secret_access_key |
AWS secret access key |
Bedrock, Bedrock Converse, Comprehend, SageMaker, Textract (open-source only) |
aws_session_token |
AWS session token |
Bedrock, Bedrock Converse, Comprehend, SageMaker, Textract (open-source only) |
region |
AWS region |
Bedrock, Bedrock Converse, Comprehend, SageMaker, Textract |
api_key |
API key for the specified model |
Aleph Alpha, Azure OpenAI, Cohere, DeepSeek, OpenAI, when required_secret=true |
connector_secret_name |
Connector secret name |
Cohere, DeepSeek, OpenAI, when required_secret=true |
endpoint_arn |
Inference endpoint ARN |
SageMaker |
endpoint_url |
Inference endpoint URL |
SageMaker |
model_id |
Model ID |
Google Cloud Platform |
project_id |
Project ID |
Google Cloud Platform |
access_token |
Access token for the model |
Google Cloud Platform |
required_policy |
Set to ‘true’ to enable IAM inline policy configuration, ‘false’ to disable |
Custom connector in custom model service (AOS only) |
required_secret |
Set to ‘true’ to enable AWS Secrets Manager configuration, ‘false’ to disable |
Custom connector in custom model service (AOS only) |
connector_role_inline_policy |
IAM inline policy configuration (JSON format) |
When required_policy=true |
Output YAML file
After successfully creating a connector, the CLI saves important information about the connector in an output YAML file. This file contains details that may be needed for future operations or reference. The output YAML file appears similar to the following:
connector_create:
- connector_id: connector123
connector_name: OpenAI embedding model connector
connector_role_arn: test-connector-role-arn
connector_role_name: test-role-openai-connector-03ae00
connector_secret_arn: test-connector-secret-arn
connector_secret_name: test-secret
Supported connectors and models
Amazon OpenSearch Service
The create connector command supports the following models for OpenSearch Service.
Platform |
Model |
|---|---|
Amazon Bedrock |
AI21 Labs Jurassic-2 Mid |
Amazon Bedrock |
Anthropic Claude v2 |
Amazon Bedrock |
Anthropic Claude v3 |
Amazon Bedrock |
Anthropic Claude v3.7 |
Amazon Bedrock |
Cohere Embed Model v3 - English |
Amazon Bedrock |
Cohere Embed Model v3 - Multilingual |
Amazon Bedrock |
Titan Text Embedding |
Amazon Bedrock |
Titan Multimodal Embedding |
Amazon Bedrock Converse |
Anthropic Claude 3 Sonnet |
Amazon Comprehend |
Metadata embedding model |
Amazon SageMaker |
DeepSeek R1 model |
Amazon SageMaker |
Embedding model |
Amazon Textract |
Amazon Textract model |
Cohere |
Embedding model |
DeepSeek |
DeepSeek Chat model |
OpenAI |
Embedding model |
Self-managed OpenSearch
The create connector command supports the following models for self-managed OpenSearch.
Platform |
Model |
|---|---|
Aleph Alpha |
Luminous-Base embedding model |
Amazon Bedrock |
AI21 Labs Jurassic-2 Mid |
Amazon Bedrock |
Anthropic Claude v2 |
Amazon Bedrock |
Anthropic Claude v3 |
Amazon Bedrock |
Anthropic Claude v3.7 |
Amazon Bedrock |
Cohere Embed Model v3 - English |
Amazon Bedrock |
Cohere Embed Model v3 - Multilingual |
Amazon Bedrock |
Titan Text Embedding |
Amazon Bedrock |
Titan Multimodal Embedding |
Amazon Bedrock Converse |
Anthropic Claude 3 Sonnet |
Amazon Comprehend |
Metadata embedding model |
Amazon SageMaker |
Embedding model |
Amazon Textract |
Amazon Textract model |
Azure OpenAI |
Chat completion model |
Azure OpenAI |
Embedding model |
Cohere |
Chat model |
Cohere |
Embedding model |
Cohere |
Multi-modal embedding model |
DeepSeek |
DeepSeek Chat model |
Google Cloud Platform |
VertexAI embedding model |
OpenAI |
Chat model |
OpenAI |
Completion model |
OpenAI |
Embedding model |
Note: Custom connectors and models are supported for all connector types in both OpenSearch Service and self-managed OpenSearch.