ai_chat_completion plugin (preview)

Learn how to use the ai_chat_completion plugin to chat with large language models, enabling AI-related scenarios such as RAG application and semantic search.

Prerequisites

Syntax

evaluate ai_chat_completion (Chat, ConnectionString [, Options [, IncludeErrorMessages]])

Parameters

NameTypeRequiredDescription
Chatdynamic✔️An array of messages comprising the conversation so far. The value can be a column reference or a constant scalar.
ConnectionStringstring✔️The connection string for the language model in the format <ModelDeploymentUri>;<AuthenticationMethod>; replace <ModelDeploymentUri> and <AuthenticationMethod> with the AI model deployment URI and the authentication method respectively.
OptionsdynamicThe options that control calls to the chat model endpoint. See Options.
IncludeErrorMessagesboolIndicates whether to output errors in a new column in the output table. Default value: false.

Options

The following table describes the options that control the way the requests are made to the chat model endpoint.

NameTypeDescription
RetriesOnThrottlingintSpecifies the number of retry attempts when throttling occurs. Default value: 0.
GlobalTimeouttimespanSpecifies the maximum time to wait for a response from the AI chat model. Default value: null.
ModelParametersdynamicParameters specific to the AI chat model. Possible values: temperature, top_p, stop, max_tokens, max_completion_tokens, presence_penalty, frequency_penalty, user, seed. Any other specified model parameters are ignored. Default value: null.

Configure Callout Policy

The azure_openai callout policy enables external calls to Azure AI services.

To configure the callout policy to authorize the AI model endpoint domain:

.alter-merge cluster policy callout
```
[
  {
    "CalloutType": "azure_openai",
    "CalloutUriRegex": "https://[A-Za-z0-9-]{3,63}\.(?:openai\\.azure\\.com|cognitiveservices\\.azure\\.com|cognitive\\.microsoft\\.com|services\\.ai\\.azure\\.com)(?:/.*)?",
    "CanCall": true
  }
]
```

Configure Managed Identity

When using managed identity to access Azure OpenAI Service, you must configure the Managed Identity policy to allow the system-assigned managed identity to authenticate to Azure OpenAI Service.

To configure the managed identity:

.alter-merge cluster policy managed_identity
```
[
  {
    "ObjectId": "system",
    "AllowedUsages": "AzureAI"
  }
]
```

Returns

Returns the following new chat completion columns:

  • A column with the _chat_completion suffix that contains the chat completion values.
  • If configured to return errors, a column with the _chat_completion_error suffix, which contains error strings or is left empty if the operation is successful.

Depending on the input type, the plugin returns different results:

  • Column reference: Returns one or more records with additional columns prefixed by the reference column name. For example, if the input column is named PromptData, the output columns are named PromptData_chat_completion and, if configured to return errors, PromptData_chat_completion_error.
  • Constant scalar: Returns a single record with additional columns that are not prefixed. The column names are _chat_completion and, if configured to return errors, _chat_completion_error.

Examples

The following example uses a system prompt to set the context for all subsequent chat messages in the input to the Azure OpenAI chat completion model.

Managed Identity

let connectionString = 'https://myaccount.openai.azure.com/openai/deployments/gpt4o/chat/completions?api-version=2024-06-01;managed_identity=system';
let messages = dynamic([{'role':'system', 'content': 'You are a KQL writing assistant'},{'role':'user', 'content': 'How can I restrict results to just 10 records?'}]);
evaluate ai_chat_completion(messages, connectionString);

Impersonation

let connectionString = 'https://myaccount.openai.azure.com/openai/deployments/gpt4o/chat/completions?api-version=2024-06-01;impersonate';
let messages = dynamic([{'role':'system', 'content': 'You are a KQL writing assistant'},{'role':'user', 'content': 'How can I restrict results to just 10 records?'}]);
evaluate ai_chat_completion(messages, connectionString);

let connectionString = 'https://myaccount.openai.azure.com/openai/deployments/gpt4o/chat/completions?api-version=2024-06-01;impersonate';
let messages = dynamic([{'role':'system', 'content': 'You are a KQL writing assistant'},{'role':'user', 'content': 'How can I restrict results to just 10 records?'}]);
evaluate ai_chat_completion(messages, connectionString);

Related content