Plugins

• 1: Data reshaping plugins
• 1.1: bag_unpack plugin
• 1.2: narrow plugin
• 1.3: pivot plugin
• 2: General plugins
• 2.1: dcount_intersect plugin
• 2.2: infer_storage_schema plugin
• 2.3: infer_storage_schema_with_suggestions plugin
• 2.4: ipv4_lookup plugin
• 2.5: ipv6_lookup plugin
• 2.6: preview plugin
• 2.7: schema_merge plugin
• 3: Language plugins
• 3.1: Python plugin
• 3.2: Python plugin packages
• 3.3: R plugin (Preview)
• 4: Machine learning plugins
• 4.1: autocluster plugin
• 4.2: basket plugin
• 4.3: diffpatterns plugin
• 4.4: diffpatterns_text plugin
• 5: Query connectivity plugins
• 5.1: ai_embed_text plugin (Preview)
• 5.2: azure_digital_twins_query_request plugin
• 5.3: cosmosdb_sql_request plugin
• 5.4: http_request plugin
• 5.5: http_request_post plugin
• 5.6: mysql_request plugin
• 5.7: postgresql_request plugin
• 5.8: sql_request plugin
• 6: User and sequence analytics plugins
• 6.1: active_users_count plugin
• 6.2: activity_counts_metrics plugin
• 6.3: activity_engagement plugin
• 6.4: activity_metrics plugin
• 6.5: funnel_sequence plugin
• 6.6: funnel_sequence_completion plugin
• 6.7: new_activity_metrics plugin
• 6.8: rolling_percentile plugin
• 6.9: rows_near plugin
• 6.10: sequence_detect plugin
• 6.11: session_count plugin
• 6.12: sliding_window_counts plugin
• 6.13: User Analytics

1 - Data reshaping plugins

1.1 - bag_unpack plugin

The bag_unpack plugin unpacks a single column of type dynamic, by treating each property bag top-level slot as a column. The plugin is invoked with the evaluate operator.

Syntax

T | evaluate bag_unpack( Column [, OutputColumnPrefix ] [, columnsConflict ] [, ignoredProperties ] ) [: OutputSchema]

Parameters

Output schema syntax

( ColumnName : ColumnType [, …] )

To add all columns of the input table to the plugin output, use a wildcard * as the first parameter, as follows:

( * , ColumnName : ColumnType [, …] )

Returns

The bag_unpack plugin returns a table with as many records as its tabular input (T). The schema of the table is the same as the schema of its tabular input with the following modifications:

• The specified input column (Column) is removed.
• The schema is extended with as many columns as there are distinct slots in the top-level property bag values of T. The name of each column corresponds to the name of each slot, optionally prefixed by OutputColumnPrefix. Its type is either the type of the slot, if all values of the same slot have the same type, or dynamic, if the values differ in type.

Examples

Expand a bag

datatable(d:dynamic)
[
    dynamic({"Name": "John", "Age":20}),
    dynamic({"Name": "Dave", "Age":40}),
    dynamic({"Name": "Jasmine", "Age":30}),
]
| evaluate bag_unpack(d)

Output

Expand a bag with OutputColumnPrefix

Expand a bag and use the OutputColumnPrefix option to produce column names that begin with the prefix ‘Property_’.

datatable(d:dynamic)
[
    dynamic({"Name": "John", "Age":20}),
    dynamic({"Name": "Dave", "Age":40}),
    dynamic({"Name": "Jasmine", "Age":30}),
]
| evaluate bag_unpack(d, 'Property_')

Output

Expand a bag with columnsConflict

Expand a bag and use the columnsConflict option to resolve conflicts between existing columns and columns produced by the bag_unpack() operator.

datatable(Name:string, d:dynamic)
[
    'Old_name', dynamic({"Name": "John", "Age":20}),
    'Old_name', dynamic({"Name": "Dave", "Age":40}),
    'Old_name', dynamic({"Name": "Jasmine", "Age":30}),
]
| evaluate bag_unpack(d, columnsConflict='replace_source') // Use new name

Output

datatable(Name:string, d:dynamic)
[
    'Old_name', dynamic({"Name": "John", "Age":20}),
    'Old_name', dynamic({"Name": "Dave", "Age":40}),
    'Old_name', dynamic({"Name": "Jasmine", "Age":30}),
]
| evaluate bag_unpack(d, columnsConflict='keep_source') // Keep old name

Output

Expand a bag with ignoredProperties

Expand a bag and use the ignoredProperties option to ignore certain properties in the property bag.

datatable(d:dynamic)
[
    dynamic({"Name": "John", "Age":20, "Address": "Address-1" }),
    dynamic({"Name": "Dave", "Age":40, "Address": "Address-2"}),
    dynamic({"Name": "Jasmine", "Age":30, "Address": "Address-3"}),
]
// Ignore 'Age' and 'Address' properties
| evaluate bag_unpack(d, ignoredProperties=dynamic(['Address', 'Age']))

Output

Expand a bag with a query-defined OutputSchema

Expand a bag and use the OutputSchema option to allow various optimizations to be evaluated before running the actual query.

datatable(d:dynamic)
[
    dynamic({"Name": "John", "Age":20}),
    dynamic({"Name": "Dave", "Age":40}),
    dynamic({"Name": "Jasmine", "Age":30}),
]
| evaluate bag_unpack(d) : (Name:string, Age:long)

Output

Expand a bag and use the OutputSchema option to allow various optimizations to be evaluated before running the actual query. Use a wildcard * to return all columns of the input table.

datatable(d:dynamic, Description: string)
[
    dynamic({"Name": "John", "Age":20}), "Student",
    dynamic({"Name": "Dave", "Age":40}), "Teacher",
    dynamic({"Name": "Jasmine", "Age":30}), "Student",
]
| evaluate bag_unpack(d) : (*, Name:string, Age:long)

Output

1.2 - narrow plugin

The narrow plugin “unpivots” a wide table into a table with three columns:

• Row number
• Column type
• Column value (as string)

The narrow plugin is designed mainly for display purposes, as it allows wide tables to be displayed comfortably without the need of horizontal scrolling.

The plugin is invoked with the evaluate operator.

Syntax

T | evaluate narrow()

Examples

The following example shows an easy way to read the output of the Kusto .show diagnostics management command.

.show diagnostics
 | evaluate narrow()

The results of .show diagnostics itself is a table with a single row and 33 columns. By using the narrow plugin we “rotate” the output to something like this:

1.3 - pivot plugin

Rotates a table by turning the unique values from one column in the input table into multiple columns in the output table and performs aggregations as required on any remaining column values that will appear in the final output.

Syntax

T | evaluate pivot(pivotColumn[, aggregationFunction] [,column1 [,column2 … ]]) [: OutputSchema]

Parameters

Returns

Pivot returns the rotated table with specified columns (column1, column2, …) plus all unique values of the pivot columns. Each cell for the pivoted columns will contain the aggregate function computation.

Examples

Pivot by a column

For each EventType and State starting with ‘AL’, count the number of events of this type in this state.

StormEvents
| project State, EventType
| where State startswith "AL"
| where EventType has "Wind"
| evaluate pivot(State)

Output

Pivot by a column with aggregation function

For each EventType and State starting with ‘AR’, display the total number of direct deaths.

StormEvents
| where State startswith "AR"
| project State, EventType, DeathsDirect
| where DeathsDirect > 0
| evaluate pivot(State, sum(DeathsDirect))

Output

Pivot by a column with aggregation function and a single additional column

Result is identical to previous example.

StormEvents
| where State startswith "AR"
| project State, EventType, DeathsDirect
| where DeathsDirect > 0
| evaluate pivot(State, sum(DeathsDirect), EventType)

Output

Specify the pivoted column, aggregation function, and multiple additional columns

For each event type, source, and state, sum the number of direct deaths.

StormEvents
| where State startswith "AR"
| where DeathsDirect > 0
| evaluate pivot(State, sum(DeathsDirect), EventType, Source)

Output

Pivot with a query-defined output schema

The following example selects specific columns in the StormEvents table. It uses an explicit schema definition that allows various optimizations to be evaluated before running the actual query.

StormEvents
| project State, EventType
| where EventType has "Wind"
| evaluate pivot(State): (EventType:string, ALABAMA:long, ALASKA:long)

Output

2 - General plugins

2.1 - dcount_intersect plugin

Calculates intersection between N sets based on hll values (N in range of [2..16]), and returns N dcount values. The plugin is invoked with the evaluate operator.

Syntax

T | evaluate dcount_intersect(hll_1, hll_2, [, hll_3, …])

Parameters

Returns

Returns a table with N dcount values (per column, representing set intersections). Column names are s0, s1, … (until n-1).

Given sets S₁, S₂, .. S_n return values will be representing distinct counts of:
S₁,
S₁ ∩ S₂,
S₁ ∩ S₂ ∩ S₃,
… ,
S₁ ∩ S₂ ∩ … ∩ S_n

Examples

// Generate numbers from 1 to 100
range x from 1 to 100 step 1
| extend isEven = (x % 2 == 0), isMod3 = (x % 3 == 0), isMod5 = (x % 5 == 0)
// Calculate conditional HLL values (note that '0' is included in each of them as additional value, so we will subtract it later)
| summarize hll_even = hll(iif(isEven, x, 0), 2),
            hll_mod3 = hll(iif(isMod3, x, 0), 2),
            hll_mod5 = hll(iif(isMod5, x, 0), 2) 
// Invoke the plugin that calculates dcount intersections         
| evaluate dcount_intersect(hll_even, hll_mod3, hll_mod5)
| project evenNumbers = s0 - 1,             //                             100 / 2 = 50
          even_and_mod3 = s1 - 1,           // gcd(2,3) = 6, therefor:     100 / 6 = 16
          even_and_mod3_and_mod5 = s2 - 1   // gcd(2,3,5) is 30, therefore: 100 / 30 = 3 

Output

Related content

• dcount() (aggregation function)
• hll() (aggregation function)
• evaluate plugin operator

2.2 - infer_storage_schema plugin

This plugin infers the schema of external data, and returns it as CSL schema string. The string can be used when creating external tables. The plugin is invoked with the evaluate operator.

Authentication and authorization

In the properties of the request, you specify storage connection strings to access. Each storage connection string specifies the authorization method to use for access to the storage. Depending on the authorization method, the principal may need to be granted permissions on the external storage to perform the schema inference.

The following table lists the supported authentication methods and any required permissions by storage type.

Syntax

evaluate infer_storage_schema( Options )

Parameters

Supported properties of the request

Returns

The infer_storage_schema plugin returns a single result table containing a single row/column containing CSL schema string.

Example

let options = dynamic({
  'StorageContainers': [
    h@'https://storageaccount.blob.core.windows.net/MobileEvents;secretKey'
  ],
  'FileExtension': '.parquet',
  'FileNamePrefix': 'part-',
  'DataFormat': 'parquet'
});
evaluate infer_storage_schema(options)

Output

Use the returned schema in external table definition:

.create external table MobileEvents(
    app_id:string, user_id:long, event_time:datetime, country:string, city:string, device_type:string, device_vendor:string, ad_network:string, campaign:string, site_id:string, event_type:string, event_name:string, organic:string, days_from_install:int, revenue:real
)
kind=blob
partition by (dt:datetime = bin(event_time, 1d), app:string = app_id)
pathformat = ('app=' app '/dt=' datetime_pattern('yyyyMMdd', dt))
dataformat = parquet
(
    h@'https://storageaccount.blob.core.windows.net/MovileEvents;secretKey'
)

Related content

• infer_storage_schema_with_suggestions plugin

2.3 - infer_storage_schema_with_suggestions plugin

This infer_storage_schema_with_suggestions plugin infers the schema of external data and returns a JSON object. For each column, the object provides inferred type, a recommended type, and the recommended mapping transformation. The recommended type and mapping are provided by the suggestion logic that determines the optimal type using the following logic:

• Identity columns: If the inferred type for a column is long and the column name ends with id, the suggested type is string since it provides optimized indexing for identity columns where equality filters are common.
• Unix datetime columns: If the inferred type for a column is long and one of the unix-time to datetime mapping transformations produces a valid datetime value, the suggested type is datetime and the suggested ApplicableTransformationMapping mapping is the one that produced a valid datetime value.

The plugin is invoked with the evaluate operator. To obtain the table schema that uses the inferred schema for Create and alter Azure Storage external tables without suggestions, use the infer_storage_schema plugin.

Authentication and authorization

In the properties of the request, you specify storage connection strings to access. Each storage connection string specifies the authorization method to use for access to the storage. Depending on the authorization method, the principal may need to be granted permissions on the external storage to perform the schema inference.

The following table lists the supported authentication methods and any required permissions by storage type.

Syntax

evaluate infer_storage_schema_with_suggestions( Options )

Parameters

Supported properties of the request

Returns

The infer_storage_schema_with_suggestions plugin returns a single result table containing a single row/column containing a JSON string.

Example

let options = dynamic({
  'StorageContainers': [
    h@'https://storageaccount.blob.core.windows.net/MobileEvents;secretKey'
  ],
  'FileExtension': '.json',
  'FileNamePrefix': 'js-',
  'DataFormat': 'json'
});
evaluate infer_storage_schema_with_suggestions(options)

Example input data

    {
        "source": "DataExplorer",
        "created_at": "2022-04-10 15:47:57",
        "author_id": 739144091473215488,
        "time_millisec":1547083647000
    }

Output

{
  "Columns": [
    {
      "OriginalColumn": {
        "Name": "source",
        "CslType": {
          "type": "string",
          "IsNumeric": false,
          "IsSummable": false
        }
      },
      "RecommendedColumn": {
        "Name": "source",
        "CslType": {
          "type": "string",
          "IsNumeric": false,
          "IsSummable": false
        }
      },
      "ApplicableTransformationMapping": "None"
    },
    {
      "OriginalColumn": {
        "Name": "created_at",
        "CslType": {
          "type": "datetime",
          "IsNumeric": false,
          "IsSummable": true
        }
      },
      "RecommendedColumn": {
        "Name": "created_at",
        "CslType": {
          "type": "datetime",
          "IsNumeric": false,
          "IsSummable": true
        }
      },
      "ApplicableTransformationMapping": "None"
    },
    {
      "OriginalColumn": {
        "Name": "author_id",
        "CslType": {
          "type": "long",
          "IsNumeric": true,
          "IsSummable": true
        }
      },
      "RecommendedColumn": {
        "Name": "author_id",
        "CslType": {
          "type": "string",
          "IsNumeric": false,
          "IsSummable": false
        }
      },
      "ApplicableTransformationMapping": "None"
    },
    {
      "OriginalColumn": {
        "Name": "time_millisec",
        "CslType": {
          "type": "long",
          "IsNumeric": true,
          "IsSummable": true
        }
      },
      "RecommendedColumn": {
        "Name": "time_millisec",
        "CslType": {
          "type": "datetime",
          "IsNumeric": false,
          "IsSummable": true
        }
      },
      "ApplicableTransformationMapping": "DateTimeFromUnixMilliseconds"
    }
  ]
}

Related content

• infer_storage_schema plugin

2.4 - ipv4_lookup plugin

The ipv4_lookup plugin looks up an IPv4 value in a lookup table and returns rows with matched values. The plugin is invoked with the evaluate operator.

Syntax

T | evaluate ipv4_lookup( LookupTable , SourceIPv4Key , IPv4LookupKey [, ExtraKey1 [.. , ExtraKeyN [, return_unmatched ]]] )

Parameters

Returns

The ipv4_lookup plugin returns a result of join (lookup) based on IPv4 key. The schema of the table is the union of the source table and the lookup table, similar to the result of the lookup operator.

If the return_unmatched argument is set to true, the resulting table includes both matched and unmatched rows (filled with nulls).

If the return_unmatched argument is set to false, or omitted (the default value of false is used), the resulting table has as many records as matching results. This variant of lookup has better performance compared to return_unmatched=true execution.

Examples

IPv4 lookup - matching rows only

// IP lookup table: IP_Data
// Partial data from: https://raw.githubusercontent.com/datasets/geoip2-ipv4/master/data/geoip2-ipv4.csv
let IP_Data = datatable(network:string, continent_code:string ,continent_name:string, country_iso_code:string, country_name:string)
[
  "111.68.128.0/17","AS","Asia","JP","Japan",
  "5.8.0.0/19","EU","Europe","RU","Russia",
  "223.255.254.0/24","AS","Asia","SG","Singapore",
  "46.36.200.51/32","OC","Oceania","CK","Cook Islands",
  "2.20.183.0/24","EU","Europe","GB","United Kingdom",
];
let IPs = datatable(ip:string)
[
  '2.20.183.12',   // United Kingdom
  '5.8.1.2',       // Russia
  '192.165.12.17', // Unknown
];
IPs
| evaluate ipv4_lookup(IP_Data, ip, network)

Output

IPv4 lookup - return both matching and nonmatching rows

// IP lookup table: IP_Data
// Partial data from: 
// https://raw.githubusercontent.com/datasets/geoip2-ipv4/master/data/geoip2-ipv4.csv
let IP_Data = datatable(network:string,continent_code:string ,continent_name:string ,country_iso_code:string ,country_name:string )
[
    "111.68.128.0/17","AS","Asia","JP","Japan",
    "5.8.0.0/19","EU","Europe","RU","Russia",
    "223.255.254.0/24","AS","Asia","SG","Singapore",
    "46.36.200.51/32","OC","Oceania","CK","Cook Islands",
    "2.20.183.0/24","EU","Europe","GB","United Kingdom",
];
let IPs = datatable(ip:string)
[
    '2.20.183.12',   // United Kingdom
    '5.8.1.2',       // Russia
    '192.165.12.17', // Unknown
];
IPs
| evaluate ipv4_lookup(IP_Data, ip, network, return_unmatched = true)

Output

IPv4 lookup - using source in external_data()

let IP_Data = external_data(network:string,geoname_id:long,continent_code:string,continent_name:string ,country_iso_code:string,country_name:string,is_anonymous_proxy:bool,is_satellite_provider:bool)
    ['https://raw.githubusercontent.com/datasets/geoip2-ipv4/master/data/geoip2-ipv4.csv'];
let IPs = datatable(ip:string)
[
    '2.20.183.12',   // United Kingdom
    '5.8.1.2',       // Russia
    '192.165.12.17', // Sweden
];
IPs
| evaluate ipv4_lookup(IP_Data, ip, network, return_unmatched = true)

Output

IPv4 lookup - using extra columns for matching

let IP_Data = external_data(network:string,geoname_id:long,continent_code:string,continent_name:string ,country_iso_code:string,country_name:string,is_anonymous_proxy:bool,is_satellite_provider:bool)
    ['https://raw.githubusercontent.com/datasets/geoip2-ipv4/master/data/geoip2-ipv4.csv'];
let IPs = datatable(ip:string, continent_name:string, country_iso_code:string)
[
    '2.20.183.12',   'Europe', 'GB', // United Kingdom
    '5.8.1.2',       'Europe', 'RU', // Russia
    '192.165.12.17', 'Europe', '',   // Sweden is 'SE' - so it won't be matched
];
IPs
| evaluate ipv4_lookup(IP_Data, ip, network, continent_name, country_iso_code)

Output

Related content

• Overview of IPv4/IPv6 functions
• Overview of IPv4 text match functions

2.5 - ipv6_lookup plugin

The ipv6_lookup plugin looks up an IPv6 value in a lookup table and returns rows with matched values. The plugin is invoked with the evaluate operator.

Syntax

T | evaluate ipv6_lookup( LookupTable , SourceIPv6Key , IPv6LookupKey [, return_unmatched ] )

Parameters

Returns

The ipv6_lookup plugin returns a result of join (lookup) based on IPv6 key. The schema of the table is the union of the source table and the lookup table, similar to the result of the lookup operator.

If the return_unmatched argument is set to true, the resulting table includes both matched and unmatched rows (filled with nulls).

If the return_unmatched argument is set to false, or omitted (the default value of false is used), the resulting table has as many records as matching results. This variant of lookup has better performance compared to return_unmatched=true execution.

Examples

IPv6 lookup - matching rows only

// IP lookup table: IP_Data (the data is generated by ChatGPT).
let IP_Data = datatable(network:string, continent_code:string ,continent_name:string, country_iso_code:string, country_name:string)
[
  "2001:0db8:85a3::/48","NA","North America","US","United States",
  "2404:6800:4001::/48","AS","Asia","JP","Japan",
  "2a00:1450:4001::/48","EU","Europe","DE","Germany",
  "2800:3f0:4001::/48","SA","South America","BR","Brazil",
  "2c0f:fb50:4001::/48","AF","Africa","ZA","South Africa",
  "2607:f8b0:4001::/48","NA","North America","CA","Canada",
  "2a02:26f0:4001::/48","EU","Europe","FR","France",
  "2400:cb00:4001::/48","AS","Asia","IN","India",
  "2801:0db8:85a3::/48","SA","South America","AR","Argentina",
  "2a03:2880:4001::/48","EU","Europe","GB","United Kingdom"
];
let IPs = datatable(ip:string)
[
  "2001:0db8:85a3:0000:0000:8a2e:0370:7334", // United States
  "2404:6800:4001:0001:0000:8a2e:0370:7334", // Japan
  "2a02:26f0:4001:0006:0000:8a2e:0370:7334", // France
  "a5e:f127:8a9d:146d:e102:b5d3:c755:abcd",  // N/A
  "a5e:f127:8a9d:146d:e102:b5d3:c755:abce"   // N/A
];
IPs
| evaluate ipv6_lookup(IP_Data, ip, network)

Output

IPv6 lookup - return both matching and nonmatching rows

// IP lookup table: IP_Data (the data is generated by ChatGPT).
let IP_Data = datatable(network:string, continent_code:string ,continent_name:string, country_iso_code:string, country_name:string)
[
  "2001:0db8:85a3::/48","NA","North America","US","United States",
  "2404:6800:4001::/48","AS","Asia","JP","Japan",
  "2a00:1450:4001::/48","EU","Europe","DE","Germany",
  "2800:3f0:4001::/48","SA","South America","BR","Brazil",
  "2c0f:fb50:4001::/48","AF","Africa","ZA","South Africa",
  "2607:f8b0:4001::/48","NA","North America","CA","Canada",
  "2a02:26f0:4001::/48","EU","Europe","FR","France",
  "2400:cb00:4001::/48","AS","Asia","IN","India",
  "2801:0db8:85a3::/48","SA","South America","AR","Argentina",
  "2a03:2880:4001::/48","EU","Europe","GB","United Kingdom"
];
let IPs = datatable(ip:string)
[
  "2001:0db8:85a3:0000:0000:8a2e:0370:7334", // United States
  "2404:6800:4001:0001:0000:8a2e:0370:7334", // Japan
  "2a02:26f0:4001:0006:0000:8a2e:0370:7334", // France
  "a5e:f127:8a9d:146d:e102:b5d3:c755:abcd",  // N/A
  "a5e:f127:8a9d:146d:e102:b5d3:c755:abce"   // N/A
];
IPs
| evaluate ipv6_lookup(IP_Data, ip, network, true)

Output

Related content

• Overview of IPv6/IPv6 functions

2.6 - preview plugin

Returns a table with up to the specified number of rows from the input record set, and the total number of records in the input record set.

Syntax

T | evaluate preview(NumberOfRows)

Parameters

Returns

The preview plugin returns two result tables:

• A table with up to the specified number of rows. For example, the sample query above is equivalent to running T | take 50.
• A table with a single row/column, holding the number of records in the input record set. For example, the sample query above is equivalent to running T | count.

Example

StormEvents | evaluate preview(5)

Table1

The following output table only includes the first 6 columns. To see the full result, run the query.

|StartTime|EndTime|EpisodeId|EventId|State|EventType|…| |–|–|–| |2007-12-30T16:00:00Z|2007-12-30T16:05:00Z|11749|64588|GEORGIA| Thunderstorm Wind|…| |2007-12-20T07:50:00Z|2007-12-20T07:53:00Z|12554|68796|MISSISSIPPI| Thunderstorm Wind|…| |2007-09-29T08:11:00Z|2007-09-29T08:11:00Z|11091|61032|ATLANTIC SOUTH| Waterspout|…| |2007-09-20T21:57:00Z|2007-09-20T22:05:00Z|11078|60913|FLORIDA| Tornado|…| |2007-09-18T20:00:00Z|2007-09-19T18:00:00Z|11074|60904|FLORIDA| Heavy Rain|…|

Table2

2.7 - schema_merge plugin

Merges tabular schema definitions into a unified schema.

Schema definitions are expected to be in the format produced by the getschema operator.

The schema merge operation joins columns in input schemas and tries to reduce data types to common ones. If data types can’t be reduced, an error is displayed on the problematic column.

The plugin is invoked with the evaluate operator.

Syntax

T | evaluate schema_merge(PreserveOrder)

Parameters

Returns

The schema_merge plugin returns output similar to what getschema operator returns.

Examples

Merge with a schema that has a new column appended.

let schema1 = datatable(Uri:string, HttpStatus:int)[] | getschema;
let schema2 = datatable(Uri:string, HttpStatus:int, Referrer:string)[] | getschema;
union schema1, schema2 | evaluate schema_merge()

Output

Merge with a schema that has different column ordering (HttpStatus ordinal changes from 1 to 2 in the new variant).

let schema1 = datatable(Uri:string, HttpStatus:int)[] | getschema;
let schema2 = datatable(Uri:string, Referrer:string, HttpStatus:int)[] | getschema;
union schema1, schema2 | evaluate schema_merge()

Output

Merge with a schema that has different column ordering, but with PreserveOrder set to false.

let schema1 = datatable(Uri:string, HttpStatus:int)[] | getschema;
let schema2 = datatable(Uri:string, Referrer:string, HttpStatus:int)[] | getschema;
union schema1, schema2 | evaluate schema_merge(PreserveOrder = false)

Output

3 - Language plugins

3.1 - Python plugin

3.2 - Python plugin packages

This article lists the available Python packages in the Python plugin. For more information, see Python plugin.

3.11.7 (Preview)

Python engine 3.11.7 + common data science and ML packages

3.11.7 DL (Preview)

Python engine 3.11.7 + common data science and ML packages + deep learning packages (tensorflow & torch)

3.10.8

Python engine 3.10.8 + common data science and ML packages

3.10.8 DL

Not supported

3.6.5 (Legacy)

Not supported

This article lists the available managed Python packages in the Python plugin. For more information, see Python plugin.

To create a custom image, see Create a custom image.

3.11.7 (Preview)

Python engine 3.11.7 + common data science and ML packages

3.11.7 DL (Preview)

Python engine 3.11.7 + common data science and ML packages + deep learning packages (tensorflow & torch)

3.10.8

Python engine 3.10.8 + common data science and ML packages

3.10.8 DL

Python engine 3.10.8 + common data science and ML packages + deep learning packages (tensorflow & torch)

3.6.5 (Legacy)

3.3 - R plugin (Preview)

The R plugin runs a user-defined function (UDF) using an R script.

The script gets tabular data as its input, and produces tabular output. The plugin’s runtime is hosted in a sandbox on the cluster’s nodes. The sandbox provides an isolated and secure environment.

Syntax

T | evaluate [hint.distribution = (single | per_node)] r(output_schema, script [, script_parameters] [, external_artifacts])

Parameters

Reserved R variables

The following variables are reserved for interaction between Kusto Query Language and the R code:

• df: The input tabular data (the values of T above), as an R DataFrame.
• kargs: The value of the script_parameters argument, as an R dictionary.
• result: An R DataFrame created by the R script. The value becomes the tabular data that gets sent to any Kusto query operator that follows the plugin.

Enable the plugin

• The plugin is disabled by default.
• Enable or disable the plugin in the Azure portal in the Configuration tab of your cluster. For more information, see Manage language extensions in your Azure Data Explorer cluster (Preview)

R sandbox image

• The R sandbox image is based on R 3.4.4 for Windows, and includes packages from Anaconda’s R Essentials bundle.

Examples

range x from 1 to 360 step 1
| evaluate r(
//
typeof(*, fx:double),               //  Output schema: append a new fx column to original table 
//
'result <- df\n'                    //  The R decorated script
'n <- nrow(df)\n'
'g <- kargs$gain\n'
'f <- kargs$cycles\n'
'result$fx <- g * sin(df$x / n * 2 * pi * f)'
//
, bag_pack('gain', 100, 'cycles', 4)    //  dictionary of parameters
)
| render linechart 

Performance tips

• Reduce the plugin’s input dataset to the minimum amount required (columns/rows).

• Use filters on the source dataset using the Kusto Query Language, when possible.

• To make a calculation on a subset of the source columns, project only those columns before invoking the plugin.

• Use hint.distribution = per_node whenever the logic in your script is distributable.

• You can also use the partition operator for partitioning the input data et.

• Whenever possible, use the Kusto Query Language to implement the logic of your R script.

  For example:

  .show operations
  | where StartedOn > ago(1d) // Filtering out irrelevant records before invoking the plugin
  | project d_seconds = Duration / 1s // Projecting only a subset of the necessary columns
  | evaluate hint.distribution = per_node r( // Using per_node distribution, as the script's logic allows it
      typeof(*, d2:double),
      'result <- df\n'
      'result$d2 <- df$d_seconds\n' // Negative example: this logic should have been written using Kusto's query language
    )
  | summarize avg = avg(d2)
  

Usage tips

• To avoid conflicts between Kusto string delimiters and R string delimiters:

  • Use single quote characters (') for Kusto string literals in Kusto queries.
  • Use double quote characters (") for R string literals in R scripts.

• Use the external data operator to obtain the content of a script that you’ve stored in an external location, such as Azure blob storage or a public GitHub repository.

  For example:

  let script = 
      externaldata(script:string)
      [h'https://kustoscriptsamples.blob.core.windows.net/samples/R/sample_script.r']
      with(format = raw);
  range x from 1 to 360 step 1
  | evaluate r(
      typeof(*, fx:double),
      toscalar(script), 
      bag_pack('gain', 100, 'cycles', 4))
  | render linechart 
  

Install packages for the R plugin

Follow these step by step instructions to install package(s) that aren’t included in the plugin’s base image.

Prerequisites

1. Create a blob container to host the packages, preferably in the same place as your cluster. For example, https://artifactswestus.blob.core.windows.net/r, assuming your cluster is in West US.

2. Alter the cluster’s callout policy to allow access to that location.

   • This change requires AllDatabasesAdmin permissions.

   • For example, to enable access to a blob located in https://artifactswestus.blob.core.windows.net/r, run the following command:

   .alter-merge cluster policy callout @'[ { "CalloutType": "sandbox_artifacts", "CalloutUriRegex": "artifactswestus\\.blob\\.core\\.windows\\.net/r/","CanCall": true } ]'
   

Install packages

The example snips below assume local R machine on Windows environment.

1. Verify you’re using the appropriate R version – current R Sandbox version is 3.4.4:

   > R.Version()["version.string"]
   
   $version.string
   [1] "R version 3.4.4 (2018-03-15)"
   

   If needed you can download it from here.

2. Launch the x64 RGui

3. Create a new empty folder to be populated with all the relevant packages you would like to install. In this example we install the brglm2 package, so creating “C:\brglm2”.

4. Add the newly created folder path to lib paths:

   > .libPaths("C://brglm2")
   

5. Verify that the new folder is now the first path in .libPaths():

   > .libPaths()
   
   [1] "C:/brglm2"    "C:/Program Files/R/R-3.4.4/library"
   

6. Once this setup is done, any package that we install shall be added to this new folder. Let’s install the requested package and its dependencies:

   > install.packages("brglm2")
   

   In case the question “Do you want to install from sources the packages which need compilation?” pops up, answer “Y”.

7. Verify that new folders were added to “C:\brglm2”:

   

8. Select all items in that folder and zip them to e.g. libs.zip (do not zip the parent folder). You should get an archive structure like this:

   libs.zip:

   • brglm2 (folder)
   • enrichwith (folder)
   • numDeriv (folder)

9. Upload libs.zip to the blob container that was set above

10. Call the r plugin.

    • Specify the external_artifacts parameter with a property bag of name and reference to the ZIP file (the blob’s URL, including a SAS token).
    • In your inline r code, import zipfile from sandboxutils and call its install() method with the name of the ZIP file.

Example

Install the brglm2 package:

print x=1
| evaluate r(typeof(*, ver:string),
    'library(sandboxutils)\n'
    'zipfile.install("brglm2.zip")\n'
    'library("brglm2")\n'
    'result <- df\n'
    'result$ver <-packageVersion("brglm2")\n'
    ,external_artifacts=bag_pack(brglm2.zip', 'https://artifactswestus.blob.core.windows.net/r/libs.zip?*** REPLACE WITH YOUR SAS TOKEN ***'))

Make sure that the archive’s name (first value in pack pair) has the *.zip suffix to prevent collisions when unzipping folders whose name is identical to the archive name.

4 - Machine learning plugins

4.1 - autocluster plugin

autocluster finds common patterns of discrete attributes (dimensions) in the data. It then reduces the results of the original query, whether it’s 100 or 100,000 rows, to a few patterns. The plugin was developed to help analyze failures (such as exceptions or crashes) but can potentially work on any filtered dataset. The plugin is invoked with the evaluate operator.

Syntax

T | evaluate autocluster ([SizeWeight [, WeightColumn [, NumSeeds [, CustomWildcard [, … ]]]]])

Parameters

The parameters must be ordered as specified in the syntax. To indicate that the default value should be used, put the string tilde value ~. For more information, see Examples.

Returns

The autocluster plugin usually returns a small set of patterns. The patterns capture portions of the data with shared common values across multiple discrete attributes. Each pattern in the results is represented by a row.

The first column is the segment ID. The next two columns are the count and percentage of rows out of the original query that are captured by the pattern. The remaining columns are from the original query. Their value is either a specific value from the column, or a wildcard value (which are by default null) meaning variable values.

The patterns aren’t distinct, may be overlapping, and usually don’t cover all the original rows. Some rows may not fall under any pattern.

Examples

Using evaluate

T | evaluate autocluster()

Using autocluster

StormEvents
| where monthofyear(StartTime) == 5
| extend Damage = iff(DamageCrops + DamageProperty > 0 , "YES" , "NO")
| project State , EventType , Damage
| evaluate autocluster(0.6)

Output

Using custom wildcards

StormEvents
| where monthofyear(StartTime) == 5
| extend Damage = iff(DamageCrops + DamageProperty > 0 , "YES" , "NO")
| project State , EventType , Damage
| evaluate autocluster(0.2, '~', '~', '*')

Output

Related content

• basket
• reduce

4.2 - basket plugin

The basket plugin finds frequent patterns of attributes in the data and returns the patterns that pass a frequency threshold in that data. A pattern represents a subset of the rows that have the same value across one or more columns. The basket plugin is based on the Apriori algorithm originally developed for basket analysis data mining.

Syntax

T | evaluate basket ([ Threshold, WeightColumn, MaxDimensions, CustomWildcard, CustomWildcard, … ])

Parameters

Returns

The basket plugin returns frequent patterns that pass a ratio threshold. The default threshold is 0.05.

Each pattern is represented by a row in the results. The first column is the segment ID. The next two columns are the count and percentage of rows, from the original query that match the pattern. The remaining columns relate to the original query, with either a specific value from the column or a wildcard value, which is by default null, meaning a variable value.

Example

StormEvents
| where monthofyear(StartTime) == 5
| extend Damage = iff(DamageCrops + DamageProperty > 0 , "YES" , "NO")
| project State, EventType, Damage, DamageCrops
| evaluate basket(0.2)

Output

Example with custom wildcards

StormEvents
| where monthofyear(StartTime) == 5
| extend Damage = iff(DamageCrops + DamageProperty > 0 , "YES" , "NO")
| project State, EventType, Damage, DamageCrops
| evaluate basket(0.2, '~', '~', '*', int(-1))

Output

4.3 - diffpatterns plugin

Compares two datasets of the same structure and finds patterns of discrete attributes (dimensions) that characterize differences between the two datasets. The plugin is invoked with the evaluate operator.

diffpatterns was developed to help analyze failures (for example, by comparing failures to non-failures in a given time frame), but can potentially find differences between any two datasets of the same structure.

Syntax

T | evaluate diffpatterns(SplitColumn, SplitValueA, SplitValueB [, WeightColumn,Threshold, MaxDimensions, CustomWildcard, …])

Parameters

Returns

diffpatterns returns a small set of patterns that capture different portions of the data in the two sets (that is, a pattern capturing a large percentage of the rows in the first dataset and low percentage of the rows in the second set). Each pattern is represented by a row in the results.

The result of diffpatterns returns the following columns:

• SegmentId: the identity assigned to the pattern in the current query (note: IDs aren’t guaranteed to be the same in repeating queries).

• CountA: the number of rows captured by the pattern in Set A (Set A is the equivalent of where tostring(splitColumn) == SplitValueA).

• CountB: the number of rows captured by the pattern in Set B (Set B is the equivalent of where tostring(splitColumn) == SplitValueB).

• PercentA: the percentage of rows in Set A captured by the pattern (100.0 * CountA / count(SetA)).

• PercentB: the percentage of rows in Set B captured by the pattern (100.0 * CountB / count(SetB)).

• PercentDiffAB: the absolute percentage point difference between A and B (|PercentA - PercentB|) is the main measure of significance of patterns in describing the difference between the two sets.

• Rest of the columns: are the original schema of the input and describe the pattern, each row (pattern) represents the intersection of the non-wildcard values of the columns (equivalent of where col1==val1 and col2==val2 and ... colN=valN for each non-wildcard value in the row).

For each pattern, columns that aren’t set in the pattern (that is, without restriction on a specific value) will contain a wildcard value, which is null by default. See in the Arguments section below how wildcards can be manually changed.

• Note: the patterns are often not distinct. They may be overlapping, and usually don’t cover all the original rows. Some rows may not fall under any pattern.

Example

StormEvents 
| where monthofyear(StartTime) == 5
| extend Damage = iff(DamageCrops + DamageProperty > 0 , 1 , 0)
| project State , EventType , Source , Damage, DamageCrops
| evaluate diffpatterns(Damage, "0", "1" )

Output

4.4 - diffpatterns_text plugin

Compares two datasets of string values and finds text patterns that characterize differences between the two datasets. The plugin is invoked with the evaluate operator.

The diffpatterns_text returns a set of text patterns that capture different portions of the data in the two sets. For example, a pattern capturing a large percentage of the rows when the condition is true and low percentage of the rows when the condition is false. The patterns are built from consecutive tokens separated by white space, with a token from the text column or a * representing a wildcard. Each pattern is represented by a row in the results.

Syntax

T | evaluate diffpatterns_text(TextColumn, BooleanCondition [, MinTokens, Threshold , MaxTokens])

Parameters

Returns

The result of diffpatterns_text returns the following columns:

• Count_of_True: The number of rows matching the pattern when the condition is true.
• Count_of_False: The number of rows matching the pattern when the condition is false.
• Percent_of_True: The percentage of rows matching the pattern from the rows when the condition is true.
• Percent_of_False: The percentage of rows matching the pattern from the rows when the condition is false.
• Pattern: The text pattern containing tokens from the text string and ‘*’ for wildcards.

Example

The following example uses data from the StormEvents table in the help cluster. To access this data, sign in to https://dataexplorer.azure.com/clusters/help/databases/Samples. In the left menu, browse to help > Samples > Tables > Storm_Events.

The examples in this tutorial use the StormEvents table, which is publicly available in the Weather analytics sample data.

StormEvents     
| where EventNarrative != "" and monthofyear(StartTime) > 1 and monthofyear(StartTime) < 9
| where EventType == "Drought" or EventType == "Extreme Cold/Wind Chill"
| evaluate diffpatterns_text(EpisodeNarrative, EventType == "Extreme Cold/Wind Chill", 2)

Output

5 - Query connectivity plugins

5.1 - ai_embed_text plugin (Preview)

The ai_embed_text plugin allows embedding of text using language models, enabling various AI-related scenarios such as Retrieval Augmented Generation (RAG) applications and semantic search. The plugin supports Azure OpenAI Service embedding models accessed using managed identity.

Prerequisites

• An Azure OpenAI Service configured with managed identity
• Managed identity and callout policies configured to allow communication with Azure OpenAI services

Syntax

evaluate ai_embed_text (text, connectionString [, options [, IncludeErrorMessages]])

Parameters

Options

The following table describes the options that control the way the requests are made to the embedding model endpoint.

Configure managed identity and callout policies

To use the ai_embed_text plugin, you must configure the following policies:

• managed identity: Allow the system-assigned managed identity to authenticate to Azure OpenAI services.
• callout: Authorize the AI model endpoint domain.

To configure these policies, use the commands in the following steps:

1. Configure the managed identity:

   .alter-merge cluster policy managed_identity
   ```
   [
     {
       "ObjectId": "system",
       "AllowedUsages": "AzureAI"
     }
   ]
   ```
   

2. Configure the callout policy:

   .alter-merge cluster policy callout
   ```
   [
       {
           "CalloutType": "azure_openai",
           "CalloutUriRegex": "https://[A-Za-z0-9\\-]{3,63}\\.openai\\.azure\\.com/.*",
           "CanCall": true
       }
   ]
   ```
   

Returns

Returns the following new embedding columns:

• A column with the _embedding suffix that contains the embedding values
• If configured to return errors, a column with the _embedding_error suffix, which contains error strings or is left empty if the operation is successful.

Depending on the input type, the plugin returns different results:

• Column reference: Returns one or more records with additional columns are prefixed by the reference column name. For example, if the input column is named TextData, the output columns are named TextData_embedding and, if configured to return errors, TextData_embedding_error.
• Constant scalar: Returns a single record with additional columns that are not prefixed. The column names are _embedding and, if configured to return errors, _embedding_error.

Examples

The following example embeds the text Embed this text using AI using the Azure OpenAI Embedding model.

let expression = 'Embed this text using AI';
let connectionString = 'https://myaccount.openai.azure.com/openai/deployments/text-embedding-3-small/embeddings?api-version=2024-06-01;managed_identity=system';
evaluate ai_embed_text(expression, connectionString)

The following example embeds multiple texts using the Azure OpenAI Embedding model.

let connectionString = 'https://myaccount.openai.azure.com/openai/deployments/text-embedding-3-small/embeddings?api-version=2024-06-01;managed_identity=system';
let options = dynamic({
    "RecordsPerRequest": 10,
    "CharsPerRequest": 10000,
    "RetriesOnThrottling": 1,
    "GlobalTimeout": 2m
});
datatable(TextData: string)
[
    "First text to embed",
    "Second text to embed",
    "Third text to embed"
]
| evaluate ai_embed_text(TextData, connectionString, options , true)

Best practices

Azure OpenAI embedding models are subject to heavy throttling, and frequent calls to this plugin can quickly reach throttling limits.

To efficiently use the ai_embed_text plugin while minimizing throttling and costs, follow these best practices:

• Control request size: Adjust the number of records (RecordsPerRequest) and characters per request (CharsPerRequest).
• Control query timeout: Set GlobalTimeout to a value lower than the query timeout to ensure progress isn’t lost on successful calls up to that point.
• Handle rate limits more gracefully: Set retries on throttling (RetriesOnThrottling).

Related content

• series_cosine_similarity()

5.2 - azure_digital_twins_query_request plugin

The azure_digital_twins_query_request plugin runs an Azure Digital Twins query as part of a Kusto Query Language (KQL) query. The plugin is invoked with the evaluate operator.

Using the plugin, you can query across data in both Azure Digital Twins and any data source accessible through KQL. For example, you can perform time series analytics.

For more information about the plugin, see Azure Digital Twins query plugin.

Syntax

evaluate azure_digital_twins_query_request ( AdtInstanceEndpoint , AdtQuery )

Parameters

Authentication and authorization

The azure_digital_twins_query_request plugin uses the Microsoft Entra account of the user running the query to authenticate. To run a query, a user must at least be granted the Azure Digital Twins Data Reader role. Information on how to assign this role can be found in Security for Azure Digital Twins solutions.

Examples

The following examples show how you can run various Azure Digital Twins queries, including queries that use additional Kusto expressions.

Retrieval of all twins within an Azure Digital Twins instance

The following example returns all digital twins within an Azure Digital Twins instance.

evaluate azure_digital_twins_query_request(
  'https://contoso.api.wcus.digitaltwins.azure.net',
  'SELECT T AS Twins FROM DIGITALTWINS T')

Projection of twin properties as columns along with additional Kusto expressions

The following example returns the result from the plugin as separate columns, and then performs additional operations using Kusto expressions.

evaluate azure_digital_twins_query_request(
  'https://contoso.api.wcus.digitaltwins.azure.net',
  'SELECT T.Temperature, T.Humidity FROM DIGITALTWINS T WHERE IS_PRIMITIVE(T.Temperature) AND IS_PRIMITIVE(T.Humidity)')
| where Temperature > 20
| project TemperatureInC = Temperature, Humidity

Output

Perform time series analytics

You can use the data history integration feature of Azure Digital Twins to historize digital twin property updates. To learn how to view the historized twin updates, see View the historized twin updates

5.3 - cosmosdb_sql_request plugin

The cosmosdb_sql_request plugin sends a SQL query to an Azure Cosmos DB SQL network endpoint and returns the results of the query. This plugin is primarily designed for querying small datasets, for example, enriching data with reference data stored in Azure Cosmos DB. The plugin is invoked with the evaluate operator.

Syntax

evaluate cosmosdb_sql_request ( ConnectionString , SqlQuery [, SqlParameters [, Options]] ) [: OutputSchema]

Parameters

Supported options

The following table describes the supported fields of the Options parameter.

Authentication and authorization

To authorize to an Azure Cosmos DB SQL network endpoint, you need to specify the authorization information. The following table provides the supported authentication methods and the description for how to use that method.

Set callout policy

The plugin makes callouts to the Azure Cosmos DB instance. Make sure that the cluster’s callout policy enables calls of type cosmosdb to the target CosmosDbUri.

The following example shows how to define the callout policy for Azure Cosmos DB. It’s recommended to restrict it to specific endpoints (my_endpoint1, my_endpoint2).

[
  {
    "CalloutType": "CosmosDB",
    "CalloutUriRegex": "my_endpoint1\\.documents\\.azure\\.com",
    "CanCall": true
  },
  {
    "CalloutType": "CosmosDB",
    "CalloutUriRegex": "my_endpoint2\\.documents\\.azure\\.com",
    "CanCall": true
  }
]

The following example shows an alter callout policy command for cosmosdb CalloutType

.alter cluster policy callout @'[{"CalloutType": "cosmosdb", "CalloutUriRegex": "\\.documents\\.azure\\.com", "CanCall": true}]'

Examples

The following examples use placeholder text, in brackets.

Query Azure Cosmos DB with a query-defined output schema

The following example uses the cosmosdb_sql_request plugin to send a SQL query while selecting only specific columns. This query uses explicit schema definitions that allow various optimizations before the actual query is run against Cosmos DB.

evaluate cosmosdb_sql_request(
  'AccountEndpoint=https://cosmosdbacc.documents.azure.com/;Database=<MyDatabase>;Collection=<MyCollection>;AccountKey='h'<AccountKey>',
  'SELECT c.Id, c.Name from c') : (Id:long, Name:string) 

Query Azure Cosmos DB

The following example uses the cosmosdb_sql_request plugin to send a SQL query to fetch data from Azure Cosmos DB using its Azure Cosmos DB for NoSQL.

evaluate cosmosdb_sql_request(
  'AccountEndpoint=https://cosmosdbacc.documents.azure.com/;Database=<MyDatabase>;Collection=<MyCollection>;AccountKey='h'<AccountKey>',
  'SELECT * from c') // OutputSchema is unknown, so it is not specified. This may harm the performance of the query.

Query Azure Cosmos DB with parameters

The following example uses SQL query parameters and queries the data from an alternate region. For more information, see preferredLocations.

evaluate cosmosdb_sql_request(
   'AccountEndpoint=https://cosmosdbacc.documents.azure.com/;Database=<MyDatabase>;Collection=<MyCollection>;AccountKey='h'<AccountKey>',
    "SELECT c.id, c.lastName, @param0 as Column0 FROM c WHERE c.dob >= '1970-01-01T00:00:00Z'",
    dynamic({'@param0': datetime(2019-04-16 16:47:26.7423305)}),
    dynamic({'preferredLocations': ['East US']})) : (Id:long, Name:string, Column0: datetime) 
| where lastName == 'Smith'

Query Azure Cosmos DB and join data with a database table

The following example joins partner data from an Azure Cosmos DB with partner data in a database using the Partner field. It results in a list of partners with their phone numbers, website, and contact email address sorted by partner name.

evaluate cosmosdb_sql_request(
    'AccountEndpoint=https://cosmosdbacc.documents.azure.com/;Database=<MyDatabase>;Collection=<MyCollection>;AccountKey='h'<AccountKey>',
    "SELECT c.id, c.Partner, c. phoneNumber FROM c') : (Id:long, Partner:string, phoneNumber:string) 
| join kind=innerunique Partner on Partner
| project id, Partner, phoneNumber, website, Contact
| sort by Partner

Query Azure Cosmos DB using token authentication

The following example joins partner data from an Azure Cosmos DB with partner data in a database using the Partner field. It results in a list of partners with their phone numbers, website, and contact email address sorted by partner name.

evaluate cosmosdb_sql_request(
    'AccountEndpoint=https://cosmosdbacc.documents.azure.com/;Database=<MyDatabase>;Collection=<MyCollection>;',
    "SELECT c.Id, c.Name, c.City FROM c",
    dynamic(null),
    dynamic({'token': h'abc123...'})
) : (Id:long, Name:string, City:string)

Query Azure Cosmos DB using Azure Resource Manager resource ID for authentication

The following example uses the Azure Resource Manager resource ID for authentication and the Microsoft Entra token of the requesting principal, since a token isn’t specified. It sends a SQL query while selecting only specific columns and specifies explicit schema definitions.

evaluate cosmosdb_sql_request(
    'AccountEndpoint=https://cosmosdbacc.documents.azure.com/;Database=<MyDatabase>;Collection=<MyCollection>;',
    "SELECT c.Id, c.Name, c.City FROM c",
    dynamic({'armResourceId': '/subscriptions/<SubscriptionId>/resourceGroups/<ResourceGroup>/providers/Microsoft.DocumentDb/databaseAccounts/<DatabaseAccount>'})
) : (Id:long, Name:string, City:string)

5.4 - http_request plugin

services: data-explorer

http_request plugin

The http_request plugin sends an HTTP GET request and converts the response into a table.

Prerequisites

• Run .enable plugin http_request to enable the plugin
• Set the URI to access as an allowed destination for webapi in the Callout policy

Syntax

evaluate http_request ( Uri [, RequestHeaders [, Options]] )

Parameters

Authentication and authorization

To authenticate, use the HTTP standard Authorization header or any custom header supported by the web service.

Returns

The plugin returns a table that has a single record with the following dynamic columns:

• ResponseHeaders: A property bag with the response header.
• ResponseBody: The response body parsed as a value of type dynamic.

If the HTTP response indicates (via the Content-Type response header) that the media type is application/json, the response body is automatically parsed as-if it’s a JSON object. Otherwise, it’s returned as-is.

Headers

The RequestHeaders argument can be used to add custom headers to the outgoing HTTP request. In addition to the standard HTTP request headers and the user-provided custom headers, the plugin also adds the following custom headers:

Example

The following example retrieves Azure retails prices for Azure Purview in west Europe:

let Uri = "https://prices.azure.com/api/retail/prices?$filter=serviceName eq 'Azure Purview' and location eq 'EU West'";
evaluate http_request(Uri)
| project ResponseBody.Items
| mv-expand ResponseBody_Items
| evaluate bag_unpack(ResponseBody_Items)

Output