Databricks integration - Kore.ai Docs

Back to list of integrations Use the Databricks integration to manage jobs and runs, execute SQL statements in a SQL warehouse, and call other Databricks REST API endpoints through Action Tools. It also provides a job notification webhook trigger.

Authentication	Actions	Triggers
OAuth 2.0 Client Credentials	(12)	(1)

Supported actions

Action	Use it to
List Jobs	List jobs, optionally filtered by name.
Get Job	Retrieve a job by its ID.
Run Job Now	Start a job run with optional parameters.
List Runs	List job runs with optional filters.
Get Run	Retrieve a run by its ID.
Cancel Run	Cancel a job run.
Get Run Output	Retrieve a run’s output.
List SQL Warehouses	List SQL warehouses available to the authenticated principal.
Execute SQL Statement	Execute a SQL statement in a warehouse.
Get SQL Statement	Retrieve statement execution details.
Cancel SQL Statement	Cancel a running SQL statement.
Custom API Call	Send a request to a Databricks REST API endpoint.

Triggers

Trigger	Fires when
Job Notification Webhook	A selected event occurs for the configured Databricks job: start, success, failure, duration warning threshold exceeded, or streaming backlog threshold exceeded. The trigger attaches a webhook notification destination to the job. It can create a destination or reuse one whose URL already points to the workflow’s trigger webhook. When disabled, the trigger removes its job attachment and deletes the destination only if it created it.

Prerequisites

A Databricks workspace and a service principal or OAuth client credentials with permission for the jobs, SQL, and API operations you plan to use.
A SQL warehouse for List SQL Warehouses and SQL statement actions.
For Job Notification Webhook, a job that the authenticated principal can edit. If reusing a notification destination, its URL must already be the workflow trigger’s webhook URL.

Set up the integration

Prepare Databricks OAuth credentials

Prepare the OAuth client ID and client secret, workspace URL, and required scopes for the Databricks APIs your tools will call. The token URL is prefilled as ${connectionConfig.workspaceHost}/oidc/v1/token; verify it resolves to the workspace token endpoint. See Databricks OAuth machine-to-machine authentication.

Create the auth profile

In Integrations, select Databricks > Connect. On Project Auth Profiles, select New Profile > Integration Profile > Databricks > Continue. Select OAuth 2.0 Client Credentials, enter the profile details, workspace Host, token URL, scopes, Client ID, and Client Secret, then select Create Profile. The Host is the Databricks workspace URL without a trailing slash. Studio also provides an optional refresh token URL field.

Create the action tool

In Tools, select New Tool > Action > Databricks > Continue. Select the Action and Auth Profile, configure the inputs, and select Create Tool. Then assign the tool to an agent.

Action reference

The inputs below match the fields shown in Studio for each action.

List Jobs

Parameter	Type	Required	Description
`limit`	number	No	Maximum number of jobs to return. Defaults to `25`; values above `100` are capped at `100`.
`offset`	number	No	Number of jobs to skip.
`name`	string	No	Filter jobs by name.

Get Job

Parameter	Type	Required	Description
`jobId`	number	Yes	ID of the job to retrieve.

Run Job Now

Parameter	Type	Required	Description
`jobId`	number	Yes	ID of the job to run.
`idempotencyToken`	string	No	Token used to make repeated run requests idempotent.
`parameters`	object	No	Job run parameters object. Defaults to `{}`. Use the parameter fields supported by the job, such as notebook, Python, JAR, or job parameters.

List Runs

Parameter	Type	Required	Description
`jobId`	number	No	Filter runs by job ID.
`activeOnly`	boolean	No	Return active runs only.
`completedOnly`	boolean	No	Return completed runs only.
`limit`	number	No	Maximum number of runs to return. Defaults to `25`; values above `100` are capped at `100`.
`offset`	number	No	Number of runs to skip.

Get Run

Parameter	Type	Required	Description
`runId`	number	Yes	ID of the run to retrieve.

Cancel Run

Parameter	Type	Required	Description
`runId`	number	Yes	ID of the run to cancel.

Get Run Output

Parameter	Type	Required	Description
`runId`	number	Yes	ID of the run whose output to retrieve.

List SQL Warehouses

This action has no input parameters.

Execute SQL Statement

Parameter	Type	Required	Description
`warehouseId`	string	Yes	ID of the SQL warehouse.
`statement`	string	Yes	SQL statement to execute.
`parameters`	object	No	Statement parameters array. Defaults to `[]`.
`waitTimeout`	string	No	How long to wait for completion. Defaults to `10s`.
`rowLimit`	number	No	Maximum rows to return. Defaults to `100`; values above `1000` are capped at `1000`.
`byteLimit`	number	No	Maximum response size in bytes. Defaults to `2097152` (2 MiB); values above `5242880` (5 MiB) are capped at `5242880`.
`disposition`	enum	No	Result delivery mode: `INLINE` (default, displayed as Inline) or `EXTERNAL_LINKS` (displayed as External Links).
`format`	enum	No	Result format: `JSON_ARRAY` (default, displayed as JSON Array), `ARROW_STREAM` (displayed as Arrow Stream), or `CSV`.

Get SQL Statement

Parameter	Type	Required	Description
`statementId`	string	Yes	ID of the SQL statement to retrieve.

Cancel SQL Statement

Parameter	Type	Required	Description
`statementId`	string	Yes	ID of the SQL statement to cancel.

Custom API Call

Parameter	Type	Required	Description
`method`	enum	Yes	HTTP method: `GET` (default), `POST`, `PATCH`, or `DELETE`.
`path`	string	Yes	Databricks API path beginning with `/api/`, such as `/api/2.2/jobs/list`.
`query`	object	No	Query string parameters. Defaults to `{}`.
`body`	object	No	Request body for non-GET methods. Defaults to `{}`; GET requests don’t use the body.

Trigger reference

Job Notification Webhook

Parameter	Type	Required	Description
`jobId`	number	Yes	ID of the job to monitor.
`events`	multi-select	Yes	Job events to receive. Defaults to `on_failure`. Options: `on_start`, `on_success`, `on_failure`, `on_duration_warning_threshold_exceeded`, and `on_streaming_backlog_exceeded`.
`destinationId`	string	No	Existing webhook notification destination to reuse. Its URL must already be the workflow trigger’s webhook URL. If omitted, the trigger creates a destination.
`displayName`	string	No	Display name for a newly created destination. Used only when creating one.

Troubleshoot common issues

Issue	Resolution
OAuth authorization or token exchange fails	Verify the workspace Host, client ID and secret, token URL, scopes, and service principal permissions.
A job or run action returns an access error	Confirm that the authenticated principal can view, run, or cancel the specified job or run.
SQL execution fails	Verify the warehouse ID, warehouse availability, SQL permissions, statement syntax, and parameter format.
SQL results are incomplete	Check the row and byte limits and the selected disposition and format.
The job notification trigger can’t be enabled	Verify that the principal can edit the job. For a reused destination, confirm its configured URL is the trigger webhook URL.
The trigger stops receiving notifications	Confirm the selected events and that the Databricks job still has the notification destination attached.

​Supported actions

​Triggers

​Prerequisites

​Set up the integration

​Action reference

​Trigger reference

​Troubleshoot common issues

​Related articles