Skip to content

configuration.md

Latest commit

 

History

History
100 lines (76 loc) · 3.26 KB

configuration.md

File metadata and controls

100 lines (76 loc) · 3.26 KB
Raw

Configuration Reference

Environment Variables (.env)

The app is configured via control-plane-app/.env. Copy .env.example to get started:

cp control-plane-app/.env.example control-plane-app/.env

Required

Variable Description Example
LAKEBASE_DNS Lakebase instance hostname instance-xxxx.database.cloud.databricks.com
LAKEBASE_DATABASE Lakebase database name control_plane
LAKEBASE_INSTANCE Lakebase instance name (for credential generation) ai-control-plane-db

Optional

Variable Description Default
DATABRICKS_HOST Workspace URL (auto-detected in Databricks Apps) Auto-detected
DATABRICKS_TOKEN PAT for local development Not needed in Apps
DATABRICKS_ACCOUNT_ID Account ID for cross-workspace features None
CORS_ORIGINS Comma-separated allowed origins (local dev only) Empty (disabled)

Workflow Configuration (databricks.yml)

The discovery workflows are configured via workflows/databricks.yml. Each target defines the environment:

Target Variables

Variable Description Example
catalog Unity Catalog name for Delta tables main
schema Schema name control_plane
lakebase_dns Lakebase hostname (same as .env) instance-xxxx.database.cloud.databricks.com
lakebase_instance Lakebase instance name ai-control-plane-db
warehouse_id SQL warehouse ID for system table queries xxxxxxxxxxxx
account_id Databricks account ID xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

Example Target

targets:
  dev:
    mode: development
    default: true
    variables:
      catalog: my_catalog
      schema: control_plane
      lakebase_dns: "instance-xxxx.database.cloud.databricks.com"
      lakebase_instance: "my-lakebase-instance"
      warehouse_id: "xxxxxxxxxxxx"
      account_id: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"

Workflow Schedule

By default, the discovery workflow runs every 30 minutes. To change:

resources:
  jobs:
    agent_discovery:
      schedule:
        quartz_cron_expression: "0 0/15 * * * ?"  # every 15 min

App Configuration (app.yaml)

app.yaml is auto-generated by deploy.sh from .env values. Do not edit manually.

Key settings:

  • Uvicorn server on port 8000
  • Lakebase connection via environment variables
  • OBO auth via the resources block with permission: "all-apis"

Finding Your Values

Lakebase DNS & Instance Name

  1. Go to SQL > Lakebase in your workspace
  2. Click your instance
  3. Instance name: shown at the top
  4. DNS: shown in the connection details

SQL Warehouse ID

  1. Go to SQL > SQL Warehouses
  2. Click your warehouse
  3. The ID is in the URL: .../sql/warehouses/<warehouse-id>

Account ID

  • Found in your workspace URL: https://<workspace>.cloud.databricks.com/?o=<org-id>
  • Or in the Databricks Account Console

App Service Principal ID

  • After creating the app, go to the app detail page
  • The SP application ID is listed under the app settings
  • Needed for granting system.mlflow access (see installation guide)