Added support for Databricks Apps in DABs

[andrewnester]

Changes

Now it's possible to configure new app resource in bundle and point it to the custom source_code_path location where Databricks App code is defined.

On databricks bundle deploy DABs will create an app. All consecutive databricks bundle deploy execution will update an existing app if there are any updated

On databricks bundle run <my_app> DABs will execute app deployment. If the app is not started yet, it will start the app first.

Bundle configuration

bundle:
  name: apps

variables:
  my_job_id:
    description: "ID of job to run app"
    lookup:
      job: "My Job"
  databricks_name:
    description: "Name for app user"
  additional_flags:
    description: "Additional flags to run command app"
    default: ""
  my_app_config:
    type: complex
    description: "Configuration for my Databricks App"
    default:
      command:
        - flask
        - --app
        - hello
        - run
        - ${var.additional_flags}
      env:
        - name: DATABRICKS_NAME
          value: ${var.databricks_name}

resources:
  apps:
    my_app:
      name: "anester-app" # required and has to be unique
      description: "My App"
      source_code_path: ./app # required and points to location of app code
      config: ${var.my_app_config}
      resources:
        - name: "my-job"
          description: "A job for app to be able to run"
          job:
            id: ${var.my_job_id}
            permission: "CAN_MANAGE_RUN"
      permissions:
        - user_name: "foo@bar.com"
          level: "CAN_VIEW"
        - service_principal_name: "my_sp"
          level: "CAN_MANAGE"

targets:
  dev:
    variables:
      databricks_name: "Andrew (from dev)"
      additional_flags: --debug
  
  prod:
    variables:
      databricks_name: "Andrew (from prod)"

Execution

1. databricks bundle deploy -t dev
2. databricks bundle run my_app -t dev

If app is started

✓ Getting the status of the app my-app
✓ App is in RUNNING state
✓ Preparing source code for new app deployment.
✓ Deployment is pending
✓ Starting app with command: flask --app hello run --debug
✓ App started successfully
You can access the app at <app-url>

If app is not started

✓ Getting the status of the app my-app
✓ App is in UNAVAILABLE state
✓ Starting the app my-app
✓ App is starting...
....
✓ App is starting...
✓ App is started!
✓ Preparing source code for new app deployment.
✓ Downloading source code from /Workspace/Users/...
✓ Starting app with command: flask --app hello run --debug
✓ App started successfully
You can access the app at <app-url>

Tests

Added unit and config tests + manual test.

--- PASS: TestAccDeployBundleWithApp (404.59s)
PASS
coverage: 36.8% of statements in ./...
ok      github.com/databricks/cli/internal/bundle       405.035s        coverage: 36.8% of statements in ./...

[github-actions]

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

• PR number: 1928
• Commit SHA: 7693f0add0ab88bd49c109047cedc75cc16edd42

Checks will be approved automatically on success.

[pietern]

For later; this duplicates the entries in SupportedResources. Can it be de-duped?

[pietern]

Could you regenerate the OpenAPI annotations as well? That will populate these descriptions.

Can be in a separate PR.

[andrewnester]

@pietern will do anyway as part of Go SDK upgarde

[denik]

None of the other resources ever set ModifiedStatusUpdated, do we need this? @andrewnester

Are there plans to implement this status and use in IDE and Dabs-in-the-workspace? @ilia-db @ilyakuz-db @fjakobs

[ilyakuz-db]

We use modified_status in DABs in WS but it defaults to updated if not set