Skip to content

feat(source-asana): migrate to manifest-only with declarative OAuth#75638

Open
Serhii Lazebnyi (lazebnyi) wants to merge 6 commits intomasterfrom
slazebnyi/source-asana-declarative-oauth
Open

feat(source-asana): migrate to manifest-only with declarative OAuth#75638
Serhii Lazebnyi (lazebnyi) wants to merge 6 commits intomasterfrom
slazebnyi/source-asana-declarative-oauth

Conversation

@lazebnyi
Copy link
Copy Markdown
Contributor

@lazebnyi Serhii Lazebnyi (lazebnyi) commented Mar 30, 2026

What

Migrates source-asana from a Python YamlDeclarativeSource connector to a manifest-only connector, adds oauth_connector_input_specification for declarative OAuth support, and adds a declarative config migration to replace the deleted Python config_migration.py.

Related: https://github.com/airbytehq/airbyte-internal-issues/issues/16023

How

  • Manifest-only migration: Removed source_asana/ Python package (source.py, run.py, config_migration.py, __init__.py). Switched base image from python-connector-base to source-declarative-manifest:7.15.0. Updated metadata.yaml tag to language:manifest-only.
  • Declarative OAuth: Added oauth_connector_input_specification to manifest.yaml with Asana consent/token URLs, authorization_code grant type, and refresh_token extraction.
  • Config migration: Added config_normalization_rules with ConfigMigration to handle the old {"access_token": "xxx"} format → {"credentials": {"option_title": "PAT Credentials", "personal_access_token": "xxx"}}.
  • components.py updates: Moved to connector root, replaced pkgutil.get_data("source_asana", ...) with a pathlib.Path-based dual-path strategy, removed broad try/except Exception in favor of guard statements, added defensive .get("properties", {}).

Version bump: 1.5.1 → 1.5.2

Updates since last revision

Fixed _get_stream_schema in components.py which went through several iterations:

  1. Initial state: Used open("/source_declarative_manifest/manifest.yaml") with broad try/except Exception: return {} — violated Python coding standards.
  2. First fix attempt: Used importlib.resources.files("source_declarative_manifest") — failed in CI with ModuleNotFoundError (module doesn't exist outside Docker).
  3. Second fix attempt: Used Path(__file__).parent / "manifest.yaml" as fallback — failed in Docker runtime with NameError because manifest-only connectors load components.py via exec() where __file__ is undefined.
  4. Final fix: Uses globals().get("__file__") guard to safely handle all three environments:
    • Docker runtime: /source_declarative_manifest/manifest.yaml exists → used directly
    • CI with module import: __file__ is defined → falls back to local path
    • CI with exec loading: __file__ is undefined → returns {} gracefully (same behavior as the old try/except)

Review guide

  1. manifest.yamloauth_connector_input_specification (lines ~2666-2678) and config_normalization_rules (lines ~2704-2732)
  2. components.py_get_stream_schema dual-path manifest loading with globals().get("__file__") guard
  3. metadata.yaml — base image and tag changes

Items for reviewer attention

  • globals().get("__file__") pattern: This is unconventional but necessary because manifest-only Docker images load components.py via exec() where __file__ is not defined. When neither path resolves, opt_fields will be empty (same graceful degradation as the original try/except Exception). Please verify this is acceptable.
  • Config migration conditions: The Jinja conditions (config.get('access_token') and not config.get('credentials')) differ slightly from the old Python migration, which also checked personal_access_token not in config. Verify against real legacy configs.
  • get_request_params signature: The override in AsanaHttpRequester uses positional args while the parent HttpRequester now uses keyword-only args (*). This works at runtime but is a signature mismatch worth noting.
  • CI test failure: The test_basic_read failure is a 401 Unauthorized from expired Asana OAuth credentials in GCP Secret Manager — not caused by this PR. All required checks pass.

User Impact

  • Users authenticating via OAuth will now use the declarative OAuth flow instead of the platform-hardcoded AsanaOAuthFlow.
  • Users with legacy configs (access_token at root) will be automatically migrated.
  • No breaking changes for existing PAT or OAuth users.

Can this PR be safely reverted and rolled back?

  • YES 💚

Link to Devin session: https://app.devin.ai/sessions/c135ab42decc46f9b389b9d45436a660
Requested by: Serhii Lazebnyi (@lazebnyi)

@github-actions
Copy link
Copy Markdown
Contributor

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

PR Slash Commands

Airbyte Maintainers (that's you!) can execute the following slash commands on your PR:

  • 🛠️ Quick Fixes
    • /format-fix - Fixes most formatting issues.
    • /bump-version - Bumps connector versions, scraping changelog description from the PR title.
  • ❇️ AI Testing and Review (internal link: AI-SDLC Docs):
    • /ai-prove-fix - Runs prerelease readiness checks, including testing against customer connections.
    • /ai-canary-prerelease - Rolls out prerelease to 5-10 connections for canary testing.
    • /ai-review - AI-powered PR review for connector safety and quality gates.
  • 🚀 Connector Releases:
    • /publish-connectors-prerelease - Publishes pre-release connector builds (tagged as {version}-preview.{git-sha}) for all modified connectors in the PR.
    • /bump-progressive-rollout-version - Bumps connector version with an RC suffix (2.16.10-rc.1) for progressive rollouts (enableProgressiveRollout: true).
      • Example: /bump-progressive-rollout-version changelog="Add new feature for progressive rollout"
  • ☕️ JVM connectors:
    • /update-connector-cdk-version connector=<CONNECTOR_NAME> - Updates the specified connector to the latest CDK version.
      Example: /update-connector-cdk-version connector=destination-bigquery
  • 🐍 Python connectors:
    • /poe connector source-example lock - Run the Poe lock task on the source-example connector, committing the results back to the branch.
    • /poe source example lock - Alias for /poe connector source-example lock.
    • /poe source example use-cdk-branch my/branch - Pin the source-example CDK reference to the branch name specified.
    • /poe source example use-cdk-latest - Update the source-example CDK dependency to the latest available version.
  • ⚙️ Admin commands:
    • /force-merge reason="<REASON>" - Force merges the PR using admin privileges, bypassing CI checks. Requires a reason.
      Example: /force-merge reason="CI is flaky, tests pass locally"
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 30, 2026

source-asana Connector Test Results

14 tests   9 ✅  59s ⏱️
 1 suites  4 💤
 1 files    1 ❌

For more details on these failures, see this check.

Results for commit e611eba.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 30, 2026

Deploy preview for airbyte-docs ready!

✅ Preview
https://airbyte-docs-nvnm2rmft-airbyte-growth.vercel.app

Built with commit e611eba.
This pull request is being automatically deployed with vercel-action

@lazebnyi Serhii Lazebnyi (lazebnyi) marked this pull request as ready for review March 30, 2026 23:41
@lazebnyi
Copy link
Copy Markdown
Contributor Author

Serhii Lazebnyi (lazebnyi) commented Mar 30, 2026

/publish-connectors-prerelease

Pre-release Connector Publish Started

Publishing pre-release build for connector source-asana.
PR: #75638

Pre-release versions will be tagged as {version}-preview.e8e82e3
and are available for version pinning via the scoped_configuration API.

View workflow run
Pre-release Publish: SUCCESS

Docker image (pre-release):
airbyte/source-asana:1.5.2-preview.e8e82e3

Docker Hub: https://hub.docker.com/layers/airbyte/source-asana/1.5.2-preview.e8e82e3

Registry JSON:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVED

@lazebnyi Serhii Lazebnyi (lazebnyi) force-pushed the slazebnyi/source-asana-declarative-oauth branch from 07b2316 to 998dcea Compare April 3, 2026 00:31
@lazebnyi
Copy link
Copy Markdown
Contributor Author

Serhii Lazebnyi (lazebnyi) commented Apr 3, 2026

/publish-connectors-prerelease

Pre-release Connector Publish Started

Publishing pre-release build for connector source-asana.
PR: #75638

Pre-release versions will be tagged as {version}-preview.52555ce
and are available for version pinning via the scoped_configuration API.

View workflow run
Pre-release Publish: SUCCESS

Docker image (pre-release):
airbyte/source-asana:1.5.2-preview.52555ce

Docker Hub: https://hub.docker.com/layers/airbyte/source-asana/1.5.2-preview.52555ce

Registry JSON:

@lazebnyi Serhii Lazebnyi (lazebnyi) force-pushed the slazebnyi/source-asana-declarative-oauth branch from 527b548 to 0d5de42 Compare April 3, 2026 03:51
@lazebnyi
Copy link
Copy Markdown
Contributor Author

Serhii Lazebnyi (lazebnyi) commented Apr 3, 2026

/publish-connectors-prerelease

Pre-release Connector Publish Started

Publishing pre-release build for connector source-asana.
PR: #75638

Pre-release versions will be tagged as {version}-preview.0d5de42
and are available for version pinning via the scoped_configuration API.

View workflow run
Pre-release Publish: SUCCESS

Docker image (pre-release):
airbyte/source-asana:1.5.2-preview.0d5de42

Docker Hub: https://hub.docker.com/layers/airbyte/source-asana/1.5.2-preview.0d5de42

Registry JSON:

@lazebnyi
Copy link
Copy Markdown
Contributor Author

Serhii Lazebnyi (lazebnyi) commented Apr 3, 2026

/publish-connectors-prerelease

Pre-release Connector Publish Started

Publishing pre-release build for connector source-asana.
PR: #75638

Pre-release versions will be tagged as {version}-preview.608e48f
and are available for version pinning via the scoped_configuration API.

View workflow run
Pre-release Publish: SUCCESS

Docker image (pre-release):
airbyte/source-asana:1.5.2-preview.608e48f

Docker Hub: https://hub.docker.com/layers/airbyte/source-asana/1.5.2-preview.608e48f

Registry JSON:

…t in components.py

Replace hard-coded open() with try/except Exception pattern with
importlib.resources.files() for loading manifest.yaml. This follows
coding standards (no broad exception catching) and uses the modern
replacement for pkgutil.get_data.

Co-Authored-By: gl_serhii.lazebnyi <serglazebny@gmail.com>
@devin-ai-integration devin-ai-integration bot changed the title feat(source-asana): add oauth_connector_input_specification for declarative OAuth feat(source-asana): migrate to manifest-only with declarative OAuth Apr 3, 2026
devin-ai-integration bot and others added 2 commits April 3, 2026 11:10
…t loading

Replace importlib.resources.files() with pathlib.Path since
source_declarative_manifest module doesn't exist in CI test environment.
Falls back to local manifest.yaml path when Docker path is unavailable.

Co-Authored-By: gl_serhii.lazebnyi <serglazebny@gmail.com>
…y runtime

Co-Authored-By: gl_serhii.lazebnyi <serglazebny@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants