feat(source-asana): migrate to manifest-only with declarative OAuth#75638
feat(source-asana): migrate to manifest-only with declarative OAuth#75638Serhii Lazebnyi (lazebnyi) wants to merge 6 commits intomasterfrom
Conversation
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. 💡 Show Tips and TricksPR Slash CommandsAirbyte Maintainers (that's you!) can execute the following slash commands on your PR:
📚 Show Repo GuidanceHelpful Resources
|
|
|
Deploy preview for airbyte-docs ready! ✅ Preview Built with commit e611eba. |
|
/publish-connectors-prerelease
|
Aldo Gonzalez (aldogonzalez8)
left a comment
There was a problem hiding this comment.
APPROVED
07b2316 to
998dcea
Compare
|
/publish-connectors-prerelease
|
527b548 to
0d5de42
Compare
|
/publish-connectors-prerelease
|
…backward compatibility Co-Authored-By: gl_serhii.lazebnyi <serglazebny@gmail.com>
|
/publish-connectors-prerelease
|
…t in components.py Replace hard-coded open() with try/except Exception pattern with importlib.resources.files() for loading manifest.yaml. This follows coding standards (no broad exception catching) and uses the modern replacement for pkgutil.get_data. Co-Authored-By: gl_serhii.lazebnyi <serglazebny@gmail.com>
…t loading Replace importlib.resources.files() with pathlib.Path since source_declarative_manifest module doesn't exist in CI test environment. Falls back to local manifest.yaml path when Docker path is unavailable. Co-Authored-By: gl_serhii.lazebnyi <serglazebny@gmail.com>
…y runtime Co-Authored-By: gl_serhii.lazebnyi <serglazebny@gmail.com>
What
Migrates source-asana from a Python
YamlDeclarativeSourceconnector to a manifest-only connector, addsoauth_connector_input_specificationfor declarative OAuth support, and adds a declarative config migration to replace the deleted Pythonconfig_migration.py.Related: https://github.com/airbytehq/airbyte-internal-issues/issues/16023
How
source_asana/Python package (source.py,run.py,config_migration.py,__init__.py). Switched base image frompython-connector-basetosource-declarative-manifest:7.15.0. Updatedmetadata.yamltag tolanguage:manifest-only.oauth_connector_input_specificationtomanifest.yamlwith Asana consent/token URLs,authorization_codegrant type, andrefresh_tokenextraction.config_normalization_ruleswithConfigMigrationto handle the old{"access_token": "xxx"}format →{"credentials": {"option_title": "PAT Credentials", "personal_access_token": "xxx"}}.pkgutil.get_data("source_asana", ...)with apathlib.Path-based dual-path strategy, removed broadtry/except Exceptionin favor of guard statements, added defensive.get("properties", {}).Version bump: 1.5.1 → 1.5.2
Updates since last revision
Fixed
_get_stream_schemaincomponents.pywhich went through several iterations:open("/source_declarative_manifest/manifest.yaml")with broadtry/except Exception: return {}— violated Python coding standards.importlib.resources.files("source_declarative_manifest")— failed in CI withModuleNotFoundError(module doesn't exist outside Docker).Path(__file__).parent / "manifest.yaml"as fallback — failed in Docker runtime withNameErrorbecause manifest-only connectors loadcomponents.pyviaexec()where__file__is undefined.globals().get("__file__")guard to safely handle all three environments:/source_declarative_manifest/manifest.yamlexists → used directly__file__is defined → falls back to local path__file__is undefined → returns{}gracefully (same behavior as the oldtry/except)Review guide
manifest.yaml—oauth_connector_input_specification(lines ~2666-2678) andconfig_normalization_rules(lines ~2704-2732)components.py—_get_stream_schemadual-path manifest loading withglobals().get("__file__")guardmetadata.yaml— base image and tag changesItems for reviewer attention
globals().get("__file__")pattern: This is unconventional but necessary because manifest-only Docker images loadcomponents.pyviaexec()where__file__is not defined. When neither path resolves,opt_fieldswill be empty (same graceful degradation as the originaltry/except Exception). Please verify this is acceptable.config.get('access_token') and not config.get('credentials')) differ slightly from the old Python migration, which also checkedpersonal_access_token not in config. Verify against real legacy configs.get_request_paramssignature: The override inAsanaHttpRequesteruses positional args while the parentHttpRequesternow uses keyword-only args (*). This works at runtime but is a signature mismatch worth noting.test_basic_readfailure is a 401 Unauthorized from expired Asana OAuth credentials in GCP Secret Manager — not caused by this PR. All required checks pass.User Impact
AsanaOAuthFlow.access_tokenat root) will be automatically migrated.Can this PR be safely reverted and rolled back?
Link to Devin session: https://app.devin.ai/sessions/c135ab42decc46f9b389b9d45436a660
Requested by: Serhii Lazebnyi (@lazebnyi)