Skip to content

Schema discovery fails on incremental replications - Coupa #65

@iamlorax

Description

@iamlorax

Hello!

When making use of source_search_field and source_search_query to incrementally load a stream, the parameter substitution occurs after schema discovery. Thus those two fields are being passed as parameters during the schema discovery GET request: TapRestApiMsdk.get_schema. The result is an exception being thrown for REST APIs that do not allow unknown parameters (ex: Coupa). Example of params:

{'return_object': 'shallow', 'source_search_field': 'updated_at[gt]', 'source_search_query': '$last_run_date'}

Exception:

  File "...\tap_rest_api_msdk\tap.py", line 494, in discover_streams
    schema = self.get_schema(
             ^^^^^^^^^^^^^^^^
  File "...\tap_rest_api_msdk\tap.py", line 618, in get_schema
    raise ValueError(r.text)
ValueError: {"errors":{"request":["I don't understand this attribute: source_search_field. Please make sure it exists and is spelled correctly."]}}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions