Skip to content

Add schema registry support to ProtoBufMessageDecoder #18221

@xiangfu0

Description

@xiangfu0

Motivation

pinot-plugins/pinot-input-format/pinot-protobuf/src/main/java/org/apache/pinot/plugin/inputformat/protobuf/ProtoBufMessageDecoder.java still carries a top-level TODO for schema registry support.

Pinot already has the Confluent-specific protobuf ingestion module, but the base protobuf decoder still requires descriptor files and class names. Supporting schema-registry-backed descriptor resolution here would close a long-standing gap, or clarify why the base decoder should stay file-based only.

Scope

  • Decide whether the base decoder should support schema registry directly or explicitly defer to the confluent plugin.
  • If direct support is desired, define config and API behavior and add tests.
  • Otherwise, replace the TODO with clear documentation pointing users to the supported path.

Notes

Observed on upstream/master on April 15, 2026.

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew functionalityingestionRelated to data ingestion pipelinejavaPull requests that update Java codepluginsRelated to the plugin systempriority: lowNice to have, can waitserializationRelated to data serialization and deserialization

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions