Support preserving unknown fields in ProtoBuf format#2860
Support preserving unknown fields in ProtoBuf format#2860xiaozhikang0916 wants to merge 14 commits intoKotlin:devfrom
Conversation
|
@sandwwraith Hi, please take a look here? |
|
@xiaozhikang0916 Sure, when I'll have time |
pdvrieze
left a comment
There was a problem hiding this comment.
The structure you use to represent protobuf content is somewhat complex, especially considering the protobuf wire format. You probably want to think a bit more how to represent protobuf data where the wire based information permits that. In particular you may want to distinguish primitives, "blobs", packed arrays. You may also allow users to observe a protobuf with provided metadata.
formats/protobuf/commonMain/src/kotlinx/serialization/protobuf/ProtoMessage.kt
Outdated
Show resolved
Hide resolved
|
@pdvrieze Actually I don't want to decode the protobuf wire data into a tree-structure type like what The raw content in protobuf wire data is quite ambiguous that, you can even not possible to tell whether a Base on it, |
|
Any progress about it? We have plan on upgrading Kotlin and serialization runtime version, and it would be great if we can have this functionality in near future. |
06df0a5 to
b4a7e01
Compare
docs/formats.md
Outdated
|
|
||
| Kotlin Serialization `ProtoBuf` format supports preserving unknown fields, as described in the [Protocol Buffer-Unknown Fields](https://protobuf.dev/programming-guides/proto3/#unknowns). | ||
|
|
||
| To keep the unknown fields, add a property in type `ProtoMessage` with default value `null` or `ProtoMessage.Empty`, and annotation `@ProtoUnknownFields` to your data class. |
There was a problem hiding this comment.
Should we be more strict here, eg use only nullable (null if there are no unknown fields) or non-null property (ProtoMessage.Empty if there are no unknown fields)?
formats/protobuf/commonMain/src/kotlinx/serialization/protobuf/internal/ProtobufDecoding.kt
Outdated
Show resolved
Hide resolved
formats/protobuf/commonMain/src/kotlinx/serialization/protobuf/internal/ProtobufDecoding.kt
Outdated
Show resolved
Hide resolved
formats/protobuf/commonMain/src/kotlinx/serialization/protobuf/internal/ProtobufDecoding.kt
Outdated
Show resolved
Hide resolved
formats/protobuf/commonMain/src/kotlinx/serialization/protobuf/ProtoMessage.kt
Outdated
Show resolved
Hide resolved
formats/protobuf/commonMain/src/kotlinx/serialization/protobuf/ProtoMessage.kt
Outdated
Show resolved
Hide resolved
formats/protobuf/commonMain/src/kotlinx/serialization/protobuf/ProtoMessage.kt
Outdated
Show resolved
Hide resolved
formats/protobuf/commonTest/src/kotlinx/serialization/protobuf/ProtobufUnknownFieldsTest.kt
Show resolved
Hide resolved
formats/protobuf/commonTest/src/kotlinx/serialization/protobuf/ProtobufUnknownFieldsTest.kt
Outdated
Show resolved
Hide resolved
Add tests for all protobuf wire types (varint, fixed32, fixed64, size-delimited)
655633a to
e4dfb20
Compare
|
LGTM, but I think @sandwwraith should also take a cursory look to check the style and documentation. |
Introducing new annotation
@ProtoUnknownFieldsand built-in typeProtoMessage, to store any fields with proto ids that are undefined in current message class.In my understanding, both ktx-serialization and protocol buffer are designed in strong typing, all messages used in code should be well defined in kt class or proto message. Therefore I would like to keep
ProtoMessageandProtoFieldminimun published. Not likeJsonObjectin json format, users cannot retrieve detailed data from it.Close #2655