Skip to content

Support schema evolution when using non-default constructors#360

Open
nicodeslandes wants to merge 8 commits intoch-robinson:mainfrom
nicodeslandes:record-init-v4
Open

Support schema evolution when using non-default constructors#360
nicodeslandes wants to merge 8 commits intoch-robinson:mainfrom
nicodeslandes:record-init-v4

Conversation

@nicodeslandes
Copy link
Copy Markdown
Contributor

@nicodeslandes nicodeslandes commented Nov 23, 2025

Support new fields in Schema when using non-default constructors

Description

This PR addresses #358 by making the record deserializer more tolerant when selecting constructors, which is critical for supporting schema evolution in immutable records.

The Problem

Previously, BinaryRecordDeserializerBuilder (and the JSON equivalent) required a constructor to match the Avro schema fields exactly. This meant that adding a new field to a schema (typically a backward-compatible change) would break deserialization for existing C# record types that hadn't yet been updated to include the new field. The deserializer would fail to find a matching constructor and throw an exception.
This is rather inconsistent with the way we handle new properties for classes that expose public properties and a default constructor

Example:

Old schema:

{
    "type": "record", 
    "name": "Person", 
    "fields": [
        {"name": "name", "type": "string"}
    ]
}

C# record / class:

public record PersonRecord(string Name);
public class PersonClass
{
    public required string Name {get; init; }
}

All good so far, we can happilly serialize/deserialize to/from either form of DTO

But when a new, backward-compatible field is added to the schema::

{
    "type": "record", 
    "name": "Person", 
    "fields": [
        {"name": "name", "type": "string"}, 
        {"name": "age", "type": "int", "default": 20}
    ]
}

PersonClass is still usable; we can generate both serializer and deserializer for it
But attempting to generate a deserializer for PersonRecord produces the following exception:

var deserializer = new BinaryDeserializerBuilder().BuildDelegate<PersonRecord>(newSchema);

💀💀💀 Exception:
Type 'TestProject1.PartialConstructorMatchingTests+PersonRecord' does not have a default constructor (Parameter 'type')
   at System.Linq.Expressions.Expression.New(Type type)
   at Chr.Avro.Serialization.BinaryRecordDeserializerBuilderCase.BuildExpression(Type type, Schema schema, BinaryDeserializerBuilderContext context)
   at Chr.Avro.Serialization.BinaryDeserializerBuilder.BuildExpression(Type type, Schema schema, BinaryDeserializerBuilderContext context)
   at Chr.Avro.Serialization.BinaryDeserializerBuilder.BuildDelegateExpression[T](Schema schema, BinaryDeserializerBuilderContext context)
   at Chr.Avro.Serialization.BinaryDeserializerBuilder.BuildDelegate[T](Schema schema, BinaryDeserializerBuilderContext context)

The Fix

The deserializer logic has been updated to select the "best match" constructor rather than requiring an exact match:

  1. It identifies constructors where all parameters can be satisfied (either by a matching schema field or a default value).
  2. It allows extra schema fields to be present. These extra fields are:
    • Mapped to writable properties if a match exists (Hybrid Construction).
    • Ignored if no match exists.

This behavior allows a client with an older version of a record (missing a new field) to successfully deserialize data from a newer schema version, ignoring the unknown fields.

Additional Improvements

  • Hybrid Construction: This change naturally supports mixing constructor initialization with property setters.
  • Default Values: Constructor parameters with default values are now correctly utilized if the corresponding field is missing from the schema.

Verification

New tests in PartialConstructorMatchTests verify:

  • Schema Evolution: Deserializing a schema with extra fields into a record that doesn't define them (fields are ignored).
  • Hybrid Construction: Populating some fields via constructor and others via properties.
  • Default Values: Using default parameter values when schema fields are missing.

@nicodeslandes nicodeslandes changed the title Support new fields in Schema when using non-default constructor Support schema evolution when using non-default constructors Nov 23, 2025
deserializers for record

The constructor candidates are sorted by best fit, and the top scorer is
selected

Add test for custom record deserializer to ensure backward compatibility
Fixed json deserialization of dynamic types
Handle case where no suitable constructor can be found to generate a deserializer
@nicodeslandes
Copy link
Copy Markdown
Contributor Author

I just rebased over the latest changes from main. This is now ready to be merged (at least as far as git is concerned :-) )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant