Support for "Dynamic Type" Formats for @register_format

[allenc97]

Hi,

I currently use inline-snapshot for testing a GRPC service, where many test inputs and outputs are stored in external files due to size. The nature of protobuf makes the serde functions the same, albeit with different Message types. If I were to use @register_format, I would need to create a format for each type, which is less ideal compared to creating a "dynamic type" format that can resolve into any protobuf message the same way. This also extends to custom pydantic BaseModels or stdlib dataclasses.

I currently get around this with the following code:

@pytest.fixture
def payload_path(request: pytest.FixtureRequest) -> Path:
    test_file_path = Path(request.node.path).resolve()
    test_function_name = request.node.originalname

    payload_path_name: str = request.param

    payloads_path = test_file_path.parent / f".{test_file_path.stem}_payloads" / test_function_name
    return payloads_path / payload_path_name


T = typing.TypeVar("T")
P = typing.TypeVar("P", bound=Message)


@contextmanager
def payload_output(
    output_payload: ExternalFile, serialize: Callable, deserialize: Callable[[Any], T]
) -> Generator[T, None, None]:
    try:
        if output_payload._is_empty():
            raise ValueError("Snapshot not created")
        yield deserialize(get_snapshot_value(output_payload))
    except AssertionError:
        # (Re)generate snapshot on failed assertions
        assert output_payload == serialize()


@pytest.fixture
def protobuf_payload_output(
    payload_path: Path,
) -> Callable[[P], AbstractContextManager[P]]:
    @contextmanager
    def _ctx(response: P) -> Generator[P, None, None]:
        with payload_output(
            external_file(
                str(payload_path.with_name(f"{payload_path.stem}_output{payload_path.suffix}")),
                format=".txt",
            ),
            serialize=lambda: MessageToJson(response, preserving_proto_field_name=True),
            deserialize=lambda value: Parse(value, type(response)()),
        ) as output:
            yield output

    return _ctx
    
@pytest.mark.parametrize(
    "payload_path",
    [
        "1.json",
        "2.json",
    ],
    indirect=["payload_path"],
)
def test_get_user_payroll_setup(
    payload_path: Path,
    protobuf_payload_output: Callable[[Response], AbstractContextManager[Response]],
    servicer: Servicer,
):
    with payload_path.open() as f:
        payload = json_format.Parse(f.read(), Request())

    response = servicer.SomeServiceMethod(payload, None)

    with protobuf_payload_output(response) as response:
        assert response.some_field == expected.some_field

This could be possible via @register_format if registered formats have some way of taking kwargs, etc.

[15r10nk]

The problem is that the serialization function is determined by the suffix.

so you can store the type info in the suffix:

class MyFormat:
    def __init__(self,typ):
        self.suffix=f".{typ.__name__}"
        self.typ=typ

    def is_format_for(self,value):
        return isinstance(value,self.typ)

    def encode(self,value,path:Path):
        path.write_text(MessageToJson(value))

    def decode(self,path:Path):
        return Parse(path.read_text(),self.typ)


for typ in (TypeA,TypeB): # register all types
   register_format(MyFormat(typ))

or in the data:

@register_format
class DynamicFormat:
    suffix=".dyn"

    def is_format_for(self,value):
        # maybe you have some way to check this
        # you can also use:
        # assert value == external(".dyn")
        # in your code

    def encode(self,value,path:Path):
        typ=type(value)
        data=json.loads(MessageToJson(value))
        path.write_text(json.dumps({"type":typ.__qualname__,data:value}))

    def decode(self,path:Path):
        data=json.loads(path.read_text())
        typ_name=data["type"]
        typ=use_importlib_to_import_your_type_by_name(type_name)
        return Parse(json.dumps(data["data"]),typ)

I have not tested this code. Let me know when you have problems or questions.

[allenc97]

I see. So under this model we would create a suffix per type, which may or may not be independent of the path used to actually store the data. This should also work for my use case. Thanks!