Adopting Zod codec

[nebbles]

Hello! I was wondering about the stance of this project on Zod Codecs¹. I did a search but could not find them referenced in documentation/issues/discussions.

Codecs would provide a more standardised way for us HTTP API maintainers to define our schemas, in ways that are more portable outside of the express-zod-api framework (such as taking the schemas to the client).

Perfect basic use case for this is dates (but i reckon useful for more complex schemas). Rather than using ez.date we could define our codec for input/output. Here's the example of "Dealing with dates" from the documentation adapted to using zod codecs:

import { z } from "zod";
import { ez, defaultEndpointsFactory } from "express-zod-api";

// This is also provided by zod as a builtin: https://zod.dev/codecs?id=isodatetimetodate#isodatetimetodate
const isoDate = z.codec(
  z.iso.datetime(),
  z.date(),
  {
    decode: (s) => new Date(s),
    encode: (d) => d.toISOString(),
  }
);

const UserSchema = z.object({
  id: z.string(),
  birthday: isoDate,
  createdAt: isoDate,
});

const updateUserEndpoint = defaultEndpointsFactory.build({
  method: "post",
  input: UserSchema.omit({ createdAt: true }),
  output: UserSchema,
  handler: async ({ input }) => ({
    id: input.id,
    birthday: input.birthday,
    createdAt: new Date("2022-01-22"), // 2022-01-22T00:00:00.000Z
  }),
});

Existing documentation example

import { z } from "zod";
import { ez, defaultEndpointsFactory } from "express-zod-api";

const updateUserEndpoint = defaultEndpointsFactory.build({
  method: "post",
  input: z.object({
    userId: z.string(),
    birthday: ez.dateIn({ examples: ["1963-04-21"] }), // string -> Date in handler
  }),
  output: z.object({
    createdAt: ez.dateOut({ examples: ["2021-12-31"] }), // Date -> string in response
  }),
  handler: async ({ input }) => ({
    createdAt: new Date("2022-01-22"), // 2022-01-22T00:00:00.000Z
  }),
});

The change required, as I understand it, is that EZA would need to use the encode/decode of Zod instead of just parse. (Fortunately this is backwards compatible²)

Less fortunately, we can't use the z.invertCodec(schema) as this doesn't walk the object - so z.invertCodec(UserSchema) wouldn't have the desired outcome of inverting the full schema for compatability with EZA.

I currently don't have any good workarounds, as fundamentally codec are the only way of 'deduplicating schemas' to avoid maintaining separate input from output. We've had uncomfortable bad compromises such as trying to use date coercion etc

Footnotes

1. https://zod.dev/codecs ↩

2. https://zod.dev/codecs?id=composability#composability ↩

[RobinTail]

Hello @nebbles

Yes, I explored codecs. The state is that they are generally supported since they extend ZodPipe.

Codecs could take place somewhere in the generated Client. For instance, to establish the Date encoding/decoding for requests and in responses. Though, I didn't yet figure out how exactly it could be offered.

Perhaps the Codec's transforming functions could be placed in the generated client using ZodCodec._zod.def.transform.toString()

Within the API itself Codecs don't bring much value. The requirement to invert them are not more convenient that using a dedicated transforming schema (in my opinion).

If you have any ideas on what/how could/should be done, please let know!

[nebbles]

The reason codecs bring value in the API for us would be to use the same schema in API definition for both input and output.

Another use case came up just today for us (in addition to the date example noted above) where we'd like to have a different 'decoded' format in the code, from the 'encoded' JSON format sent over the wire. Here's a simplified example of it:

const ScopesSchema = z.codec(
  z.string(),
  z.array(z.string),
  {
    decode: (s) => s.split(" "),
    encode: (a) => a.join(" "),
  }
);

const SessionSchema = z.object({
  id: z.string(),
  scopes: ScopesSchema,
})

If under the hood, EZA called inputSchema.decode(input) and outputSchema.encode(output) instead of parse(), then as a library user I'd be able to defined codecs for my schemas and provide them to EZA input and output params of endpoint factory build. (This is my naive guess at how EZA is implemented!)

[RobinTail]

the framework can't really use decode on inputs because that method is strictly typed on the argument (does not accept unknown, which incoming data generally is), however, decode is effectively the same as parse in runtime, @nebbles :
https://zod.dev/codecs?id=type-safe-inputs#type-safe-inputs

[RobinTail]

I checked the strictly typed methods, @nebbles in #3421
And I'm glad to inform that they do actually still perform the argument validation, so in runtime they would really work fine. The only problem to solve is typing.

UPD: I found a way to solve this problem by doing z.unknown().pipe(inputSchema).decode(input) — this works great

[RobinTail]

Found another blocker with this proposition:

outputSchema.encode(output)

It won't work if output schema has .transform():

ZodEncodeError: Encountered unidirectional transform during encode: ZodTransform

So, it means that developers would have to migrate from transformations to codecs entirely, which is a breaking change, @nebbles

also, described in docs: https://zod.dev/codecs?id=transforms#transforms

[nebbles]

Ah yes. That's a pain. Taking a look at it, in a basic example, it seems Zod does make a distinction between a pipe and a codec, so that in a rudimentary case its possible to use outputSchema instanceof ZodCodec to conditionally run encode instead of parse on the output.

Example

import { z, ZodPipe, ZodCodec } from 'zod';

const say = console.info;

const inputScopeSchema = z.string().transform((s) => s.split(' '));
const outputScopeSchema = z
  .string()
  .array()
  .transform((a) => a.join(' '));

const scopeSchema = z.codec(z.string(), z.string().array(), {
  decode: (s) => s.split(' '),
  encode: (a) => a.join(' '),
});

const schema = z.object({
  name: z.string(),
  year: z.number().optional(),
});

const in_old = inputScopeSchema.parse('let me');
const in_new = scopeSchema.parse('foo bar');
say(in_old);
say(in_new);

say(outputScopeSchema);
say(scopeSchema);

say(outputScopeSchema instanceof ZodPipe);
say(scopeSchema instanceof ZodPipe);

say(outputScopeSchema instanceof ZodCodec);
say(scopeSchema instanceof ZodCodec);

const out_old = outputScopeSchema.encode(['let', 'me']);
const out_new = scopeSchema.encode(['let', 'me']);
say(out_old);
say(out_new);

Annoyingly this doesn't work with complex types, e.g. a plain zod object (pipe) which contains a nested codec. So isn't a viable approach to conditionally handling the encode() case.

I think then, if not possible without breaking change, my approach will need to be experimenting with 'adapting' my codec schema to a schema that EZA can just operate with parse()

I think something along these lines might work, but i am yet to properly unit test (only checked in a JS sandbox)

const scopeSchema = z.codec(z.string(), z.string().array(), {
  decode: (s) => s.split(' '),
  encode: (a) => a.join(' '),
});

const complexSchema = z.object({
  id: z.string(),
  scopes: scopeSchema,
});

type Input = z.input<typeof complexSchema>;
type Output = z.output<typeof complexSchema>;

const adapter = z.custom<Input>().transform((v) => complexSchema.encode(v));

// Pass 'adapter' as the output schema
// EZA can call `adapter.parse(outputData)` and it will *just work*