feat(ai-agents): support custom structured output schemas for AI Agent responses

[asastre]

Description

Allow the AI Agent to return custom structured output message types beyond the built-in ones (text, carousel, etc.). This enables the Flow Builder plugin to instruct the AI Agent to respond with a specific Flow Builder content ID, which is then resolved into its full content with all follow-ups.

Context

Currently, the AI Agent can only return a fixed set of message types. There are scenarios where the AI Agent needs to respond by referencing existing Flow Builder content (e.g. a specific node in the flow) instead of generating a free-form response. This PR makes the output schema extensible so that each consumer can inject its own message types.

Approach taken / Explain the design

1. Generic core interfaces (botonic-core)

• Made BaseMessage, OutputMessage, AgenticOutputMessage, RunResult, and InferenceResponse generic with an Extra type parameter, so they can carry additional message types while remaining backward-compatible (defaults to never).
• Added outputMessagesSchemas to AiAgentArgs to pass extra Zod schemas to the agent.

2. Dynamic output schema (botonic-plugin-ai-agents)

• Extracted base message schemas into a constant array and introduced getOutputSchema(externalSchemas) that merges them with any externally provided Zod schemas at runtime.
• AIAgentBuilder now receives and forwards outputMessagesSchemas.

3. Flow Builder content message (botonic-plugin-flow-builder)

• Created FlowBuilderContentMessage type and its corresponding FlowBuilderContentSchema Zod object, representing a message with a contentId pointing to a Flow Builder node.
• Updated getContentsByAiAgent to:
  • Pass FlowBuilderContentSchema to the AI Agent plugin.
  • Separate the AI Agent response into regular messages and flowBuilderContent messages.
  • Resolve each flowBuilderContent message by fetching the referenced node from the CMS and returning its full contents.

To document / Usage example

To add a custom output message type, define a Zod schema and pass it in outputMessagesSchemas:

const MyCustomSchema = z.object({
  type: z.enum(['myCustomType']),
  someField: z.string(),
})

// When calling the AI Agent plugin
aiAgentArgs.outputMessagesSchemas = [MyCustomSchema]

The AI Agent will then be able to return messages of that type alongside the built-in ones.

As for the new FlowBuilderContentSchema, in the AI Agent prompt you could do something like this:
"If the user is asking for the status of an order, display the Flow Builder content 'OPEN_WISMO_WEBVIEW'"

This will result in the AI Agent returning as a response the contents of the Flow Builder content ID OPEN_WISMO_WEBVIEW directly.

Testing

The pull request has unit tests.

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[github-actions]

Test Results

42 tests   42 ✅  1m 21s ⏱️
 5 suites   0 💤
 1 files     0 ❌

Results for commit 88a52ee.

[Iru89]

I think it would be simpler to declare the ZOD schema within the plugin-ai-agent, just like the other schemas. You’d save yourself all the logic in the plugin-flow-builder when declaring the schema (this is the only one here). You’d also avoid all the logic within the plugin-ai-agent for fetching external schemas and type inference.

[asastre]

I agree, but doing that you would both plugins would be coupled and I don't think it's a good idea...

Also, doing it the way it is done we allow the bots to add other possible schemas more specific for its use cases, expanding the possibilities of the AI Agents. If we add this new schema in the AI Agent's plugin we will lose this improvement.

As a summary, I would leave it as it is now to expand the possibilities of the AI Agents plugin.

[vanbasten17]

then should this get typed as z.ZodObject<OutputSchema[]>?

[asastre]

I don't think so because it won't contain any of the baseMessageSchemas.