Ad-hoc aggregation pipelines

[KamikX]

Hello,

Why does RESTHeart require aggregations to be pre-defined in collection metadata (aggrs) instead of allowing ad-hoc aggregation pipelines to be executed directly via a single HTTP request (similar to running db.collection.aggregate([...]) in mongosh)?

I understand from the documentation that only named aggregations can be invoked using
GET /collection/_aggrs/{name}, and that pipelines must be defined via a PATCH/metadata update.
However, this creates limitations for dynamic dashboards (e.g., using Grafana Infinity), where a single-request, inline aggregation would be very useful.

Is this design choice intentional for security or performance reasons, or is it simply a current implementation limitation?
Additionally, is support for ad-hoc aggregation pipelines planned (e.g., POST /collection/_aggregate with a pipeline in the request body)?

[ujibang]

You're correct—this is by design for security and performance reasons. Allowing clients to execute arbitrary aggregations would be dangerous, and there's no straightforward way to define permissions that restrict aggregation execution.

Our security guidelines recommend:

Use only aggregations and forbid the filter query parameter

Aggregations are parametric, so you can pre-define them and invoke them from Grafana with different parameters. If this doesn't cover your use case, you can easily create a custom service that accepts aggregation pipelines in the request body and executes them.

[KamikX]

@ujibang thanks for your suggestions, maybe the last one with some restrictions could be a viable solution.

[ujibang]

For an example of service executing custom command on mongodb see https://github.com/SoftInstigate/restheart/tree/master/examples/mongo-status-service

[KamikX]

Hello,

After exploring the possibilities, I encountered a problem with achieving the same behavior of my plugin as the original API provides. I have implemented custom JsonService plugin where I access MongoDB directly using the injected MongoClient (via @Inject("mclient")).
I would like to preserve full compatibility with RESTHeart’s native behavior, especially:
1. ETag generation and validation (_etag field, strong/weak etags, automatic update on document changes, and the logic behind If-Match / If-None-Match).
2. Metadata structure created by RESTHeart for collections and databases (_properties, _properties.collection, etc.).

My questions:

A) Is there a way to reuse or invoke RESTHeart’s existing ETag logic from inside my custom plugin when using the MongoDB Java client directly?
For example, can I call an internal API, helper class, or service that RESTHeart already uses for computing and validating ETags, instead of re-implementing this logic manually?

B) Is the logic that creates and maintains _properties (at DB/collection level) exposed in a reusable form for plugins?
If yes, how can a custom plugin trigger or reuse the mechanisms that normally generate:
• _properties,
• _properties.collection,
• other metadata fields automatically created by RESTHeart?

C) If reuse is not currently possible, what is the recommended pattern for achieving consistent ETag behavior and _properties metadata management in custom plugins that do not go through the REST API layer (MongoService)?

I want my plugin to behave identically to RESTHeart’s built-in Mongo REST API, but at the same time I need the performance and flexibility of using the MongoDB Java driver directly.

Thanks in advance for any guidance.

[ujibang]

Let me clarify - I'm not suggesting you rewrite the RESTHeart Mongo service. It's quite complex, and those features aren't meant to be available to custom plugins. A plugin should only depend on restheart-commons and avoid using internal APIs.

What I actually suggested was simply adding a BsonService to execute aggregations from the request body, which would work alongside the Mongo service.

[KamikX]

For clarification, I abandoned the original intention of creating a service to run ad-hoc aggregation channels, but I wanted to stick with my own service that would have restricted access (admin only) and would be able to create a database, a collection, and upload aggregations to it in one call, then the aggregations could be classically called via the native API. Then I ran into a problem with the metadata that RESTHeart creates internally.

Maybe it would be enough to execute some routine only after creating a new collection, at which point the metadata could be created "natively" and it would be possible to work with them. I found this example slackNotifierInterceptor interceptor the plugin will only perform the action when a document is created in a specific collection. It is possible to track the creation of the collection?

[ujibang]

You can create a database and a collection with aggregation with just two calls and no custom plugins:

PUT /db
PUT /db/coll { "aggrs": [ { "uri": "paginated-results", "stages": [ { "$match": { "name": { "$var": "name" } } } ] } ] }