Skip to content

[Bug] consumer export worker streams entire collection with no document count cap, enabling resource exhaustion on collections with millions of documents #249

Description

@anshul23102

Bug Summary

The export worker in apps/consumer/src/workers/export.worker.js opens a MongoDB cursor over the entire requested collection and streams every document to storage without any limit on the number of documents processed:

const cursor = Model.find().lean().cursor();

for await (const doc of cursor) {
    // streams every document, no count cap
}

The dbExportHandler controller enforces a per-project daily export quota (1 export/day for free, 5 for pro), but it does not cap the size of the collection being exported. A project that has accumulated millions of documents can trigger an export job that:

  1. Holds a MongoDB cursor open for a prolonged period, consuming a connection from the pool.
  2. Writes gigabytes of JSON to the storage backend (Supabase or external S3-compatible).
  3. Keeps the BullMQ worker occupied indefinitely, blocking other queued export jobs from executing.
  4. Generates a presigned download URL for a potentially multi-gigabyte file delivered via email.

There is no MAX_EXPORT_ROWS constant, no .limit() call, and no streaming byte-count guard in either the controller or the worker.

Expected Behavior

The export worker should enforce a maximum document count (for example, 100,000 documents) or a maximum byte limit on the streamed JSON payload. When the limit is reached, the export should either stop and note the truncation, or reject the job and return an informative error to the user.

Actual Behavior

All documents in the collection are exported with no count or size ceiling.

Affected File

apps/consumer/src/workers/export.worker.js, the cursor iteration loop.


@geturbackend I would like to work on this issue. Could you please assign/ it to me? Contributing under NSoC '26.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions