3 - feat: PyMongo Async Migration

[deepansh96]

What this PR does

This PR makes our MongoDB calls non-blocking. Right now, every database call (like fetching a quiz or saving a session answer) freezes the FastAPI worker until MongoDB responds. With this change, the worker can handle other requests while waiting for the database, which means better performance under load.

What changed and why

1. New database setup (app/database.py)

Before: The app created a MongoDB connection the moment database.py was imported, and every file grabbed that connection directly.

After: The connection is created only when the app starts up, managed through three simple functions:

• init_db() — creates the connection on startup
• get_quiz_db() — gives you the database handle whenever you need it
• close_db() — cleanly shuts down the connection on app exit

This also means importing database.py no longer has side effects, which makes testing much more predictable.

2. App factory (app/main.py)

Before: The FastAPI app was created at import time as a global variable.

After: A create_app() function builds the app. It uses FastAPI's lifespan to start and stop the database connection automatically. On startup, it pings MongoDB to make sure the connection works — if it can't reach the database, the app fails immediately instead of silently serving errors.

3. All routers updated (6 files, 38 DB calls)

Every database call across all router files now uses await. The patterns are straightforward:

• collection.find_one(...) → await collection.find_one(...)
• list(collection.find(...)) → await collection.find(...).to_list(length=None)
• list(collection.aggregate(...)) → cursor = await collection.aggregate(...); await cursor.to_list(length=None)

4. MongoDB settings (app/settings.py)

A new MongoSettings class keeps all Mongo config (MONGO_DB_NAME, MONGO_AUTH_CREDENTIALS, pool sizes) separate from the general app settings. It reads environment variables only when needed, never at import time — this prevents test configuration from being ignored.

5. Test harness rewrite (app/tests/base.py)

Before: Tests shared the app's database connection and used hardcoded paths to find test data files.

After:

• Tests use their own separate sync MongoDB connection (self.db) for setup and assertions
• Test database is always quiz_test (never quiz) with a safety guard that refuses to touch production data
• Each test cleans up after itself
• Test data files are found relative to the test file location, not the working directory

6. Backfill script decoupled

backfill_time_limits_and_spent.py used to share the app's database connection. Now it creates its own, so it keeps working independently of the async migration.

7. Import cleanup

All test files now use consistent top-level imports (from database import ...) instead of a mix of relative imports (from ..database import ...). A pre-commit hook prevents the old style from coming back.

8. Infrastructure

• Terraform: Both testing and prod environments now pass MONGO_DB_NAME to ECS containers, so the database name is explicit instead of being buried in the connection URI
• CI: Tests run with MONGO_DB_NAME=quiz_test set explicitly
• Driver version: Pinned to pymongo==4.16.0 — versions before 4.15.1 have a timeout bug that hits our exact stack (AsyncMongoClient + uvicorn + FastAPI)

9. Documentation

• README, ENV docs, and CLAUDE.md updated to reflect the new architecture
• New docs/pymongo-async-staging-smoke.md with a manual smoke test checklist for deployments
• .env.example includes the new MONGO_DB_NAME variable

Test plan

• All 133 tests pass with real MongoDB (pytest)
• Pre-commit checks pass (pre-commit run --all-files)
• App refuses to start when MongoDB is unreachable
• Backfill script runs independently with its own connection
• Docker image builds (ARM64)
• Staging smoke test: create org → create quiz → create session → submit answer
• Verify Atlas connection limits can handle: 4 workers × 20 pool size × N tasks
• terraform plan succeeds for testing and prod

🤖 Generated with Claude Code

[suryabulusu]

left comment to look into startServerMac.sh and startServerLinux.sh

otherwise code is fine

[suryabulusu]

for local it can be 'quiz' only na

[suryabulusu]

Example: after ./startServerMac.sh --freshSync --source ..., Mongo restores data into quiz, while the backend now reads
from quiz_test, so quiz-frontend will look empty even though the sync succeeded. I’d
keep .env.example on quiz for local runtime and reserve quiz_test for the test
harness / CI only.

[suryabulusu]

• use MONGO_DB_NAME instead of hardcoded quiz in the mongosh drop

[suryabulusu]

• make mongorestore restore into the configured DB, not implicitly into quiz