🛡️ FDE Guard

FDE Guard (Forward Deployed Engineering Guard) is a local-first toolkit that helps Forward Deployed Engineers diagnose and de-risk AI deployments at client sites — without shipping any client data to a third-party API. The entire stack runs locally via a single docker compose up, using lightweight local models served by Ollama — and the exact same codebase deploys to Google Cloud Run (including push-to-deploy from GitHub) with no forked Dockerfiles or separate cloud build config. See Running in Google Cloud below.

It bundles four tools behind one Streamlit UI:

1. RAG Ground-Truth Alignment Tool — turns "the AI is broken" into a data-driven retrieval diagnosis.
2. Config-to-Spec Semantic Validator — catches config drift before it causes an on-site incident.
3. Guardrail Smoke Tester — proves privacy/safety guardrails work before go-live, with a PDF report.
4. Executive Dashboard — translates infra telemetry into a TCO-vs-business-value summary for leadership.

Every tab has a "Generate synthetic data" button so you can see the tool work end-to-end before you ever point it at a real client environment.

---

Architecture

Service	Role	Port
frontend	Streamlit single-page app, 4 tabs	7501
backend	FastAPI — business logic, synthetic data, PCA, diffing, PDF reports	7080
ollama	Local LLM serving (chat + embeddings), CPU-only, models baked into the image	7434
chromadb	Local vector database (two collections: client_kb, runbook_specs)	7000

Every service above lives in its own folder with its own Dockerfile (backend/, frontend/, ollama/, chromadb/). docker-compose.yml builds all four for local development, and the exact same folders/Dockerfiles are what you point Cloud Run's GitHub auto-deploy at for a cloud deployment — see Running in Google Cloud below. There's one codebase, not a local version and a separate cloud version.

Models used by default (chosen to be small enough for an older Intel MacBook):

• Chat: llama3.2:1b (~1.3 GB)
• Embeddings: all-minilm (~45 MB)

Both are pulled and baked into the ollama image at build time (not at container startup), so there's no separate init step to wait on — docker compose up and Cloud Run cold starts both just work once the image is built. You can swap either model via environment variables — see Configuration.

---

The 4 Tools

1. RAG Pipeline "Ground-Truth" Alignment Tool

Problem it solves: A client says "the AI gives bad answers, your system is broken" — but usually the issue is messy source documents or poor chunking, not the model.

What it does:

• Paste the client's problematic query and the bad answer they received (or generate a synthetic example with one click).
• The tool embeds the query locally and retrieves the top-K chunks from the client's knowledge base (stored in ChromaDB).
• Runs PCA (scikit-learn, CPU-only) to project the embedding space down to a few dimensions and renders it as an interactive parallel-coordinates plot plus a topic-cluster scatter plot, so you can see whether the right context was ever retrieved or was buried by poor ranking.
• Produces a plain-English verdict (retrieved_correctly, not_retrieved, buried_by_ranking, empty_corpus) and a client-ready talking point generated by the local LLM.

Demo data: "Generate synthetic client KB + sample query" creates ~25 topic-tagged chunks (billing, auth, data retention, onboarding, plus irrelevant "noise" documents) and a sample bad-answer scenario.

2. Config-to-Spec Semantic Validator

Problem it solves: Config drift (YAML/JSON) between staging and a client's specific VPC is a constant source of "it worked in staging, why is it failing here?" debugging loops.

What it does:

• Paste your golden deployment runbook (markdown) and the client's actual config.
• A structural pass checks well-known fields (temperature ranges, token limits, retrieval top-K, telemetry hooks, guardrail flags, prompt-cache settings) against sane defaults.
• A semantic pass sends both documents to the local LLM to catch drift that isn't a simple key/value rule (naming conventions, narrative constraints in the runbook, etc.).
• The golden runbook is also embedded and stored in ChromaDB (runbook_specs collection) so it becomes searchable across future engagements.
• Returns a severity verdict (pass / warning / critical) you can paste straight into a ticket.

Demo data: "Generate synthetic pair" toggle lets you preview both a clean config and one with deliberate drift (disabled telemetry, out-of-range temperature, missing guardrail flags).

3. LLM Telemetry & Guardrail "Smoke Tester"

Problem it solves: FDEs need to prove to client info-sec stakeholders that guardrails work before production traffic is connected.

What it does:

• Fires a small adversarial prompt dataset (prompt injection, PII leakage attempts, out-of-bounds business queries) at a target endpoint — either a real client endpoint URL, or simulation mode if nothing is wired up yet.
• Async, sequential execution (no heavyweight load-testing infra needed) keeps it light enough to run from a laptop.
• Visualizes block rate, leak rate, and latency per category in real time.
• Generates a downloadable PDF compliance/readiness report for the client's info-sec team.

Demo data: Simulation mode is itself the synthetic-data path — no separate generation step needed, just toggle "Simulation" and run.

4. Executive Dashboard

Problem it solves: At the end of an engagement, the FDE needs a clean, non-technical summary for the client's VP of Product or CFO.

What it does:

• Takes TCO inputs (cloud compute, vector DB storage, FDE engineering hours) and business value inputs (token-optimization savings, customers retained, regulatory risk avoided).
• Calculates one-time engineering cost, monthly TCO, monthly business value, net monthly impact, and payback period.
• Renders gauges for retrieval accuracy and guardrail failure rate, a cost-vs-value bar chart, and the infra-telemetry → business-metric → financial-impact table from the original spec.

Demo data: "Pull synthetic engagement metrics" prefills the form with a plausible random engagement so you can see the full dashboard immediately.

---

Prerequisites

• Docker Desktop (or Docker Engine + Compose plugin) — works fine on Intel or Apple Silicon Macs, Linux, and Windows/WSL2.
• ~3 GB free disk space for model + container images on first run.
• No GPU required — everything runs on CPU.

Setup & Run

# 1. Clone or copy this project, then cd into it
cd fdeguard

# 2. (Optional) copy the example env file if you want to override model choices
cp .env.example .env

# 3. Build and start everything
docker compose up --build

What happens on first run:

1. ollama builds (this is the slow step the first time — it downloads and bakes llama3.2:1b and all-minilm into the image at build time, a few minutes depending on your connection — and is skipped on subsequent runs thanks to Docker's layer cache, unless you change the Dockerfile).
2. chromadb builds and starts with a persistent volume (chroma_data).
3. backend (FastAPI) starts once Ollama and ChromaDB are healthy.
4. frontend (Streamlit) starts and connects to the backend.

Once it's up, open:

• App UI: http://localhost:7501
• API docs (Swagger): http://localhost:7080/docs
• ChromaDB: http://localhost:7000

The sidebar in the UI shows live status for the backend, Ollama, and ChromaDB so you know everything is wired up correctly.

Stopping / resetting

# Stop everything, keep the ChromaDB volume cached for next time
docker compose down

# Stop and wipe cached data (ChromaDB vector store)
docker compose down -v

Running only specific services

docker compose up ollama chromadb backend   # backend stack only, no UI

---

Running in Google Cloud

This same repo — the same backend/, frontend/, ollama/, and chromadb/ folders and Dockerfiles used by docker-compose.yml above — deploys directly to Google Cloud Run, including via Cloud Run's built-in "continuously deploy from a repository" GitHub integration (push to deploy, no separate CI config to maintain). There's no code fork or divergent Dockerfile between local and cloud.

See docs/CLOUD_RUN.md for the full walkthrough: setting up GitHub auto-deploy for all 4 services, the env vars each one needs (Cloud Run has no docker-compose-style service-name networking, so backend/ollama/chromadb are wired together via their real *.run.app URLs instead), sizing guidance for running the LLM on Cloud Run CPU, and keeping the services private via Internal ingress.

---

Configuration

All configuration is via environment variables, set either in docker-compose.yml directly or in a .env file (see .env.example):

Variable	Default	Description
OLLAMA_CHAT_MODEL	llama3.2:1b	Any Ollama-compatible chat model
OLLAMA_EMBED_MODEL	all-minilm	Any Ollama-compatible embedding model

To use a larger/more capable model (if your machine can handle it), edit the RUN ollama pull ... lines in ollama/Dockerfile to pull it, rebuild (docker compose build ollama), and update the matching environment variable on the backend service.

---

Project structure

fdeguard/
├── docker-compose.yml         # local dev: builds all 4 folders below
├── .env.example
├── docs/
│   ├── architecture.svg
│   └── CLOUD_RUN.md            # Cloud Run deployment guide (incl. GitHub auto-deploy)
├── ollama/
│   ├── Dockerfile              # bakes chat + embedding models into the image at build time
│   └── entrypoint.sh           # binds to $PORT for Cloud Run compatibility
├── chromadb/
│   └── Dockerfile              # thin wrapper around the official chromadb/chroma image
├── backend/                    # FastAPI service
│   ├── main.py
│   ├── routers/
│   │   ├── rag_alignment.py
│   │   ├── config_validator.py
│   │   ├── guardrail_tester.py
│   │   └── executive_dashboard.py
│   ├── services/
│   │   ├── llm_client.py       # Ollama wrapper (chat + embeddings)
│   │   ├── vectorstore.py      # ChromaDB wrapper (Compose host:port OR Cloud Run HTTPS URL)
│   │   └── synthetic_data.py   # Synthetic data generators for all 4 tools
│   └── Dockerfile
└── frontend/                   # Streamlit single-page app
    ├── app.py                  # 4-tab entrypoint
    ├── tabs/
    │   ├── tab1_rag_alignment.py
    │   ├── tab2_config_validator.py
    │   ├── tab3_guardrail_tester.py
    │   └── tab4_executive_dashboard.py
    ├── utils/api_client.py
    └── Dockerfile

Troubleshooting

• docker compose build takes a long time on ollama — expected on first build (or after changing ollama/Dockerfile): it's downloading and baking llama3.2:1b + all-minilm into the image. Subsequent builds reuse Docker's layer cache and are fast.
• Sidebar shows "Ollama: not ready" — the ollama container is still starting up (Cloud Run: still cold-starting). Check with docker compose logs -f ollama. Since models are baked into the image rather than pulled at runtime, this should resolve in a few seconds, not minutes.
• First request to a tab is slow — the CPU-bound chat model takes a few seconds per call on older hardware; this is expected and only affects the LLM-summary calls, not the chart/diagnostic logic.
• Port already in use — change the left-hand side of the relevant ports: mapping in docker-compose.yml (e.g. "7502:7501").
• Deploying to Cloud Run and getting Failed to resolve 'backend' or similar — that means Cloud Run env vars are still set to Compose-style hostnames (http://backend:7080). See docs/CLOUD_RUN.md for the correct per-service env var setup.