SupportBot: LangChain Ecosystem Demo

SupportBot is a local CLI demo of LangGraph agents, LangChain tool calling, local RAG with FAISS, and optional LangSmith tracing.

It answers questions about the LangChain ecosystem from a small local markdown corpus. It also delegates arithmetic to a calculator tool. The project is intentionally small: no Docker, no hosted database, no OpenAI or Anthropic key required.

What this demonstrates

• A LangGraph agent loop: model decides, tools run, model answers.
• LangChain tool calling with search_docs and calculator.
• Local RAG: Hugging Face embeddings, FAISS index, markdown docs.
• Optional LangSmith tracing for every graph step.
• Local evals that inspect graph messages, not just the final answer.

Architecture

User Query
    |
    v
LangGraph StateGraph
    |
    |-- agent node: OpenRouter or Venice.ai via ChatOpenAI
    |
    |-- tools node:
    |      - search_docs -> FAISS -> data/docs/*.md
    |      - calculator -> numexpr
    |
    |-- conditional edge:
           tool call requested -> tools -> agent
           no tool call -> END

Final Answer + Sources + LangSmith Trace Info

Why OpenRouter and Venice.ai

This demo uses OpenRouter by default and supports Venice.ai as a backup provider. Both expose OpenAI-compatible APIs, so LangChain's ChatOpenAI integration works without an OpenAI account. The goal is low inference cost and no direct dependence on OpenAI, Anthropic, or DeepSeek.

Setup

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env

Edit .env and add either OpenRouter or Venice.ai credentials.

LLM_PROVIDER=openrouter
OPENROUTER_API_KEY=sk-or-...
OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
OPENROUTER_MODEL=mistralai/mistral-small-3.2-24b-instruct
OPENROUTER_APP_NAME=SupportBot LangChain Demo

EMBEDDING_MODEL=sentence-transformers/all-MiniLM-L6-v2

LANGCHAIN_TRACING_V2=true
LANGCHAIN_API_KEY=lsv2_...
LANGCHAIN_PROJECT=supportbot-mvp

To run without LangSmith:

LANGCHAIN_TRACING_V2=false

Example queries

python app.py "What is LangGraph?"
python app.py "According to the local docs, what codename does the SupportBot LangGraph agent use?"
python app.py "Calculate 1500 * 0.15 / 1000000"

The codename query should return River Otter. That fact exists only in data/docs/langgraph-overview.md, so it proves the answer came through local retrieval instead of model prior knowledge.

Optional visual UI

The demo is CLI-first. For a visual inspection panel, install the optional Streamlit wrapper:

pip install -r requirements-ui.txt
streamlit run ui.py

The UI panel shows the answer, tools used, sources, message timeline, raw messages, and LangSmith trace status. It is an inspection wrapper for understanding the graph, not a production chat app.

For a step-by-step review checklist, see docs/manual-ui-review-and-screenshots.md.

Evals

python eval.py

The eval runner covers 8 cases:

Check	What it verifies
Source citation	RAG answers include source filenames
Numeric accuracy	Calculator answers match expected values
Out-of-scope	Answers don't fake local documentation
Tool use	Tools were called, verified from graph messages

Local docs

The docs in data/docs/ are short, manually adapted summaries from official LangChain, LangGraph, and LangSmith documentation. They are intentionally small so the retrieval path is easy to inspect without scrolling through large chunks.

LangSmith tracing

When LangSmith is configured, traces should show this pattern for RAG questions:

agent -> tools/search_docs -> agent -> final answer

That trace is the proof that the agent used retrieval. The final answer alone is not enough evidence.

Known limitations

Limitation	Details
Small corpus	Retrieval coverage is limited to the six included docs
First-run download	Local embedding model downloads on first use
Venice model support	Tool calling depends on the selected Venice model
FAISS safety	Cache loading is for this app's own index only. Do not load untrusted indexes
Minimal scope	No production web app, auth, rate limiting, Docker, streaming, or multi-turn memory
UI is local only	The Streamlit wrapper runs locally and is not a deployed web service

More detail

For design decisions and the reasoning behind the project, start with docs/WRITEUP.md.

Additional docs:

• docs/LangChainEcosystemMVP.md
• docs/supportbot-mvp-implementation.md
• docs/adr/0001-supportbot-mvp-architecture.md
• docs/manual-ui-review-and-screenshots.md