🤖 J.A.R.V.I.S — Personal AI Assistant

Just A Rather Very Intelligent System
A real-time, voice-enabled AI assistant inspired by Iron Man's JARVIS — built with LiveKit, Gemini, and React.

---

✨ Features

• 🎙️ Real-Time Voice Conversation — Full-duplex voice chat powered by LiveKit WebRTC
• 🧠 Gemini 2.0 Flash — Google's LLM for fast, intelligent responses
• 🗣️ Google TTS & STT — Natural text-to-speech and speech-to-text
• 🌤️ Weather Lookup — Get current weather for any city (Open-Meteo API)
• 🔍 Web Search — Search the web via DuckDuckGo
• 📧 Send Emails — Compose and send emails through Gmail SMTP
• 📷 Live Camera Feed — Webcam integration with HUD overlay
• 💬 Text Chat Fallback — Type messages when voice isn't convenient
• 🎨 Iron Man HUD UI — Futuristic interface with scan lines, arc reactor animations, and Orbitron typography

---

🏗️ Architecture

┌──────────────────────────────────────────────────┐
│                   Frontend (React + Vite)         │
│          LiveKit Room · Camera · Text Chat        │
│              Port 5173 (dev server)               │
└──────────┬──────────────────┬────────────────────┘
           │ WebRTC           │ REST API
           ▼                  ▼
┌──────────────────┐  ┌──────────────────────────┐
│   LiveKit Cloud   │  │  Token Server (FastAPI)   │
│   (SFU Server)    │  │  /token · /api/start      │
│                   │  │  /api/chat                 │
└────────┬─────────┘  │  Port 8001                 │
         │            └──────────────────────────┘
         ▼
┌──────────────────────────────────────────────────┐
│              AI Agent (LiveKit Agent)             │
│   Gemini LLM · Google TTS/STT · Silero VAD       │
│   Tools: Weather · Web Search · Email             │
└──────────────────────────────────────────────────┘

---

Tech Stack

Layer	Technology
LLM	Google Gemini 2.0 Flash
Voice	Google TTS (Journey-D) + Google STT
VAD	Silero Voice Activity Detection
Realtime	LiveKit (WebRTC)
Backend	FastAPI + Uvicorn
Frontend	React 18 + Vite
Chat fallback	GitHub Models (GPT-5) via OpenAI SDK
Search	DuckDuckGo (via LangChain)

📁 Project Structure

AI-agent/
├── jarvis/
│   ├── agent.py            # LiveKit AI agent (Gemini + voice pipeline)
│   ├── tools.py            # Function tools (weather, search, email)
│   ├── prompts.py          # JARVIS persona & system prompt
│   ├── token_server.py     # FastAPI server (tokens + chat API)
│   ├── requirements.txt    # Python dependencies
│   ├── verify_imports.py   # Import verification script
│   ├── .env                # Environment variables (not tracked)
│   └── frontend/
│       ├── index.html      # Entry HTML
│       ├── package.json    # Node dependencies
│       ├── vite.config.js  # Vite configuration
│       └── src/
│           ├── App.jsx     # Main React app (HUD UI)
│           ├── main.jsx    # React entry point
│           └── index.css   # Global styles
└── .gitignore

---

🚀 Getting Started

Prerequisites

• Python 3.10+
• Node.js 18+
• LiveKit Cloud account → livekit.io
• Google AI API key (for Gemini, TTS, STT)
• GitHub PAT (for GitHub Models / GPT-5 fallback)

git clone https://github.com/LashminiAD/AI-agent.git
cd AI-agent/jarvis

python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -r requirements.txt

Create a .env file inside the jarvis/ directory:

# LiveKit
LIVEKIT_URL=wss://your-project.livekit.cloud
LIVEKIT_API_KEY=your_api_key
LIVEKIT_API_SECRET=your_api_secret

GOOGLE_API_KEY=your_google_api_key
GITHUB_TOKEN=your_github_pat

GMAIL_USER=your_email@gmail.com
GMAIL_APP_PASSWORD=your_app_password

Then Set Up the Frontend

cd frontend
npm install

Run Everything

You need 3 terminals:

Terminal 1 — Token Server:

cd jarvis
python token_server.py

Terminal 2 — AI Agent:

cd jarvis
python agent.py dev

Terminal 3 — Frontend:

cd jarvis/frontend
npm run dev

Then open http://localhost:5173 and click INITIATE to connect! 🚀

---

🛠️ Tools Available

Tool	Description	API
🌤️ get_weather	Current weather for any city	Open-Meteo (free)
🔍 search_web	Web search via DuckDuckGo	DuckDuckGo (free)
📧 send_email	Send emails with optional CC	Gmail SMTP

Example Voice Commands

• "Jarvis, what's the weather in London?"
• "Search the web for latest AI news"
• "Send an email to john@example.com with subject Hello"

---

Notes

This is a personal project built for learning and demo purposes — it isn't hardened for production use (no rate limiting, minimal input validation on the email tool, and credentials are managed via plain .env files).

---

📄 License

This project is for educational and personal use.

---

Built with ❤️ by LashminiAD
_{STARK INDUSTRIES · AI DIVISION}