Skip to content

CLAUDE.md

Latest commit

 

History

History
285 lines (231 loc) · 10.1 KB

CLAUDE.md

File metadata and controls

285 lines (231 loc) · 10.1 KB
Raw

JewelAI — 3D Ring Designer

AI-driven 2D-to-3D jewelry generation with real-time customization

Project Overview

JewelAI is a web application that lets users describe a ring in natural language, have an AI chatbot interrogate their requirements (without assuming any detail), then generate and render a fully interactive 3D model with real-time customization controls.

The MVP covers rings only. Users can customize metal type, gemstone type/size, band width, and ring style — and see updates reflected instantly in a 3D viewer supporting front, top, side, and bottom perspectives.

Tech Stack

Frontend

  • Framework: Next.js 14 (App Router) + TypeScript
  • Styling: Tailwind CSS (utility-first, no CSS modules)
  • 3D Viewer: Three.js (via @react-three/fiber + @react-three/drei)
  • State: Zustand for global ring config state
  • Deploy: Vercel

Backend

  • Framework: FastAPI + Python 3.11+
  • 3D Generation: TripoSG (HuggingFace) or InstantMesh (OSS fallback)
  • Component Recognition: GPT-4o Vision API (OpenAI)
  • AI Chatbot: Claude API — model claude-sonnet-4-5
  • File Formats: GLB (primary render), STL (export only)
  • Deploy: Render (free tier)

APIs & Keys (never commit — use .env.local / Render env vars)

ANTHROPIC_API_KEY=          # Claude API — chatbot
OPENAI_API_KEY=             # GPT-4o Vision — component recognition
HUGGINGFACE_API_TOKEN=      # TripoSG inference
NEXT_PUBLIC_API_URL=        # FastAPI backend URL (Render)

Repository Structure

jewel-ai/
├── frontend/                    # Next.js 14 app
│   ├── app/
│   │   ├── layout.tsx
│   │   ├── page.tsx             # Main designer page
│   │   ├── api/                 # Next.js API routes (thin proxies only)
│   │   └── globals.css
│   ├── components/
│   │   ├── viewer/
│   │   │   ├── RingViewer.tsx   # Three.js canvas wrapper
│   │   │   ├── RingModel.tsx    # Procedural / GLB mesh renderer
│   │   │   └── ViewControls.tsx # Front/Top/Side/Bottom snap buttons
│   │   ├── chat/
│   │   │   ├── ChatPanel.tsx    # AI chat UI
│   │   │   ├── MessageList.tsx
│   │   │   └── ChatInput.tsx
│   │   ├── controls/
│   │   │   ├── MetalPicker.tsx
│   │   │   ├── GemPicker.tsx
│   │   │   ├── SizeSlider.tsx
│   │   │   └── StyleSelect.tsx
│   │   └── ui/                  # Shared primitives (Button, Badge, etc.)
│   ├── store/
│   │   └── ringStore.ts         # Zustand: metal, gem, gemSize, style, bandW
│   ├── lib/
│   │   ├── api.ts               # Typed fetch wrappers → FastAPI
│   │   └── types.ts             # RingConfig, ChatMessage, etc.
│   ├── public/
│   │   └── models/              # Pre-generated GLB files (static fallback)
│   ├── .env.local               # NEVER commit
│   └── package.json
│
├── backend/                     # FastAPI app
│   ├── main.py                  # App entry point + CORS
│   ├── routers/
│   │   ├── chat.py              # POST /chat — Claude API streaming
│   │   ├── generate.py          # POST /generate — TripoSG 3D generation
│   │   ├── recognize.py         # POST /recognize — GPT-4o Vision
│   │   └── export.py            # GET /export/stl — GLB → STL conversion
│   ├── services/
│   │   ├── claude_service.py    # Claude API client + system prompt
│   │   ├── tripo_service.py     # TripoSG / HuggingFace client
│   │   ├── vision_service.py    # GPT-4o Vision client
│   │   └── mesh_service.py      # trimesh GLB/STL utilities
│   ├── models/
│   │   └── schemas.py           # Pydantic: RingConfig, ChatRequest, etc.
│   ├── requirements.txt
│   └── .env                     # NEVER commit
│
├── CLAUDE.md                    # ← this file
└── README.md

Development Commands

Frontend

cd frontend
npm install
npm run dev          # http://localhost:3000
npm run build
npm run lint
npm run type-check   # tsc --noEmit

Backend

cd backend
python -m venv venv
source venv/bin/activate      # Windows: venv\Scripts\activate
pip install -r requirements.txt
uvicorn main:app --reload     # http://localhost:8000

Both Together (from root)

# Terminal 1
cd backend && uvicorn main:app --reload

# Terminal 2
cd frontend && npm run dev

Core Data Structures

RingConfig (shared frontend ↔ backend)

interface RingConfig {
  style: 'solitaire' | 'halo' | 'eternity' | 'cluster' | 'split' | 'pave';
  metal: 'gold' | 'silver' | 'platinum' | 'rose-gold' | 'blackened';
  gem: 'diamond' | 'ruby' | 'emerald' | 'sapphire' | 'amethyst' | 'topaz' | 'morganite' | 'none';
  gemSize: number;       // 0.3 (tiny) → 1.0 (medium) → 2.2 (statement)
  bandWidth: number;     // 0.07 (thin) → 0.32 (bold)
  bandProfile: 'flat' | 'semi-flat' | 'round' | 'high-dome' | 'knife-edge';
  engraving?: string;    // future
  ringSize?: number;     // US ring size, future
}

ChatMessage

interface ChatMessage {
  role: 'user' | 'assistant';
  content: string;
  timestamp: number;
  configSnapshot?: Partial<RingConfig>; // emitted when AI confirms a spec
}

Claude API Integration

System Prompt Philosophy

The Claude chatbot must never assume any design detail. It asks exactly 1–2 focused questions per turn and only confirms a spec when the user explicitly states it.

When all required fields are confirmed, Claude emits a structured config block:

<RING_CONFIG>{"metal":"gold","gem":"diamond","gemSize":1.2,"style":"solitaire","bandWidth":0.14,"bandProfile":"round"}</RING_CONFIG>

The frontend parses this tag from the streamed response and applies it to the Zustand store in real-time.

Required fields to confirm before emitting config

  1. Ring style
  2. Metal type
  3. Gemstone type
  4. Gemstone size (relative: tiny / small / medium / large / statement)
  5. Band width (thin / medium / thick)
  6. Band profile (optional but ask)

Claude model

Always use: claude-sonnet-4-5

Streaming

Use the Anthropic streaming API. Frontend reads the stream chunk-by-chunk, renders text in real-time, and parses <RING_CONFIG> when the stream closes.

3D Viewer Behaviour

View Modes

Button Camera position Notes
Free Rotate Default Auto-rotates; drag to orbit
Front Eye-level, face-on Ring torus visible
Top Directly above Stone setting prominent
Side 90° rotation Profile / band thickness
Bottom Below ring Underside of setting

Interaction

  • Mouse drag / touch drag: orbit rotation (lerp smoothed)
  • Scroll / pinch: zoom in/out (min 2.0, max 10.0 camera Z)
  • View buttons: snap to preset with 0.08 lerp factor

Config → Mesh mapping

Zustand ringStore updates trigger a buildRing() call that rebuilds the Three.js scene. Avoid full scene recreation — update material colors and geometry in-place where possible for performance.

API Endpoints (FastAPI)

POST /chat                    # Body: {messages: ChatMessage[], config?: RingConfig}
                              # Returns: SSE stream of Claude response

POST /generate                # Body: {prompt: string, config: RingConfig}
                              # Returns: {glb_url: string, generation_id: string}

POST /recognize               # Body: multipart/form-data image
                              # Returns: {detected_components: string[], suggested_config: RingConfig}

GET  /export/stl/{id}         # Returns: STL file download
GET  /health                  # Returns: {status: "ok"}

Coding Conventions

TypeScript / React

  • Use functional components only, no class components
  • Named exports for all components, default export only for pages
  • Props interfaces named ComponentNameProps
  • All async operations wrapped in try/catch with typed error states
  • No any types — use unknown and narrow
  • Tailwind only — no inline styles, no CSS modules

Python / FastAPI

  • Pydantic v2 for all request/response models
  • Async route handlers (async def) throughout
  • All external API calls in services/ — never directly in routers
  • Environment variables via python-dotenv, validated on startup
  • Return HTTPException with clear status codes, never raw exceptions

File naming

  • React components: PascalCase.tsx
  • Utilities / hooks: camelCase.ts
  • Python modules: snake_case.py

Git commits

Format: type(scope): message Types: feat fix refactor style docs chore Example: feat(chat): add streaming Claude response parser

Environment Setup Checklist

  • Node.js 18+ installed
  • Python 3.11+ installed
  • frontend/.env.local created with all required keys
  • backend/.env created with all required keys
  • Anthropic API key obtained from console.anthropic.com
  • OpenAI API key obtained from platform.openai.com
  • HuggingFace token obtained from huggingface.co/settings/tokens
  • Vercel project linked to frontend/
  • Render service linked to backend/

Known Constraints (MVP scope)

  • Rings only — no necklaces, bracelets, earrings yet
  • Static procedural geometry in MVP — TripoSG generation is an async job (show skeleton + progress bar)
  • No user accounts — configs are session-only in MVP
  • Free-tier Render — backend cold starts ~30s; show loading state
  • No AR — future feature post-MVP

Future Features (post-hackathon)

  1. Ring sizing (inner diameter, US/EU/IN sizing)
  2. Engraving text preview
  3. AR try-on via WebXR / MediaPipe hand tracking
  4. STL export for 3D printing
  5. User accounts + saved designs
  6. Image upload → component recognition (GPT-4o Vision full pipeline)
  7. Price estimation based on metal weight + gem carat
  8. Expand to necklaces, earrings, bracelets