AI license-plate recognition and vehicle intelligence for US plates. Built as a final project for an AI & Deep Learning course by Gaurav Singh Thakur, Navatej Reddy Muddam, and Sai Pranay Gundu.
This project is source-available, not open source. Read the full LICENSE file for the binding terms.
| You can | You cannot |
|---|---|
| Read the source code on GitHub | Copy code into another project, public or private |
| Run it locally for evaluation and learning | Redistribute the source or compiled artifacts |
| Cite this work in papers, with attribution | Use the code commercially without written consent |
| Fork the repo to submit pull requests back here | Modify and re-publish under a different name |
| Reference the architecture and methodology | Add owner-PII lookups (legal exposure under DPPA) |
For permission to use this code outside the scope above, open an issue on this repository.
Copyright (c) 2026 — Gaurav Singh Thakur, Navatej Reddy Muddam, Sai Pranay Gundu. All rights reserved.
You point a camera at a US license plate, the model finds it, OCR reads it, and the backend returns the full public vehicle profile — make, model, year, VIN-decoded specs, manufacturer recalls. The owner of the vehicle is not returned; that data is restricted by federal law (see Legal below).
There's a web demo (python demo.py) and a React Native Android app
(python run.py). Both share the same backend.
git clone <this repo>
cd autolens
python demo.pyFirst launch installs fastapi, uvicorn, pillow, opencv-python and a
few other dependencies (about 60 seconds). Your browser opens to
http://127.0.0.1:7860.
The demo works with zero API keys and no trained model — the providers fall back to mock data so the UI is fully exercisable. To get real detection or vehicle lookups, see Configuration below.
autolens/ Web-demo Python package (FastAPI + samples)
backend/ Standalone backend used by the mobile app
PlateScanApp/ React Native Android client
webapp/ Static SPA served by the web demo
(HTML/CSS/JS + WebGL shader)
demo.py Web demo launcher (one command)
run.py / run.bat / run.sh Mobile-app launcher (backend + Metro + Android)
evaluate.py Detection + OCR evaluation harness
yolov12_ocr_dataset.ipynb Colab training notebook
PROJECT_REPORT.md Academic writeup
CONTRIBUTORS.md Who built what (3 of us)
LICENSE Source-available terms
- Scan — upload, webcam, or manual entry (type a plate + state). Returns the plate text, confidence, and full vehicle card.
- Examples — six pre-rendered US state plates so you can demo the pipeline without hunting for car photos.
- Batch — drag-drop multiple images, get a sortable table, export CSV.
- Video — upload a dashcam clip; we sample frames and dedupe plates.
- Analytics — Chart.js dashboard reading from the local scan history.
- Settings — provider status and env-var documentation.
- About — pipeline summary and feature comparison vs. CarFax, Plate Recognizer, CarsXE, OpenALPR.
Press ⌘K / Ctrl+K anywhere for the command palette.
How we approached the problem.
Three questions at scan time:
- Where is the plate in the image?
- What does the plate say?
- Given the plate text and state, what do we know about the car?
We drew the line at the car. Owner identity is restricted by US federal law (DPPA — see Legal below) and we built no path to retrieve it.
We trained on the Roboflow dataset freedomtech/yolo12-nnefg
(~5,000 labeled US plate images, free). We picked it over scraping our
own dataset because it was already cleaned, labelled, and split into
train/val/test. The training notebook
(yolov12_ocr_dataset.ipynb) downloads it
on first run.
YOLOv12-s, fine-tuned for 100 epochs at 640×640. Default Ultralytics
augmentations except we turned off horizontal flip (a flipped plate isn't
a plate). After training we exported best.pt to ONNX so the model runs
on CPU at around 60 ms per image. The goal from the start was a model
that could run on a laptop or a phone, not a GPU server.
We tried two libraries:
- PaddleOCR in our earliest prototype (
test.py) — accurate but a heavy install, especially on Windows. - EasyOCR in the current pipeline — lighter, simpler API, better Windows experience.
EasyOCR won on installation friction. The pipeline crops the plate, and if the crop is narrower than 120 px we upscale 3× with cubic interpolation before OCR. That single step measurably improves character accuracy on low-resolution inputs (see PROJECT_REPORT §5.1).
US plate-to-VIN is gated commercial data — there's no free public source for it. We use the CarsXE API for that step (paid, has a trial). Once we have a VIN, two free NHTSA government APIs do the rest:
- NHTSA vPIC decodes the VIN into make, model, year, body, engine, drivetrain, fuel type, plant, doors, GVWR.
- NHTSA Recalls returns active manufacturer recalls by year/make/model.
If CarsXE isn't configured the chain falls through: NHTSA on a known
VIN, mock data on nothing. Every result carries its source field so
the UI can show a "demo data" pill when the response is synthetic.
evaluate.py runs the pipeline against a labelled test set and
produces:
- Detection rate — fraction of images where YOLO found at least one plate
- OCR exact-match accuracy — full string equality after normalization
- Character-level accuracy — partial credit for almost-right reads
- Mean inference latency on CPU
- Top-10 character confusion pairs (O→0, I→1, B→8 are the usual suspects)
Run it with python evaluate.py --test-dir <dir> --ground-truth gt.csv.
Numbers are written to evaluation_report.md. Our held-out set numbers
are in PROJECT_REPORT.md §5.
- One JSON shape for two clients. The web SPA and the React Native
app render the same vehicle card. We locked down the schema in
backend/models.pyfirst and built both UIs against it second. The Pydantic models are the contract. - No build step for the web demo. Tailwind, Alpine.js, and Chart.js via CDN. Lower friction than a Vite/webpack toolchain for three contributors who needed to iterate fast.
- WebGL background as a single fragment shader. Two layered FBM simplex-noise fields. No three.js, no model files, runs on a cheap GPU.
- First-run pip installer (
autolens/deps.py). The graders open the repo and run one command — we didn't want them fighting requirements.txt and venvs. - Split into three modules along team-member boundaries so each of us could work on our tier without stepping on the other two. See CONTRIBUTORS.md.
The web demo reads backend/.env (copy from backend/.env.example).
All values are optional — the server falls through to mock data if a
provider isn't configured.
| Variable | What it turns on | Where to get it |
|---|---|---|
PLATE_RECOGNIZER_TOKEN |
Cloud plate OCR | platerecognizer.com — 2,500/month free |
CARSXE_KEY |
Plate → VIN lookup (US) | carsxe.com — paid trial |
YOLO_MODEL_PATH |
Your own ONNX detector | Output of the training notebook |
USE_MOCK_FALLBACK |
Demo data when nothing's wired | true / false (default true) |
Drop a best.onnx at the project root and the local YOLO + EasyOCR path
turns on, no API keys needed.
python run.py --setup # first time
python run.py # backend + Metro + Android buildPrereqs: Node 18+, Android Studio with an emulator, JDK 17, JAVA_HOME and
ANDROID_HOME set. The launcher streams all three processes into one
terminal with prefixed log output.
The mobile app talks to the backend over HTTP. The URL is configurable
from the Settings tab inside the app (default http://10.0.2.2:8000 for
the Android emulator; change to your LAN IP for a physical device).
A click-by-click walk-through of what happens when you hit Scan in the web demo, with file pointers so you can follow it in the source.
image → YOLOv12 → crop → EasyOCR → plate text
↓
plate + state → CarsXE → VIN
↓
VIN → NHTSA vPIC → full specs
↓
year + make + model → NHTSA recalls
Each stage falls back if the next provider is missing or down. With zero keys configured the chain still produces a deterministic mock response so the UI never breaks.
┌───────────────┐ ┌──────────────────┐ ┌─────────────────────┐
│ Browser SPA │ │ FastAPI app │ │ Provider chain │
│ webapp/ │ │ autolens/app.py│ │ backend/providers/│
└───────┬───────┘ └────────┬─────────┘ └─────────┬───────────┘
│ 1. POST /api/scan │ │
│ image + state │ │
├─────────────────────>│ │
│ │ 2. read_plate(image) │
│ ├────────────────────────>│ plate_ocr.py
│ │ │ ├─ Plate Recognizer (cloud)
│ │ │ ├─ YOLO + EasyOCR (local)
│ │ │ ├─ EasyOCR alone
│ │ │ └─ Mock fallback
│ │ 3. List[PlateCandidate] │
│ │<────────────────────────┤
│ │ │
│ │ 4. lookup_vehicle(...) │
│ ├────────────────────────>│ vehicle_info.py
│ │ │ ├─ CarsXE plate -> VIN
│ │ │ ├─ NHTSA vPIC VIN -> specs
│ │ │ ├─ NHTSA recalls
│ │ │ └─ Mock fallback
│ │ 5. VehicleInfo │
│ │<────────────────────────┤
│ │ │
│ │ 6. Draw bbox overlay │
│ │ (PIL ImageDraw) │
│ │ │
│ 7. JSON: plate + │ │
│ vehicle + timing │ │
│<─────────────────────┤ │
│ │ │
│ 8. Alpine.js renders │ │
│ VehicleCard │ │
│ │ │
│ 9. (optional) Save │ │
│ POST /api/scans │ │
├─────────────────────>│ 10. Append to │
│ │ backend/scans.json │
This is what gets logged when you hit Scan with a 1.2 MB plate photo and the server has no provider keys configured:
[browser] POST /api/scan image=1.2MB, state=CA
[server] autolens/app.py::api_scan
└─ backend/providers/plate_ocr.py::read_plate
├─ Plate Recognizer -- skipped, PLATE_RECOGNIZER_TOKEN unset
├─ Local YOLO+EasyOCR -- skipped, best.onnx not found
├─ EasyOCR alone -- ran, 240 ms, 0 candidates
└─ Mock fallback -- returned ['ABC1234', conf=0.42]
OCR: 248 ms
└─ backend/providers/vehicle_info.py::lookup_vehicle("ABC1234", "CA")
├─ CarsXE plate->VIN -- skipped, CARSXE_KEY unset
├─ NHTSA vPIC -- skipped, no VIN
└─ Mock -- returned 2021 Toyota Camry SE
Vehicle lookup: 0.2 ms
└─ Draw bounding box (PIL) -- 12 ms
Total: 260 ms
[browser] 200 OK
JSON: { plate, vehicle, candidates, timing_ms, overlay_png_b64 }
[alpine] Renders <plate-hero>, <spec-grid>, <recall-card>, <source-chip>
Same flow with real keys would replace those "skipped" lines with actual provider calls (Plate Recognizer ~600 ms, CarsXE ~800 ms, NHTSA vPIC ~300 ms — all over the public internet, so latency varies).
| Method | Route | Handler | What it does |
|---|---|---|---|
| GET | / |
index() |
Serves the SPA |
| GET | /static/* |
StaticFiles |
CSS / JS / shader files |
| GET | /api/providers |
api_providers |
Which provider chain is wired up |
| POST | /api/scan |
api_scan |
Image + state → plate + vehicle |
| POST | /api/lookup-plate |
api_lookup_plate |
Manual plate text + state → vehicle (no OCR) |
| POST | /api/batch |
api_batch |
Multi-image scan |
| POST | /api/video |
api_video |
Sample frames from a clip, scan each |
| GET | /api/samples |
api_samples |
Examples-tab gallery list |
| GET | /api/samples/{name}.png |
api_sample_image |
One sample image |
| POST | /api/scan-sample |
api_scan_sample |
Pre-baked response for a sample tile |
| GET | /api/scans |
api_scans |
Saved history |
| POST | /api/scans |
api_save_scan |
Append to history |
| DELETE | /api/scans/{id} |
api_delete_scan |
Remove one |
| DELETE | /api/scans |
api_clear_scans |
Wipe history |
| GET | /api/analytics |
api_analytics |
Aggregates for the dashboard |
| GET | /api/export/{csv,json} |
api_export_* |
Full history download |
Every handler lives in autolens/app.py.
| Layer | Choice |
|---|---|
| Detection | YOLOv12 (Ultralytics, ONNX export for CPU) |
| OCR | EasyOCR (CRNN + CTC) |
| VIN decode | NHTSA vPIC (free, official US government API) |
| Plate → VIN | CarsXE (commercial, optional) |
| Backend | FastAPI + Uvicorn, Pydantic v2 schemas |
| Storage | Local JSON (backend/scans.json) |
| Web frontend | Tailwind CSS, Alpine.js, Chart.js, raw WebGL background |
| Mobile | React Native 0.76 (TypeScript) |
Three of us built this together; each took ownership of one tier. See CONTRIBUTORS.md for the file-by-file split.
- Gaurav Singh Thakur — ML & Computer Vision (training, OCR, evaluation, sample generation, project report)
- Navatej Reddy Muddam — Backend & Infrastructure (FastAPI server, vehicle lookup, providers, run scripts, configuration)
- Sai Pranay Gundu — Frontend & Mobile (web SPA, WebGL shader, React Native Android app)
AutoLens does not surface personal information about vehicle owners. US DMV records are protected by the Driver's Privacy Protection Act (18 U.S.C. §2721), which restricts the data to law enforcement, licensed investigators, insurers, and a short list of other permissible users that does not include consumer applications. Every field AutoLens returns (make, model, year, VIN-decoded specs, recalls) is public information.
If you're reading this code with the intent of extending it: do not add owner-PII lookups. That's both a legal problem and a violation of this project's LICENSE.