llm.go

GPT-2 implementation written in go only using the standard library.

🪞 Quick start

Install python dependencies, output tokenized dataset

make setup

Run the training script:

make train

This will run go run ./cmd/traingpt2/main.go

Run the testing script:

make test

This will run go run ./cmd/testgpt2/main.go

Tokenize input text, the implementation of this is incorrect. Need to do pair matching not tries
Very slow, need to improve performance.
It runs in WASM but using WebGPU bindings might be fun.
More refactoring.
Running as CLI.

See LICENSE for more details.