talkie - a 13B vintage language model from 1930

talkie is an inference library for the talkie 13B language model family developed by Alec Radford, Nick Levine, and David Duvenaud.

talkie-1930-13b-base is a 13b language model trained on pre-1931 English-language text.

talkie-1930-13b-it has been instruction-tuned using a novel instruction-following dataset built from pre-1931 reference works including etiquette manuals, letter-writing manuals, encyclopedias, and poetry collections. It has also undergone reinforcement learning using online DPO to improve instruction-following capabilities.

We also provide a 'modern' base model, talkie-web-13b-base, with the same architecture and training FLOPs as talkie-1930, but trained on FineWeb, to allow for controlled comparisons between modern and vintage models. Note that we need to be careful about the claims we make contrasting the behavior and capabilities of the models, because temporal coverage is not the only difference in the pretraining corpora. For example, the distribution of subject matters differs significantly.

See our blog post for details.

This package provides a simple Python API and CLI to download models from HuggingFace and run inference.

Models

Name	HuggingFace	Style	Description
talkie-1930-13b-base	talkie-lm/talkie-1930-13b-base	Base	1930-era base language model
talkie-1930-13b-it	talkie-lm/talkie-1930-13b-it	IT	1930-era instruction-tuned model
talkie-web-13b-base	talkie-lm/talkie-web-13b-base	Base	Same architecture as talkie-1930, but trained on FineWeb

Installation

git clone https://github.com/talkie-lm/talkie.git
cd talkie
uv sync

Requirements

• Python >= 3.11
• PyTorch >= 2.1
• CUDA GPU with >= 28 GB VRAM (bfloat16 inference)
• ~26-50 GB disk space per model

Quick Start

Python API

from talkie import Talkie

# Load a base model (downloads from HuggingFace on first use)
model = Talkie("talkie-1930-13b-base")

# Generate a completion
result = model.generate("If scientists discover life on other planets,", temperature=0.7, max_tokens=300)
print(result.text)

# Stream tokens
for token in model.stream("The effects of the automobile on public morality have"):
    print(token, end="", flush=True)

Chat (instruction-tuned model)

from talkie import Talkie, Message

model = Talkie("talkie-1930-13b-it")

# Single-turn
result = model.generate("Write an essay predicting what life will be like in the year 1960.", max_tokens=600)
print(result.text)

# Multi-turn chat
messages = [
    Message(role="user", content="What were the causes of the French Revolution?"),
]
result = model.chat(messages, temperature=0.7)
print(result.text)

# Stream a chat reply
messages.append(Message(role="assistant", content=result.text))
messages.append(Message(role="user", content="Which of those causes was the most significant?"))
for token in model.chat_stream(messages):
    print(token, end="", flush=True)

Pre-download models

from talkie import download_model

# Download before loading (useful for setup scripts)
download_model("talkie-1930-13b-base")

CLI

# Generate text
uv run talkie generate "Once upon a time" --model talkie-1930-13b-base -t 0.8

# Interactive chat
uv run talkie chat --model talkie-1930-13b-it

# Download a model
uv run talkie download talkie-1930-13b-base

# Download all models
uv run talkie download all

# List available models
uv run talkie list

License

Apache 2.0