babylm

Code & data for the EMNLP 2024 paper: Is Child-Directed Speech Effective Training Data for Language Models?

Code implementation for our paper "BERTtime Stories: Investigating the Role of Synthetic Story Data in Language Pre-training" as part of the 2024 BabyLM Challenge

UT Austin's submission to BabyLM Challenge

Code and data for the paper "Bringing Up a Bilingual BabyLM: Investigating Multilingual Language Acquisition Using Small-Scale Models"

Do state space models dream of recursive sheep? No, it was actually sinks all the way down.

Multilingual Age-Ordered CHILDES

Learning language by compression — an MDL language learner with no transformer and no backprop. Predicts, chunks, induces grammatical categories, and learns new words in one shot.

Looped transformers vs. multimodal degradation under the BabyLM 2026 100M-word budget (2x2 study)

ACL Computational Developmental Linguistics 2026: Barbenel et al (2026)

This repository contains the training and evaluation code accompanying the thesis "Learning with Less: Contrastive Weight Tying on the BabyLM Challenge" (Ino van de Wouw, VU Amsterdam, 2025). The project studies headless language models, models pretrained with Contrastive Weight Tying (CWT) (Godey et al., 2024) instead of a standard cross-entropy

🍼 I taught a baby-sized LLM to chat. It went... poorly. Here's the code, try it for yourself

Baby AI is a minimalistic artificial intelligence system designed to evolve from scratch by observing character sequences and forming abstract concepts.