Skip to content

Pull requests: KellerJordan/modded-nanogpt

New pull request New
32 Open 223 Closed
32 Open 223 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Tune AdamW baseline to 4950 steps
#353 opened Aug 14, 2026 by konstmish Contributor Loading…
1
MuonH fast-slow-decay schedule: 3125 steps (n=20, sig=0.00450)
#351 opened Aug 10, 2026 by Yufei-Gu-451 Loading…
Track 1: Canonical token masking (-0.8s?, -15 steps)
#350 opened Aug 6, 2026 by jvarho Contributor Loading…
New record: 1.079 min (64.72s) — new ANVIL optimizer + full-stack fp8 and schedule overhaul (−9.3s / −12.6% same-hardware)
#349 opened Aug 6, 2026 by devenpzak Loading…
11
Track 1: Topological Layer Dropout (-0.2, +8 steps)
#348 opened Jul 29, 2026 by jon123boss Loading…
1
Track 1: terminal-blended readout TailEMA (1.261 min, -0.895%)
#347 opened Jul 28, 2026 by deepmatmul • Draft
1
New record: 1384 steps / 1.312 min — Ember optimizer on embedding tables + earlier context window extension
#346 opened Jul 24, 2026 by katop1234 Loading…
1
Track 3: MuonH power-0.4 schedule reaches 3.28 at 3150 steps
#345 opened Jul 22, 2026 by mangocrazz Loading…
Track 1 (1.167 min on 8xH100, -5.15% same-node): Reduced-width QK and packed FP8 attention
#344 opened Jul 21, 2026 by Glitchfix Loading…
7
Track 3 (Per-optimizer SOTA): MuonH with minus-sqrt LR schedule — 3175 steps (n=9)
#343 opened Jul 20, 2026 by zzp1012 Loading…
1
Track 3 bi-maxwell kfac 2600 steps (n=12)
#341 opened Jul 17, 2026 by jn2clark Contributor Loading…
Track 3: bi-Maxwell momentum on the tuned Muon baseline -- 3210 steps (n=8)
#340 opened Jul 15, 2026 by orange4664 Loading…
Track 3: Bi-Maxwell dual-timescale momentum -- 2635 steps (n=8)
#339 opened Jul 14, 2026 by orange4664 Loading…
1
Track 3: Projected Tail-EMA, 2645 steps (n=19)
#338 opened Jul 14, 2026 by jn2clark Contributor Loading…
Improve shifted keys
#336 opened Jul 11, 2026 by vfedosov77 • Draft
2
Make key-offset shifts deterministic
#335 opened Jul 11, 2026 by ezfoom • Draft
Track 3: Soft-polar Muon -- 2765 steps
#332 opened Jun 29, 2026 by liuxuanzhang718 Loading…
Track 3: Tail-EMA tau retune -- 2660 steps (n=8)
#331 opened Jun 22, 2026 by didiforgithub Loading…
1
Track 3: SOAP-H baseline improved: 3025 steps (n=7)
#330 opened Jun 21, 2026 by Fullfix Contributor Loading…
Track 3: Shampoo with exponent -1/3 reached 3.28 in 3150 steps (n=4)
#327 opened Jun 15, 2026 by usamec Loading…
Track 3: Sign-based entrywise baselines (Lion, signSGD)
#326 opened Jun 12, 2026 by LauraGomezjurado Loading…
More portability
#314 opened May 25, 2026 by xor2k Contributor Loading…
5
Track 3: 3275 steps (n=8) Arbor Muon
#310 opened May 23, 2026 by ignorejjj Loading…
2
Track 3: Normalized Correction Reduce Muon 50 steps for free (3325, No hyperparameter tuning required, no extra memory needed, almost no computational cost)
#295 opened May 13, 2026 by Yuan-Jinghui Loading…
2
Track 3: Aurora + SOAP-MLP + Contra-Muon + NorMuon-lite: 3175 steps (n=4)
#286 opened May 7, 2026 by cmpatino Loading…
2
ProTip! Filter pull requests by the default branch with base:master.