Libraries: Language modeling with PyTorch (1)

[edoardob90]

Part 1 notebook introduces the fundamentals of neural network-based language modeling, from traditional bi-gram approaches to the simplest neural networks.

• Implementation of a single-layer neural network for character-level language modeling
• Comparison with the bigram model approach, highlighting similarities in performance but differences in flexibility
• Step-by-step explanation of the neural network pipeline:
  • One-hot encoding of character inputs
  • Forward pass through a weight matrix
  • Softmax transformation to obtain probability distributions
  • Loss calculation using negative log-likelihood
  • Backward pass for gradient computation
  • Weight updates using gradient descent
• Introduction to regularization
• Demonstration of sampling from the trained model

It's the first step of a step-by-step introduction/overview of language modeling using PyTorch library.

[edoardob90]

Left to do:

• Add more references to extra material
• Update Table of Contents
• Finalize the exercises

[edoardob90]

Left to do:

• Add solutions notebook

[despadam]

Also, this should be included in 00_index.ipynb

[despadam]

LGTM 👏

[Snowwpanda]

Looks nice, good work.

[Snowwpanda]

Suggested change

	"words = open('data/names.txt', 'r').read().splitlines()\n",
	"words = open('data/lm/names.txt', 'r').read().splitlines()\n",

names is in a subfolder