TENSOR BY TENSOR
Sitemap
Browse every public learning page in the curriculum.
Start here and standalone guides
- Learn AI Fundamentals
- Attention Is All You Need: a careful paper walkthrough
- Build causal attention with a correct incremental KV cache
- Why softmax cross-entropy produces p − y
- Linear regression: the first complete model
- Tokenization, vocabularies, and sequence batches
- Bigram language models and probabilistic baselines
- Fixed-window neural language models
- Recurrent neural networks and backpropagation through time
- LSTM and GRU gates: controlling memory
- Attention scores, masking, and context
- Residual connections, normalization, and feed-forward layers
- Encoder-only models and masked-token objectives
- Decoder-only language modeling and generation
- Checkpoints, inference, and responsible model reuse
- Lab: implement multi-head attention
- Lab: train and evaluate embeddings
- Encoder-decoder transformers and teacher forcing
- KV caches, incremental attention, and correctness
- Lab: train a gated sequence predictor
- Rotary position embeddings and offsets
- Lab: build and test every transformer block
- Lab: train complete transformer models
- Lab: build a variational autoencoder
- Bag of words, TF-IDF, and classical text baselines
- From logistic regression to nonlinear MLPs
- Convolutional networks: locality and shared weights
- Datasets, batching, splits, and trustworthy metrics
- PyTorch & NLP Course Syllabus
- BERT, RoBERTa, DistilBERT, DeBERTa, and ModernBERT
- NLP applications: classification, tagging, retrieval, and reranking
- What is PyTorch
- Tensors, shapes, and data types
- Computation graphs
- Automatic differentiation
- Multilayer neural networks
- Data loaders
- The training loop
- Saving and loading models
- Training on GPUs
- Optimization, initialization, and generalization
- Bag of words
- Classification metrics and baselines
- Cosine similarity and text retrieval
- Cross-entropy and logits
- Train, validation, and test splits
- Embedding lookup and learned vectors
- Gradient descent and learning rates
- Logistic regression
- Mini-batches and data loading
- Nonlinear features and multilayer networks
- Regularization and generalization
- TF-IDF
- Tokenization and vocabulary design
- Weight initialization
- Capstone: build and defend your own transformer
Plain-language lessons
- Chain rule: ELI5
- Derivatives: ELI5
- Entropy: ELI5
- Functions: ELI5
- Gradients: ELI5
- Singular value decomposition: ELI5
- Vectors: ELI5
- Lab: implement multi-head attention: ELI5
- Attention scores, masking, and context: ELI5
- Automatic differentiation: ELI5
- Encoder-only models and masked-token objectives: ELI5
- Bigram language models and probabilistic baselines: ELI5
- Lab: build and test every transformer block: ELI5
- Residual connections, normalization, and feed-forward layers: ELI5
- Bag of words, TF-IDF, and classical text baselines: ELI5
- From logistic regression to nonlinear MLPs: ELI5
- Convolutional networks: locality and shared weights: ELI5
- Computation graphs: ELI5
- Fixed-window neural language models: ELI5
- Datasets, batching, splits, and trustworthy metrics: ELI5
- Data loaders: ELI5
- Encoder-decoder transformers and teacher forcing: ELI5
- Training on GPUs: ELI5
- KV caches, incremental attention, and correctness: ELI5
- Lab: train and evaluate embeddings: ELI5
- Linear regression: the first complete model: ELI5
- LSTM and GRU gates: controlling memory: ELI5
- Lab: train a gated sequence predictor: ELI5
- Decoder-only language modeling and generation: ELI5
- Checkpoints, inference, and responsible model reuse: ELI5
- BERT, RoBERTa, DistilBERT, DeBERTa, and ModernBERT: ELI5
- Multilayer neural networks: ELI5
- NLP applications: classification, tagging, retrieval, and reranking: ELI5
- Optimization, initialization, and generalization: ELI5
- Recurrent neural networks and backpropagation through time: ELI5
- Rotary position embeddings and offsets: ELI5
- Saving and loading models: ELI5
- Tensors, shapes, and data types: ELI5
- Tokenization, vocabularies, and sequence batches: ELI5
- The training loop: ELI5
- Capstone: build and defend your own transformer: ELI5
- Lab: train complete transformer models: ELI5
- Lab: build a variational autoencoder: ELI5
- What is PyTorch: ELI5
Detailed lesson topics
- Backpropagation in depth
- Beyond gradient descent
- Gradient descent
- Directional derivatives
- Finite differences
- The gradient vector
- The Hessian
- Why gradients?
- Optimizers
- Partial derivatives
- Gradient problems
- Gradient prerequisites
- Further reading
- Steepest ascent
- Backpropagation
- Chain rule
- Differentiation rules
- Derivatives
- Entropy
- Functions
- Gradients
- Hadamard products
- Jacobian matrices
- KL divergence
- Matrices
- Moving averages
- Singular value decomposition
- Vectors