AI — 300 free lessons

ml · neural nets · transformers · llms · agents · rag · eval

Start lesson 1 →

AI Tutorial

1. What is AI · The Field 2. AI History · The Three Winters 3. Symbolic vs Statistical · The Two Camps 4. Machine Learning · Learning from Data

AI Learning Types

5. Supervised Learning · Labeled Data 6. Unsupervised Learning · Finding Structure 7. Reinforcement Learning · Reward Signals 8. Self-Supervised Learning · The Modern Shift

Classical ML

9. Classification vs Regression 10. Clustering · K-Means · Dbscan 11. Dimensionality Reduction · Pca · T-Sne · Umap 12. Linear Regression · The Simplest Model 13. Logistic Regression · Binary Classifier 14. Decision Trees · Gini · Entropy 15. Random Forests · Ensemble of Trees 16. Gradient Boosting · Xgboost · Lightgbm 17. Svm · Support Vector Machines 18. Naive Bayes · Probability Over Features 19. Knn · K-Nearest Neighbors

Neural Networks

20. Neural Networks · The Perceptron Origin 21. Activation Functions · Relu · Sigmoid · Tanh 22. Loss Functions · Mse · Cross-Entropy 23. Gradient Descent · The Optimizer 24. Backpropagation · How Gradients Flow 25. Learning Rate · The Most Important Knob 26. Optimizers · Sgd · Adam · Adamw 27. Regularization · L1 · L2 · Dropout 28. Batch Norm · Layer Norm · Group Norm 29. Overfitting · The Bias-Variance Trade

AI Training Methodology

30. Cross-Validation · K-Fold · Stratified 31. Train · Val · Test Splits 32. Data Augmentation · Flips · Crops · Mixup 33. Feature Engineering · The Old Craft

AI Embeddings

34. Embeddings · Turning Things Into Vectors 35. Word2vec · The Breakthrough 36. Glove · Subword Embeddings

AI Architectures

37. Cnns · Convolutional Neural Networks 38. Cnns · Filters · Pooling · Stride 39. Rnns · Recurrence · Vanishing Gradients 40. Lstms · Gates · Long-Term Memory 41. Grus · Simpler Gated Recurrence 42. Seq2seq · Encoder-Decoder 43. Attention · The Seq2seq Fix 44. Transformers · Attention is all you Need 45. Self-Attention · Queries · Keys · Values 46. Multi-Head Attention · Parallel Views 47. Positional Encoding · Giving Order Back 48. Encoder-Only · Bert Family 49. Decoder-Only · GPT Family 50. Encoder-Decoder · T5 · Bart

Tokens & Scale

51. Tokenization · Bpe · Wordpiece · Unigram 52. Context Windows · How Much Fits at Once 53. Scaling Laws · Params · Data · Compute 54. Emergent Abilities · The Surprise Phase

Pretraining & Tuning

55. Pretraining · Next-Token Prediction 56. Fine-Tuning · Adapting a Base Model 57. Lora · Parameter-Efficient Tuning 58. Rlhf · Reinforcement Learning from Human Feedback 59. Dpo · Direct Preference Optimization 60. Constitutional AI · Self-Critique Training 61. Instruction Tuning · Making Models Follow

Prompting

62. Prompt Engineering · The New Programming 63. Zero-Shot · One-Shot · Few-Shot 64. Chain of Thought · Think Before Answering 65. Tree of Thoughts · Branching Reasoning 66. React · Reason + Act Loops 67. Self-Consistency · Sample and Vote

RAG & Retrieval

68. Retrieval Augmented Generation · RAG Basics 69. RAG · Vector Stores · Embeddings · Chunking 70. RAG · Hybrid Search · Rerank

Agents

71. Agents · LLMS with Tools 72. Tool Use · Function Calling 73. Multi-Agent Systems · Roles + Handoffs 74. Memory · Short Term · Long Term · Episodic

Evaluation

75. Evaluation · Benchmarks vs Vibes 76. Benchmarks · Mmlu · Gsm8k · Humaneval 77. Eval Harnesses · Lm-Eval · Helm · Big-Bench

Safety & Alignment

78. Hallucinations · Why Models Invent 79. Groundedness · Citations · Attribution 80. Red Teaming · Finding Failures on Purpose 81. Jailbreaks · Prompt Injection · Countermeasures 82. Safety · Refusals · Over-Refusals 83. Alignment · The Meta-Objective

Inference

84. Inference · Greedy · Sampling · Temperature 85. Top-K · Top-P · Repetition Penalty 86. Speculative Decoding · Faster Inference 87. Quantization · Int8 · Int4 · Ggml · Gguf 88. Distillation · Teacher to Student 89. Moe · Mixture of Experts

Multimodal

90. Multimodal · Vision + Language 91. Clip · Contrastive Image-Text Training 92. Image Generation · Diffusion Models 93. Diffusion · Forward · Reverse · Sampling 94. Controlnet · Guiding Generation 95. Text-To-Speech · Vits · Xtts · Elevenlabs 96. Speech Recognition · Whisper 97. Video Generation · Sora · Veo 98. Embeddings Models · OpenAI · Cohere · Bge 99. Vector Dbs · Pinecone · Weaviate · Chroma

MLOps

100. Mlops · Pipelines · Monitoring · Drift 101. Data Labeling · The Human Bottleneck 102. Synthetic Data · Self-Instruct · Distill 103. Context Caching · Cutting Cost 104. Structured Outputs · JSON Mode · Constrained 105. Function Calling · Tools as Schemas

Models & Computer Use

106. Computer Use · Operating the Desktop 107. Claude · GPT · Gemini · Llama · The Lineup 108. Open Weights vs Closed · The Divide 109. Local Inference · Ollama · Llama.cpp · Lmstudio 110. Fine-Tuning Hosted · OpenAI · Anthropic Platforms

Compute & Interpretability

111. Compute · Gpus · Tpus · Accelerators 112. Interpretability · Features · Circuits 113. Mechanistic Interpretability · Sae · Probing

AI Industry

114. AI Safety Institute · Evals · Frontier Rules 115. Agi · The Moving Target 116. Cost · Per-Token Economics 117. Latency · Ttft · Tps · Streaming 118. Carbon · The Energy Footprint 119. Ethics · Bias · Privacy · Displacement

AI References

120. References · Papers with Code · Arxiv · Sota

Transformer Internals Deep

121. Attention Math · Scaled Dot-Product from Scratch 122. The Kv Cache · Why Generation is Autoregressive 123. Flash Attention · IO-Aware Exact Attention 124. Rotary Embeddings · Rope · Relative Position 125. Alibi · Attention with Linear Biases 126. Grouped-Query Attention · Gqa · Mqa 127. Rmsnorm vs Layernorm · Pre-Norm vs Post-Norm 128. Swiglu · Gated Feed-Forward Networks 129. The Residual Stream · The Model’s Working Memory 130. Weight Tying · Input and Output Embeddings 131. Attention Sinks · The First-Token Anchor 132. Logit Soft-Capping · Taming the Output 133. The Transformer Block · End to End 134. Parameter Counting · Where the Weights Live

Attention & Long Context

135. Sliding-Window Attention · Local Context 136. Sparse Attention · Longformer · Bigbird 137. Linear Attention · Kernel Tricks 138. Ring Attention · Context Across Devices 139. Yarn · Ntk-Aware Context Extension 140. Position Interpolation · Stretching the Window 141. Lost in the Middle · Recency and Primacy Bias 142. Retrieval Heads · How Models Recall from Context 143. Streaming LLMS · Infinite-Context Attention 144. Kv Cache Compression · H2o · Scissorhands 145. Paged Attention · Virtual Memory for the Cache 146. Prompt Caching · Reusing the Prefix

Pretraining at Scale

147. The Pretraining Data Pipeline · Crawl to Tokens 148. Deduplication · Why Duplicates Hurt 149. Data Mixtures · Domain Weighting 150. Data Curriculum · Ordering the Corpus 151. Tokenizer Training · Vocabulary Size Trade-Offs 152. Chinchilla · Compute-Optimal Scaling 153. Maximal Update Parametrization · MuP 154. Learning-Rate Schedules · Warmup · Cosine · Wsd 155. Gradient Clipping · Taming Exploding Grads 156. Loss Spikes · Detection and Recovery 157. Checkpointing · Resuming a Giant Run 158. Training Stability · The Dark Art

Distributed Training

159. Data Parallelism · Replicate and All-Reduce 160. Tensor Parallelism · Splitting the Matmul 161. Pipeline Parallelism · Staged Layers 162. Zero & Fsdp · Sharding Optimizer State 163. 3D Parallelism · Combining the Three 164. Gradient Accumulation · Big Batches on Small Gpus 165. Mixed Precision · Bf16 · Fp16 · Loss Scaling 166. Fp8 Training · The H100 Frontier 167. Activation Checkpointing · Recompute to Save Memory 168. Collective Communication · Nccl · All-Reduce …169. Megatron & Deepspeed · The Training Stacks 170. Fault Tolerance · Surviving Node Failures

Post-Training Deep

171. Sft · Supervised Fine-Tuning Data Curation 172. The Rlhf Pipeline · Sft → Rm → Ppo 173. Reward Modeling · Learning Human Preferences 174. Ppo for LLMS · The Policy-Gradient Core 175. Dpo · Skipping the Reward Model 176. Ipo · Kto · Preference-Optimization Variants 177. Rejection Sampling · Best-Of-N Distillation 178. Rlaif · AI Feedback at Scale 179. Process Reward Models · Grading Each Step 180. Rlvr · Reinforcement Learning from Verifiable…181. Online vs Offline Preference Learning 182. Reward Hacking · Gaming the Proxy 183. Length Bias · The Verbosity Trap 184. Preference Data · Collection and Quality

Reasoning & Test-Time Compute

185. Chain of Thought · Reasoning in Tokens 186. Self-Consistency · Sample and Majority-Vote 187. Tree of Thoughts · Search Over Reasoning 188. Best-Of-N · Sampling with a Verifier 189. Verifier Models · Checking the Answer 190. Process Supervision · Rewarding the Path 191. O1-Style Reasoning · Long Internal Thought 192. Test-Time Scaling Laws · Compute at Inference 193. Mcts + LLM · Guided Search 194. Self-Refine · Critique and Revise 195. Multi-Agent Debate · Arguing to Truth 196. Budget-Aware Reasoning · When to Stop Thinking

Agent Architectures Deep

197. The Agent Loop · Perceive · Plan · Act · Observe 198. React Deep · Interleaving Reason and Action 199. Plan-And-Execute · Decompose Then Run 200. Reflexion · Learning from Failed Attempts 201. The Model Context Protocol · Mcp Tools 202. Function Calling Internals · Schemas to Calls 203. Agent Memory Systems · Scratchpad to Vector Store 204. Multi-Agent Orchestration · Supervisor + Workers 205. Agent Routing · Picking the Right Specialist 206. Environment Design · The Agent’s Sandbox 207. Computer-Use Agents · Pixels to Clicks 208. Browser Agents · Navigating the Web 209. Coding Agents · The Swe-Bench Frontier 210. Agent Evaluation · Trajectories not Just Outputs

RAG Deep

211. Chunking Strategies · Fixed · Semantic · Recursive 212. Embedding Models Deep · Choosing a Retriever 213. Hybrid Search · Bm25 + Dense Fusion 214. Rerankers · Cross-Encoders for Precision 215. Query Rewriting · Expansion and Decomposition 216. Hyde · Hypothetical Document Embeddings 217. Multi-Hop RAG · Chaining Retrievals 218. Graph RAG · Retrieval Over Knowledge Graphs 219. Contextual Retrieval · Prepending Context 220. Late Interaction · Colbert Token-Level Match 221. Agentic RAG · The Model Drives Retrieval 222. RAG Evaluation · Faithfulness and Relevance

Embeddings & Vector Search

223. Contrastive Learning · Pulling Pairs Together 224. Sentence Transformers · Sbert 225. Matryoshka Embeddings · Nested Dimensions 226. Hnsw · Navigable Small-World Graphs 227. Ivf · Inverted File Indexes 228. Product Quantization · Compressing Vectors 229. Ann Trade-Offs · Recall vs Latency 230. Vector Database Internals · Sharding and Filtering 231. Multimodal Embeddings · Shared Spaces 232. Embedding Drift · When to Re-Index

Inference Optimization Deep

233. Prefill vs Decode · The Two Inference Phases 234. Continuous Batching · Keeping the Gpu Full 235. Paged Attention · The Vllm Memory Engine 236. Speculative Decoding · Draft and Verify 237. Medusa & Eagle · Multi-Token Heads 238. Gptq & Awq · Post-Training Quantization 239. Smoothquant · Activation-Aware Quant 240. Fp8 Inference · The Throughput Frontier 241. Tensor-Parallel Inference · Sharding at Serve Time 242. Chunked Prefill · Interleaving Long Prompts 243. Disaggregated Serving · Prefill / Decode Split 244. Structured Decoding · Grammars and JSON 245. Throughput vs Latency · The Serving Trade-Off 246. Kv Cache Quantization · Shrinking the Cache

Beyond Transformers

247. Mixture of Experts · Sparse Activation 248. Switch Transformers · One Expert Per Token 249. Expert Routing · Load Balancing the Gates 250. State Space Models · The Ssm Idea 251. Mamba · Selective State Spaces 252. Rwkv · Rnn with Transformer-Level Quality 253. Hybrid Architectures · Attention + Ssm 254. Mixture of Depths · Dynamic Compute Per Token 255. Diffusion Language Models · Parallel Decoding 256. The Architecture Frontier · What is Next

Multimodal Deep

257. Vision Transformers · Patches as Tokens 258. Clip Deep · Contrastive Image-Text 259. Vision-Language Models · Llava · The Projector 260. Diffusion Math · Forward and Reverse Process 261. Latent Diffusion · Compressing to Latent Space 262. Diffusion Transformers · Dit 263. Flow Matching · The Rectified-Flow Path 264. Classifier-Free Guidance · Steering Strength 265. Image Tokenizers · Vq-Vae · Vqgan 266. Audio Models · Codecs · Tts · Asr 267. Video Diffusion · Temporal Consistency 268. Any-To-Any Models · Unified Multimodal

Evaluation Deep

269. Benchmark Design · Validity and Coverage 270. Contamination · When Test Data Leaks Into Training 271. LLM-As-Judge · Models Grading Models 272. Pairwise Comparison · Elo and Arenas 273. Rubric Grading · Structured Scoring 274. Agentic Benchmarks · Swe-Bench · Webarena 275. Capability Evals · What Can the Model Do 276. Safety Evals · What Will the Model Refuse 277. Statistical Significance · Error Bars on Evals 278. Eval Gaming · Overfitting to the Benchmark 279. Holistic Evaluation · Helm-Style Coverage 280. Human Evaluation · The Gold Standard and its Cost

Interpretability Deep

281. The Residual Stream · Reading the Model Internals 282. Induction Heads · In-Context Learning Circuits 283. Superposition · More Features Than Neurons 284. Sparse Autoencoders · Extracting Features 285. Feature Circuits · Tracing Computation 286. Activation Patching · Causal Tracing 287. The Logit Lens · Decoding Intermediate Layers 288. Probing Classifiers · What Layers Know 289. Steering Vectors · Editing Behavior at Inference 290. Dictionary Learning · The Monosemanticity Goal

Safety & Alignment Deep

291. The Limits of Rlhf · What Preferences Miss 292. Scalable Oversight · Supervising Superhuman Models 293. Weak-To-Strong Generalization 294. Deceptive Alignment · The Inner-Alignment Risk 295. Evaluation Awareness · Models Knowing They are…296. Jailbreak Taxonomy · How Guardrails Fail 297. Prompt Injection Defense · Trust Boundaries 298. Machine Unlearning · Removing Knowledge 299. Watermarking · Detecting AI Output 300. The Alignment Frontier · Open Problems