LLMs From Scratch
24 个课时
01 Tokenizers: BPE, WordPiece, SentencePiece
CODE QUIZ 2 OUTPUTS
✓ → 02 Building a Tokenizer from Scratch CODE QUIZ 1 OUTPUTS
✓ → 03 Data Pipelines for Pre-Training CODE QUIZ 1 OUTPUTS
✓ → 04 Pre-Training a Mini GPT (124M Parameters) CODE QUIZ 1 OUTPUTS
✓ → 05 Scaling: Distributed Training, FSDP, DeepSpeed CODE QUIZ 1 OUTPUTS
✓ → 06 Instruction Tuning (SFT) CODE QUIZ 1 OUTPUTS
✓ → 07 RLHF: Reward Model + PPO CODE QUIZ 1 OUTPUTS
✓ → 08 DPO: Direct Preference Optimization CODE QUIZ 1 OUTPUTS
✓ → 09 Constitutional AI and Self-Improvement CODE 1 OUTPUTS
✓ → 10 Evaluation: Benchmarks, Evals, LM Harness CODE QUIZ 2 OUTPUTS
✓ → 11 Quantization: Making Models Fit CODE QUIZ 1 OUTPUTS
✓ → 12 Inference Optimization CODE QUIZ 1 OUTPUTS
✓ → 13 Building a Complete LLM Pipeline CODE 1 OUTPUTS
✓ → 14 Open Models: Architecture Walkthroughs CODE 1 OUTPUTS
✓ → 15 Speculative Decoding and EAGLE-3 CODE 1 OUTPUTS
✓ → 16 Differential Attention (V2) CODE 1 OUTPUTS
✓ → 17 Native Sparse Attention (DeepSeek NSA) CODE 1 OUTPUTS
✓ → 18 Multi-Token Prediction (MTP) CODE 1 OUTPUTS
✓ → 19 DualPipe Parallelism CODE 1 OUTPUTS
✓ → 20 DeepSeek-V3 Architecture Walkthrough CODE 1 OUTPUTS
✓ → 21 Jamba — Hybrid SSM-Transformer CODE 1 OUTPUTS
✓ → 22 Async and Hogwild! Inference CODE 1 OUTPUTS
✓ → 25 Speculative Decoding and EAGLE CODE 1 OUTPUTS
✓ → 34 Gradient Checkpointing and Activation Recomputation CODE 1 OUTPUTS
✓ →