The Ultiamte Step-By-Step LLM Engineering Projects Roadmap

- Build a tokenizer
- Learn embeddings
- Implement RoPE / ALiBi
- Hand-wire attention
- Build MHA
- Build a Transformer block
- Train a …