Articles | Natural Language Processing

Natural Language Processing

A six-part series on the evolution from classical language processing to modern large language models, and the transformer architecture that made it possible.

• Part 1: From Rule-Based NLP to the TransformerFifty years of language processing, the bottlenecks each generation left behind, and the architecture that resolved them.
• Part 2: Pre-training and the BERT/GPT DivergenceMasked language modeling, autoregressive generation, and the transfer learning paradigm.
• Part 3: Scaling Laws and Emergent AbilitiesWhat happens when language models grow by orders of magnitude.
• Part 4: Embeddings, Retrieval, and RAGVector representations, semantic search, and grounding language models in external knowledge.
• Part 5: Alignment and AdaptationInstruction tuning, RLHF, and the challenge of making language models useful and safe.
• Part 6: Agents, Reasoning, and Compound AITool use, multi-step reasoning, and the shift from single models to AI systems.