nlp

Transformers - Part 1 NLP

Transformer architectures have unlocked tremendous potential in the context of Machine Learning problems. It has become the basic building block for learning and generating all modalities: language, vision, speech. But what changed with Transformers? We had kernel methods available for decades. In short, transformers allows for efficient context-aware learning. A

Progress on LLMs

The field of NLP has made tremendous progress in last 10 years. These Deep Learning approaches, from Word2Vec to GPT4, show remarkable capabilities can be achieved through representation learning. Tracing the journey back 5 years shows progress has mostly been to scale and engineer these LLMs on data and compute.