Sign in

Sahil Thaker

Managing Research vs Engineering

Don't manage AI or research they way we'd manage shipping features. 🔸Improving AI systems leans towards research and exploration of ideas. Research-y vs engineering projects have to be managed distinctively, where the former benefits from freedom, iterations, while aligning on higher-level objectives, and product engineering from

Reasoning in LLMs

I've been thinking about reasoning abilities in LLMs. To what degree do they exhibit reasoning abilities vs reformulation via statistical patterns? After all, they are increasingly referred to as "reasoning-engines" these days, but do they reason? I also came across an interesting post that solidified some

Transformers - Part 1 NLP

Transformer architectures have unlocked tremendous potential in the context of Machine Learning problems. It has become the basic building block for learning and generating all modalities: language, vision, speech. But what changed with Transformers? We had kernel methods available for decades. In short, transformers allows for efficient context-aware learning. A

Progress on LLMs

The field of NLP has made tremendous progress in last 10 years. These Deep Learning approaches, from Word2Vec to GPT4, show remarkable capabilities can be achieved through representation learning. Tracing the journey back 5 years shows progress has mostly been to scale and engineer these LLMs on data and compute.

What changed in AI?

2022 was a remarkable year for the field of Machine Learning AI as years of landmark research ideas culminated into applications with strong PMF. Image generation and dialog systems surpassed most of our expectations, shifting near-term upper-bound on coherence and reasoning abilities of these approaches. Products like Stable Diffusion, Midjourney,

PyTorch Design Philosophy

This is an ongoing note around Pytorch and its quirks. PyTorch is a library for Neural Net model building using Python. It provides: a) an array-based programming model, known as Tensors b) an automatic differentiation engine, utilizing the chain-rule of differentiation c) hardware acceleration through cuDNN, parallelization, and graph optimizers

Blockchains and Scale: Part 1

Can current blockchain architectures scale securely? Blockchains are a secure database technology suitable for fintech or other peer-to-peer interactions. This ability to interact with peers without centralized intermediaries offers a unique value-proposition: sovereignty. The promise of Bitcoin is sovereign currency, free from any one government's policies. The promise

Why start now?

In some ways, it is back to the roots. Social media posts, even on on informative topics, tend to gravitate toward the platform objectives: followership and engagement. I contributed to one and I appreciate the nuances and care taken to craft a platform that is safe and engaging for all.