Our transition from heuristic to goal-based software is just beginning, and while we have lessons from the content-era (search, ads, feed) on applying predictive ML to consumer facing products, GenAI opens up a whole new vector of product application of Machine Learning (AI) tech. Traditional software practices focus on deterministic
There are few narratives on the impact of AI*. I’d like to propose 1 more. 1. AI is a Sustaining technology 2. AI is a Disruptive technology Sustaining Technology Under this view, advantage lies with the incumbent. Existing businesses benefit due to their distribution, proprietary datasets, and infrastructure advantages.
Ethereum infrastructure is constantly evolving. The big change in 2022 was the move from Proof-of-Work (PoW) to Proof-of-Stake (PoS) consensus. This upgrade also came with a big architectural shift: from monolith to modular. The direction of modular architecture starts decoupling responsibilities across various parts of the network, creating new roles
Public Cloud Infrastructure adoption is expected to grow at CAGR 20%. What’s more interesting is seeing whether it can sustain this growth. * 2024 Revenue is forecasted to reach over $300B. A 20% CAGR is stellar for an industry this large. * Its biggest component revolves around data - Analytics, Warehousing,
I was surprised to find that Github Copilot still doesn't support answering questions over your entire codebase through @workspace /explain command. This feature seems to be part of their roadmap. It got me thinking what the real challenges are behind scaling reasoning across entire codebase given that they
How could copyright laws and the legal intellectual property framework impact the development of AI technology? Generative AI has resurfaced questions around intellectual property rights in the era of AI based automation. Many of the GenAI products (OpenAI, Stability, and MidJourney) are already facing lawsuits around copyright infringement the precedents
A very high-level summary of RL. Dall-e's somewhat inaccurate representation of RL state-space RL is an approach to efficiently search state-space of a well-defined system. We can think of the system as a graph with states (vertex) and actions (edges) that lead to change in state. In its
Don't manage AI or research they way we'd manage shipping features. 🔸Improving AI systems leans towards research and exploration of ideas. Research-y vs engineering projects have to be managed distinctively, where the former benefits from freedom, iterations, while aligning on higher-level objectives, and product engineering from
I've been thinking about reasoning abilities in LLMs. To what degree do they exhibit reasoning abilities vs reformulation via statistical patterns? After all, they are increasingly referred to as "reasoning-engines" these days, but do they reason? I also came across an interesting post that solidified some
Transformer architectures have unlocked tremendous potential in the context of Machine Learning problems. It has become the basic building block for learning and generating all modalities: language, vision, speech. But what changed with Transformers? We had kernel methods available for decades. In short, transformers allows for efficient context-aware learning. A
The field of NLP has made tremendous progress in last 10 years. These Deep Learning approaches, from Word2Vec to GPT4, show remarkable capabilities can be achieved through representation learning. Tracing the journey back 5 years shows progress has mostly been to scale and engineer these LLMs on data and compute.
2022 was a remarkable year for the field of Machine Learning AI as years of landmark research ideas culminated into applications with strong PMF. Image generation and dialog systems surpassed most of our expectations, shifting near-term upper-bound on coherence and reasoning abilities of these approaches. Products like Stable Diffusion, Midjourney,