Research News
AI research covers academic papers, breakthroughs and findings from labs like DeepMind, Berkeley AI Research and Stanford HAI. This page aggregates the latest peer-reviewed and preprint AI science.
783 articles · updated 24/7 from 100+ sources
Build Production-Ready LLMs From Scratch
Big news! I am now partnering with Maven as an instructor to teach the Build Production-Ready LLMs From…
Zhipu.AI’s Open-Source Power Play: Blazing-Fast GLM Models & Global Expansion Ahead of Potential IPO
Beijing, China – April 15, 2025 – In a strategic move that underscores its technological prowess and…
DeepSeek Signals Next-Gen R2 Model, Unveils Novel Approach to Scaling Inference with SPCT
DeepSeek AI, a prominent player in the large language model arena, has recently published a research paper…
Defending against Prompt Injection with Structured Queries (StruQ) and Preference Optimization (SecAlign)
Recent advances in Large Language Models (LLMs) enable exciting LLM-integrated applications. However, as LLMs…
Chapter 4 of The Big Book of Large Language Models is Here!
Chapter 4 of the Big Book of Large Language Models is finally here!That was a difficult chapter to write!…
Reduce AI Model Operational Costs With Quantization Techniques
Model quantization is becoming a core strategy for training and deployment! I am excited to introduce you to…
Analyzing long agent transcripts (Docent)
This is a brief overview of a recent release by Transluce. You can see the full write-up on the Transluce…
How To Construct Self-Attention Mechanisms For Arbitrary Long Sequences
With Gemini models having a 2M tokens context size and Claude having a 200K tokens context size while having…
How To Improve Decoding Latency With Faster Self-Attention Mechanisms
In LLMs, handling large sequences is not enough, we need to make sure the decoding process is fast. Here we…
How To Reduce The Memory Usage Of The Self-Attention
With a bit of magic, we take a very inefficient computation like the Self-Attention and make it super…
How To Linearize The Attention Mechanism!
Today, we talk about how to engineer attention mechanisms in O(n) complexity instead of O(n2). This…
Understanding The Sparse Transformers!
The First Sparse Attention: Sparse TransformersChoosing Sparsity Efficiently: ReformerLocal vs Global…
Attention Is All You Need: The Original Transformer Architecture
This newsletter is the latest chapter of the Big Book of Large Language Models. You can find the preview…
Introducing The Big Book of Large Language Models!
For the past years, I have been creating educational content around machine learning and, specifically, large…
Shape, Symmetries, and Structure: The Changing Role of Mathematics in Machine Learning Research
What is the Role of Mathematics in Modern Machine Learning?The past decade has witnessed a shift in how…
Introducing Transluce — A Letter from the Founders
We are launching an independent research lab that builds open, scalable technology for understanding AI…
Augmenting Statistical Models with Natural Language Parameters
This is a guest post by my student Ruiqi Zhong, who has some very exciting work defining new families of…
What's Missing From LLM Chatbots: A Sense of Purpose
LLM-based chatbots’ capabilities have been advancing every month. These improvements are mostly measured by…
We Need Positive Visions for AI Grounded in Wellbeing
IntroductionImagine yourself a decade ago, jumping directly into the present shock of conversing naturally…
Financial Market Applications of LLMs
The AI revolution drove frenzied investment in both private and public companies and captured the public’s…
A Brief Overview of Gender Bias in AI
AI models reflect, and often exaggerate, existing gender biases from the real world. It is important to…
Mamba Explained
The State Space Model taking on TransformersRight now, AI is eating the world.And by AI, I mean Transformers.…
Car-GPT: Could LLMs finally make self-driving cars happen?
In 1928, London was in the middle of a terrible health crisis, devastated by bacterial diseases like…
Do text embeddings perfectly encode text?
The rise of the vector databaseAs a result of the rapid advancement of generative AI in recent years, many…