Research News
AI research covers academic papers, breakthroughs and findings from labs like DeepMind, Berkeley AI Research and Stanford HAI. This page aggregates the latest peer-reviewed and preprint AI science.
482 articles · updated 24/7 from 100+ sources
Build Production-Ready LLMs From Scratch Starting on July 12th!
Get ready! The latest iteration of the Build Production-Ready LLMs From Scratch live course is starting on…
MIT Researchers Unveil “SEAL”: A New Step Towards Self-Improving AI
The concept of AI self-improvement has been a hot topic in recent research circles, with a flurry of papers…
Researchers from PSU and Duke introduce “Multi-Agent Systems Automated Failure Attribution
Share My Research is Synced’s column that welcomes scholars to share their own research breakthroughs with…
AGI Is Not Multimodal
"In projecting language back as the model for thought, we lose sight of the tacit embodied understanding that…
Adobe Research Unlocking Long-Term Memory in Video World Models with State-Space Models
Video world models, which predict future frames conditioned on actions, hold immense promise for artificial…
Last Week to Register to the Build Production-Ready LLMs From Scratch Course!
This Saturday, we kick off the Build Production-Ready LLMs From Scratch course! This is the last week to…
DeepSeek-V3 New Paper is coming! Unveiling the Secrets of Low-Cost Large Model Training through Hardware-Aware Co-design
A newly released 14-page technical paper from the team behind DeepSeek-V3, with DeepSeek CEO Wenfeng Liang as…
DeepSeek Unveils DeepSeek-Prover-V2: Advancing Neural Theorem Proving with Recursive Proof Search and a New Benchmark
DeepSeek AI has announced the release of DeepSeek-Prover-V2, a groundbreaking open-source large language…
All About The Modern Positional Encodings In LLMs
The Positional Encoding in LLMs may appear somewhat mysterious the first time we come across the concept, and…
Join us for a Free LIVE Coding Event: Build The Self-Attention in PyTorch From Scratch
Next Friday, I am inviting you to join me for an exciting live coding event. It is a completely free event…
Can GRPO be 10x Efficient? Kwai AI’s SRPO Suggests Yes with SRPO
The remarkable success of OpenAI’s o1 series and DeepSeek-R1 has unequivocally demonstrated the power of…
Build Production-Ready LLMs From Scratch
Big news! I am now partnering with Maven as an instructor to teach the Build Production-Ready LLMs From…
Zhipu.AI’s Open-Source Power Play: Blazing-Fast GLM Models & Global Expansion Ahead of Potential IPO
Beijing, China – April 15, 2025 – In a strategic move that underscores its technological prowess and…
DeepSeek Signals Next-Gen R2 Model, Unveils Novel Approach to Scaling Inference with SPCT
DeepSeek AI, a prominent player in the large language model arena, has recently published a research paper…
Defending against Prompt Injection with Structured Queries (StruQ) and Preference Optimization (SecAlign)
Recent advances in Large Language Models (LLMs) enable exciting LLM-integrated applications. However, as LLMs…
Chapter 4 of The Big Book of Large Language Models is Here!
Chapter 4 of the Big Book of Large Language Models is finally here!That was a difficult chapter to write!…
Reduce AI Model Operational Costs With Quantization Techniques
Model quantization is becoming a core strategy for training and deployment! I am excited to introduce you to…
Analyzing long agent transcripts (Docent)
This is a brief overview of a recent release by Transluce. You can see the full write-up on the Transluce…
How To Construct Self-Attention Mechanisms For Arbitrary Long Sequences
With Gemini models having a 2M tokens context size and Claude having a 200K tokens context size while having…
How To Improve Decoding Latency With Faster Self-Attention Mechanisms
In LLMs, handling large sequences is not enough, we need to make sure the decoding process is fast. Here we…
How To Reduce The Memory Usage Of The Self-Attention
With a bit of magic, we take a very inefficient computation like the Self-Attention and make it super…
How To Linearize The Attention Mechanism!
Today, we talk about how to engineer attention mechanisms in O(n) complexity instead of O(n2). This…
Understanding The Sparse Transformers!
The First Sparse Attention: Sparse TransformersChoosing Sparsity Efficiently: ReformerLocal vs Global…
Attention Is All You Need: The Original Transformer Architecture
This newsletter is the latest chapter of the Big Book of Large Language Models. You can find the preview…