The mathematician Tai-Danae Bradley is using category theory to try to understand both human and AI-generated language.
Her lens is category theory,
A category consists of some objects, together with relationships between them, called morphisms.
A Blog post by Ksenia Se on Hugging Face
four advanced attention mechanisms which improve how models handle long sequences, cut memory use and make attention learnable
Self-Attention which lets each token “look” at all others
/https://tf-cmsv2-smithsonianmag-media.s3.amazonaws.com/filer_public/49/d4/49d4cd58-5332-4784-9f63-60895717fca3/mar2025_a12_mespoptamia.jpg)
In the 1850s, cuneiform was just a series of baffling scratches on clay, waiting to spill the secrets of the ancient civilizations of Mesopotamia
When I was a kid, I was fascinated by a traditional katana my grandfather had brought home from Japan in 1945. Years later, I decided it was time to find the heirloom’s rightful owner.
Engineers at the Naval Research Lab launched a spy satellite program called Parcae and revolutionized signals intelligence at the height of the Cold War. The program relied on computers to sift through intelligence data, providing a technological edge at a pivotal moment in the…
We picked 50 paper/models/blogs across 10 fields in AI Eng: LLMs, Benchmarks, Prompting, RAG, Agents, CodeGen, Vision, Voice, Diffusion, Finetuning. If you're starting from scratch, start here.
Note: gist: https://gist.github.com/bjpcjp/117ecf144c062c3c933f2690e7ece5c2
Compute costs scale with the square of the input size. That’s not great.
This is probably why Google charges twice as much, per token, for Gemini 1.5 Pro once the context gets longer than 128,000 tokens. Generating token number 128,001 requires comparisons with all 128,000 previous tokens, making it significantly more expensive than producing the first or 10th or 100th…
In a series of papers, Princeton computer scientist Tri Dao and several collaborators have developed FlashAttention, which calculates attention in a way that minimizes the number of these slow memory operations.
Visualisations and metrics from the Common Crawl Web Graph dataset
After mastering regex, tools like grep become much more useful.
The grep flags can roughly be divided by what to read, what to match, what to report, and in what context.
grep -r --include=*.json 'pattern' ./
There has been an increasing amount of fear, uncertainty and doubt (FUD) regarding AI Scaling laws. A cavalcade of part-time AI industry prognosticators have latched on to any bearish narrative the…
Other areas that deliver model improvements with more compute include Synthetic Data Generation, Proximal Policy Optimization (PPO), Functional Verifiers, and other training infrastructure for reasoning
we will outline the old pre-training scaling trend as well as the new scaling trends for post-training and inference time.
No one in my family wanted to talk about Harold’s life as a contract killer for the Mob. Then one day he called me.
In today’s world, you’ve probably heard the term “Machine Learning” more than once. It’s a big topic, and if you’re new to it, all the technical words might feel confusing. Let’s start with the basics and make it easy to understand. Machine Learning, a subset of Artificial…
Notes from the Latent Space paper club. Follow along or start your own! - eugeneyan/llm-paper-notes
A combination of classical search and machine learning may be the way forward
BM25 is a widely used algorithm for full text search. I wanted to understand how it works, so here is my attempt at understanding by re-explaining.
can I compare the BM25 scores of documents across multiple queries to determine which query the document best matches?
goal of a full text search algorithm is to take a query and find the most relevant documents
It’s time to take a comprehensive look at one of America’s most controversial plane crashes, and ask how it ended up that way.
Over the past two decades, Google has hosted two different internal platforms for predictions. Why did the first one fail — and will the other endure?
Prophit
Gleangen
Personal writing and other things by Nabeel Qureshi.
Aman's AI Journal | Course notes and learning material for Artificial Intelligence and Deep Learning Stanford classes.
Hey guys, welcome back to my R-tips newsletter. Supply chain management is essential in making sure that your company’s business runs smoothly. One of the key elements is managing inventory efficiently. Today, I’m going to show you how to estimate inve...
One way that first principles thinking fails is when you build your analysis up from a deficient set of base principles. Everything is correct and true, but you still end up mistaken. Here's how that looks like in practice.
Because sometimes you have to fact-check your grandmother
Books in Progress is what we call a “public drafting tool”: Drafts will be made available for comment from the public, allowing for direct collaboration between author and reader.
🖥 📊 🕹 🛠 A curated list of command line apps.