System-Technology Co-Evaluation of A7 CFET and A10 NSFET...

Complementary FETs (CFETs) extend nanosheet FET (NSFET) scaling by vertically stacking n- and p-type gate-all-around (GAA) devices, thereby shrinking standard-cell area. The performance gain, however, cannot be assessed from device metrics alone, as CFET layouts also introduce…

Note: Summary: This arXiv paper compares two next-generation chip technologies — A7 CFET and A10 NSFET — using a physics-based evaluation flow spanning from cell parasitics up to full-chip reliability. Approach: CFETs stack n- and p-type devices vertically to shrink cell area, but add extra cell-level resistance and capacitance. The authors built a flow linking calibrated device models, standard-cell…

ClusterMAX 3.0: The Industry Standard GPU Cloud Rating System Returns

In gory detail: reliability, performance, support, pricing—and, of course, security—in our most thorough analysis of GPU cloud providers globally.

wafer-ai/gpu-perf-engineering-resources: A curriculum for learning about gpu performance engineering, from scratch to what the frontier AI…

A curriculum for learning about gpu performance engineering, from scratch to what the frontier AI labs do - wafer-ai/gpu-perf-engineering-resources

Are Open Models Catching Up?

Comparing open vs. closed models across the eras of frontier models

What Happens When the Cost of Intelligence Drops 100x — CatalystNeuro

A given level of LLM intelligence costs about a thirtieth of what it did a year ago, and the rate of decline is accelerating. Using Artificial Analysis's measured cost per task, this post traces the falling price of each capability level, explains why cheaper intelligence leads…

A great deal of useful work does not require the smartest model available, but a model that is good enough, applied many thousands of times

At today’s prices a full pass over the corpus costs a little over a hundred dollars with a model whose capability was at the frontier in the spring. At the prices of this past March, the same pass with the same level of capability would have cost several thousand dollars, and a year ago that…

FareedKhan-dev/kimi-k3-in-c: A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS…

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU. - FareedKhan-dev/kimi-k3-in-c

How much of ML research is about AI safety, what is it about, and who's doing it? — LessWrong

How many AI safety papers are at the big ML conferences, what do they study, and who writes them? A comprehensive analysis. > Website: https://ai-saf…

Claude Code Sends 4.7x More Tokens Than OpenCode Before Reading Your Prompt

We measured what Claude Code and OpenCode spend before reading your prompt, then added instruction files, MCP servers and subagents to the bill.

https://open.substack.com/pub/semianalysis/p/deepseekv4-16t-day-0-to-day-43-performance?r=oc5d&utm_medium=ios

Nvidia's Nemotron 3 Ultra becomes the smartest open US model, but China still leads

According to benchmark platform Artificial Analysis, Nvidia's new Nemotron 3 Ultra is the most capable open AI model from the US to date.

Best AI Agents for Software Development Ranked: A Benchmark-Driven Look at the Current Field - MarkTechPost

The AI coding agent field in 2026 is more capable, more fragmented, and harder to benchmark than it looks. Claude Code leads on code quality at 87.6% SWE-bench Verified. GPT-5.5 tops Terminal-Bench at 82.7%. But the benchmark OpenAI itself declared contaminated in February 2026…

SWE-bench

stopped reporting SWE-bench Verified

Qwen Introduces Qwen3.7-Max: A Reasoning Agent Model With a 1M-Token Context Window - MarkTechPost

Alibaba's Qwen3.7-Max is a reasoning agent model with a 1M-token context window, scoring 56.6 on the Artificial Analysis Intelligence Index.

An Overview of AI Accelerators: MAC Operations, DRAM/SRAM, Performance Metrics, and Architecture

Learning from a classic AI accelerator - Eyeriss

A Coding Implementation to Master GPU Computing with CuPy, Custom CUDA Kernels, Streams, Sparse Matrices, and Profiling - MarkTechPost

Note: https://gist.github.com/bjpcjp/c85fa4214c1b59c3aa578a6e8e9d871c

AI IQ is here: a new site scores frontier AI models on the human IQ scale. The results are already dividing tech.

AI IQ ranks frontier AI models like ChatGPT, Claude and Gemini on the human IQ scale, sparking debate over how artificial intelligence should be measured.

The Coding Assistant Breakdown: More Tokens Please

Hands On With GPT 5.5, Opus 4.7, DeepSeek V4, Why Benchmarks Are Bad, and Who’s Going To Win

GPT-5.5 is now materially better

GLM-5.1, Qwen3.6-Plus, Kimi K2.6, Composer 2, and Gemini 3.1 Pro

The Optimization Ladder

Python loses every public benchmark by 21-875x. I took the exact problems people use to dunk on Python and climbed every rung of the optimization ladder -- from CPython version upgrades to Rust. Real numbers, real code, real effort costs.

CanIRun.ai — Can your machine run AI models?

Detect your hardware and find out which AI models you can run locally. GPU, CPU, and RAM analysis in your browser.

Cloud VM benchmarks 2026: performance / price

nvidia's B200 does 1,760 TFLOPS on a square GEMM and 4 TFLOPS on a skinny one. the shape of your matmul is the single biggest variable in…

Alibaba unveils new Qwen3.5 model for 'agentic AI era'

Alibaba on Monday unveiled a new artificial intelligence model Qwen 3.5 designed to execute complex tasks independently, with big improvements in performance and cost that the Chinese tech giant claims beat major U.S. rival models on several benchmarks.

Autonomous Driving: Assessment Of YOLO Algorithms (RMIT et al.)

A new technical paper titled “Advances in You Only Look Once (YOLO) algorithms for lane and object detection in autonomous vehicles” was published by RMIT University, Kyungpook National University, Deakin University and the RCA Robotics Laboratory, Royal College of Art. Abstract…

'Observational memory' cuts AI agent costs 10x and outscores RAG on long-context benchmarks

As AI agents move into production, teams are rethinking memory. Mastra’s open-source observational memory shows how stable context can outperform RAG while cutting token costs.

LLM Research Papers: The 2025 List (July to December)

In June, I shared a bonus article with my curated and bookmarked research paper lists to the paid subscribers who make this Substack possible.

LLM Research Papers: The 2025 List (July to December)

A curated list of LLM research papers from July–December 2025, organized by reasoning models, inference-time scaling, architectures, training efficiency, and...

Reasoning

Reinforcement Learning

GraphQL: the enterprise honeymoon is over

A production-tested take on GraphQL in enterprise systems, why the honeymoon phase fades, and when its complexity outweighs the benefits.

The 70% factuality ceiling: why Google’s new ‘FACTS’ benchmark is a wake-up call

Comparing the Top 7 Large Language Models LLMs/Systems for Coding in 2025

Compare the top 7 large language models and systems for coding in 2025. Discover which ones excel for software engineering tasks.

OpenAI GPT-5 / GPT-5-Codex Anthropic Claude 3.5 Sonnet / Claude 4.x Sonnet with Claude Code Google Gemini 2.5 Pro Meta Llama 3.1 405B Instruct DeepSeek-V2.5-1210 (with DeepSeek-V3 as the successor) Alibaba Qwen2.5-Coder-32B-Instruct Mistral Codestral 25.01

Core coding quality: HumanEval, MBPP / MBPP EvalPlus, code generation and repair quality on standard Python tasks. Repo and bug-fix performance: SWE-bench Verified (real GitHub issues), Aider Polyglot (whole-file edits), RepoBench, LiveCodeBench. Context and long-context behavior: Documented…

The Big LLM Architecture Comparison

From DeepSeek-V3 to Kimi K2: A Look At Modern LLM Architecture Design

this section, I’ll focus on two key architectural techniques introduced in DeepSeek V3 that improved its computational efficiency and distinguish it from many other LLMs:

Multi-Head Latent Attention (MLA)

Understanding the 4 Main Approaches to LLM Evaluation (From Scratch)

Multiple-Choice Benchmarks, Verifiers, Leaderboards, and LLM Judges with Code Examples

four common ways of evaluating trained LLMs in practice: multiple choice, verifiers, leaderboards, and LLM judges

training loss, perplexity, and rewards, but they are usually used internally during model development.)

319 Top Ecommerce Sites Ranked by User Experience Performance – Baymard

See the ranked UX performance of the 319 leading ecommerce sites in the US and Europe. The chart summarizes 100,000+ UX performance ratings.

AI Model & API Providers Analysis | Artificial Analysis

Comparison and analysis of AI models and API hosting providers. Independent benchmarks across key performance metrics including quality, price, output speed & latency.

A Technical Roadmap to Context Engineering in LLMs: Mechanisms, Benchmarks, and Open Challenges

Context engineering for large language models—frameworks, architectures, and strategies to optimize AI reasoning, and scalability

Rather than viewing context as a static string (the premise of prompt engineering),

The Leaderboard Illusion

Measuring progress is fundamental to the advancement of any scientific field. As benchmarks play an increasingly central role, they also grow more susceptible to distortion. Chatbot Arena has emerged as the go-to leaderboard for ranking the most capable AI systems. Yet, in this…

QOpt / QOBLIB - Quantum Optimization Benchmarking Library · GitLab

This is the ZIB GitLab instance

The Man Out to Prove How Dumb AI Still Is

François Chollet has constructed the ultimate test for the bots.

A look at the ARC-AGI exam designed by French computer scientist François Chollet to show the gulf between AI models' memorized answers and…

By Matteo Wong / The Atlantic. View the full context on Techmeme.

LLM Benchmarking: Fundamental Concepts | NVIDIA Technical Blog

The past few years have witnessed the rise in popularity of generative AI and large language models (LLMs), as part of a broad AI revolution.

What is METEOR score? - Dataconomy

METEOR Score is a metric used to evaluate the quality of machine translation based on precision, recall, word alignment, and linguistic flexibility.

LLM Leaderboard

Comparison and ranking the performance of over 30 AI models (LLMs) across key metrics including quality, price, performance and speed (output speed - tokens per second & latency - TTFT), context window & others.

aidanmclaughlin/AidanBench: Aidan Bench attempts to measure in LLMs.

Aidan Bench attempts to measure in LLMs. - aidanmclaughlin/AidanBench

AMD’s Instinct MI300X AI Throughput Performance & Latency Improved By 7x Wi

Nscale has tested AMD's flagship Instinct MI300X AI accelerator utilizing the GEMM tuning framework, achieving 7x faster performance.

Key Metrics for Evaluating Large Language Models (LLMs)

Evaluating Large Language Models (LLMs) is a challenging problem in language modeling, as real-world problems are complex and variable. Conventional benchmarks frequently fail to fully represent LLMs' all-encompassing performance. A recent LinkedIn post has emphasized a number…

Nvidia Conquers Latest AI Tests​

GPU maker tops new MLPerf benchmarks on graph neural nets and LLM fine-tuning

Open LLM Leaderboard : a Hugging Face Space by HuggingFaceH4

Track, rank and evaluate open LLMs and chatbots

Asking 60+ LLMs a set of 20 questions

Human-readable benchmarks of 60+ open-source and proprietary LLMs.

A Deep Dive Into LLaMA, Falcon, Llama 2 and Their Remarkable Fine-Tuned Ver

Exploring the Development of the 3 Leading Open LLMs and Their Chatbot Derivatives

Calculate Computational Efficiency of Deep Learning Models with FLOPs and M

In this article we will learn about its definition, differences and how to calculate FLOPs and MACs using Python packages.

Towards a Benchmarking Suite for Kernel Tuners

As computing system become more complex, it is becoming harder for programmers to keep their codes optimized as the hardware gets updated. Autotuners try to alleviate this by hiding as many archite…

How to Choose the Best Machine Learning Technique: Comparison Table