SVD is the one matrix decomposition that rules them all. Any matrix - any shape, any rank, symmetric or not - unconditionally breaks into three pieces. Every recommendation system, every image compression algorithm, and every transformer attention head in every LLM running today…
đź“Ť The Mathematics Behind Heat, Waves, Fluids and Modern Engineering
Tensors are used all over math and science to reveal hidden geometric truths. What are they?
The thing that makes 4-bit AI work is a grid of nothing but +1s and -1s called a Hadamard matrix. Here's how a 159 year old matrix ended up rescuing modern AI model performance: In 1867 James Joseph Sylvester was studying how to color tiles. He found you could build square grids…
Comprehensive taxonomy of linear algebra algorithms with Python & Ruby implementations. Covers vector operations, eigenvalues, SVD, decompositions, and BLAS/LAPACK routines.
The idea isn't novel, but presents major challenges. Tensordyne thinks it has solved them, and promises massive speed and efficiency gains as a result.
By using Logarithmic Number Systems (LNS), you can slash both the energy cost and silicon area cost of AI inference computations. How much? Tensordyne says that its upcom
Because AI compute is primarily composed of matrix multiplication, this radically simplifies the workload, and allows the functional units of Tensordyne's chip to be significantly smaller. In turn, more of the die can be SRAM cache (6x as much versus the competition), which improves both…
There is hardly any theory which is more elementary than linear algebra, in spite of the fact that generations of professors and textbook writers have obscured its simplicity by preposterous calcul...
This article explains how these four things fit together and shows some examples of what they are used for. Derivatives Derivatives are the most fundamental concept in calculus. If you have a funct…
Matrix algebra for a data scientist
Matrix Product Interpretations and Visualizations: Learn Linear Algebra from scratch. Build a foundation for Machine Learning and other key technologies.
MatrixCalculus provides matrix calculus for everyone. It is an online tool that computes vector and matrix derivatives (matrix calculus).
Part 4: A comprehensive step-by-step guide to solving a linear system with LU Decomposition
An eigenvalue of a square matrix $LATEX A$ is a scalar $latex \lambda$ such that $latex Ax = \lambda x$ for some nonzero vector $latex x$. The vector $latex x$ is an eigenvector of $LATEX A$ and it…
Understand how linear programming can be the most powerful tool for a supply chain continuous improvement engineer
In this article, we discuss the importance of linear algebra in data science and machine learning.
Quantifying Uncertainty in Computation.
This article gives an introduction to vector norms, vector distances and their application in the field of data science
In January 2016, I was honored to receive an “Honorable Mention” of the John Chambers Award 2016. This article was written for R-bloggers, whose builder, Tal Galili, kindly invited me to write an introduction to the rARPACK package. A Short Story of rARPACK Eigenvalue…
Efficient matrix multiplication · GitHub
Multiplying matrices is among the most fundamental and compute-intensive operations in machine learning. Consequently, there has been significant work on efficiently approximating matrix...
Poisson's Equation is an incredibly powerful tool...
Linear algebra is essential to deep learning and scientific computing, and it’s always been a core part of PyTorch. PyTorch 1.9 extends PyTorch’s support for linear algebra operations with the torch.linalg module. This module, documented here, has 26 operators, including faster…
Current custom AI hardware devices are built around super-efficient, high performance matrix multiplication. This category of accelerators includes the
Free online textbook of Jupyter notebooks for fast.ai Computational Linear Algebra course - fastai/numerical-linear-algebra
Linear algebra is foundational in data science and machine learning. Beginners starting out along their learning journey in data science--as well as established practitioners--must develop a strong familiarity with the essential concepts in linear algebra.
The geometric intuition behind determinants could change how you think about them.
A recent paper set the fastest record for multiplying two matrices. But it also marks the end of the line for a method researchers have relied on for decades to make improvements.
Intuitive Math Descriptions
Most of us last saw calculus in school, but derivatives are a critical part of machine learning, particularly deep neural networks, which are trained by optimizing a loss function. This article is an attempt to explain all the matrix calculus you need in order to understand the…
All of the Linear Algebra Operations that You Need to Use in NumPy for Machine Learning. The Python numerical computation library called NumPy provides many linear algebra functions that may be useful as a machine learning practitioner. In this tutorial, you will discover the…