Vectors and Matrices
The basic objects of linear algebra: directed quantities with magnitude and direction, and rectangular arrays of numbers with defined arithmetic.
A matrix is a rectangular array of numbers, written with rows and columns, e.g. a 2×3 matrix has 2 rows and 3 columns. Matrices add element-wise, but multiplication follows a row-times-column rule: the (i, j) entry of AB is the dot product of row i of A with column j of B, which requires A's column count to equal B's row count. Matrix multiplication is not commutative (AB ≠ BA in general), and it is associative, which enables fast exponentiation and efficient evaluation of expressions. The identity matrix I acts as the multiplicative identity, and the transpose flips rows and columns. A square matrix with a nonzero determinant has an inverse A⁻¹ satisfying AA⁻¹ = I.
[1 2] [5 6] [1·5+2·7 1·6+2·8] [19 22]
[3 4] [7 8] = [3·5+4·7 3·6+4·8] = [43 50]
Linear systems — many equations, many unknowns — are compactly written as Ax = b and solved with Gaussian elimination; the theory of when solutions exist is the rank/determinant theory of matrices. Matrices describe linear transformations: rotations, scaling, shears, and projections, which is why computer graphics represents every 3D operation as a 4×4 matrix multiplication. In machine learning, datasets are matrices, model parameters are matrices, and the gradient vectors of gradient descent and the word embeddings used in language models are both linear-algebra objects; nearly all of deep learning is applied matrix arithmetic.
Tags
linear algebra mathematics matrices vectors