Linear Algebra is a branch of mathematics. It is studied at university level, by maths, physics and engineering students. An introductory Linear Algebra class is usually taken by all maths students, but higher lever Linear Algebra course options are available for students who want to specialise in related subjects later on.

This Entry considers some of the topics that are covered in an introductory Linear Algebra course. It does not attempt to describe any computational methods, or to give any proofs. The purpose of the Entry is to give people a flavour of the subject, not to teach it to them.

Very little mathematical expertise is assumed on the part of the reader here, although this Entry will probably be much more interesting to those with a mathematical inclination. In order to understand the material in a Linear Algebra course, nothing more than school level algebra is really required, although most Linear Algebra students will have already received a grounding in a variety of mathematical topics including proof and logic.

What's It All For?

For some people, studying Linear Algebra, or any branch of mathematics, is its own reward, and there is no need to think of applications. Outside the proverbial mathematical ivory tower, people use linear algebra for all sorts of things. It is an essential tool for engineers, physicists, biologists and all kind of analysts, who use it for everything from calculating drag on an aeroplane wing, to predicting the behaviour of subatomic particles, to understanding the interactions of different sectors of an economy.

Within mathematics, Linear Algebra is one of the pillars of the gateway to advanced study¹. Knowledge of Linear Algebra is assumed in the study of Real and Complex Analysis, Abstract Algebra, Topology, Differential Geometry, and so on, and so on.

Why 'Linear' Algebra?

What does the word 'linear' mean to a mathematician? A real world example can be helpful. The clearest way to understand something that's linear is to compare it with something that's non-linear.

Think about the fuel gauge in a car. When the fuel tank is filled up, the needle on the gauge points at the 'full' mark. As the fuel is used up, the needle drops towards the 'empty' mark. The idea behind an accurate gauge is that each litre of fuel used causes the needle to move by the same amount. That's linear behaviour. Even though the point of the needle moves along a curve in the gauge, if you drew a graph of the gauge's motion, with fuel use on one axis and distance moved by the needle on the other, you would get a straight line.

If the car is like most others however, the gauge actually behaves non-linearly. As the car is driven, the needle moves from 'full' down towards 'empty', but things aren't so simple in between. Depending on the shape of the fuel tank, some gauges' needles will linger near the top for a long time, and then drop relatively quickly to the ¼ mark, after which they move steadily towards empty. Others drop within the first few litres to the ½ mark, where they wait for a while before gradually descending to the big 'E'. These are examples of non-linear behaviour. One litre near the top of the tank doesn't produce the same needle-movement as one near the bottom of the tank. A graph of fuel use and a non-linear needle's motion would not be a straight line but would be some kind of curve.

Linear Algebra is the study of mathematical objects that act like a well-behaved fuel gauge.

Systems of Linear Equations

The jumping-off point for the study of Linear Algebra is something most people learn in secondary school - solving simultaneous equations. A linear equation is one like this: 5x = 45, or like: 2x - y = 7. What makes these linear is that they don't have any funny stuff like x², or the cosine of x, or any logarithms or square roots, or anything but multiplication or division by ordinary numbers, and addition and subtraction. A system of linear equations is just a group of them to be solved together. The system could contain just one equation, or any number of them.

Readers may remember that the first example above, 5x = 45, can be solved simply with division. Divide both sides of the equation by 5, and it becomes clear that x = 9. The second equation cannot be solved all by itself, because it has two unknowns: x and y. To solve for two unknowns, you need two different² equations, which you can then solve simultaneously. In general, if you have n unknowns, then you need at least n equations.

If you don't have enough equations to solve for all the unknowns, then there will be multiple solutions. For example, the equation from above, 2x - y = 7 will be true if x = 4 and y = 1, but it is also true for x = 6 and y = 5 or x = 1.2 and y = -4.6. You can actually assign any value at all to x, as long as y is given the compatible value y = 2x - 7. This equation therefore has an infinite number of solutions.

When an equation like the above example has two unknowns, or variables, the solutions can be visualised as a line in two-dimensional space (like the graph of the fuel gauge). A system of linear equations is then visualised as parallel or intersecting lines in the 2D space. Similarly, if the equations in a linear system contain three variables, they can be visualised as two-dimensional planes in three-dimensional space (such as a ramp meeting the horizontal floor at the top of a set of steps). This is fine for equations with only two or three variables, but once there are four or more, graphical representations become more difficult, if not impossible, to deal with. In Linear Algebra, it is much more common to leave the equation in the form 2x - y = 7 than to think about objects in space.

Sometimes, a system of more than one linear equation will have no solutions at all. For example:

2x + y = 10
4x + 2y = 12

There are no values of x and y that will make both of these true. If you'd like to convince yourself of that, it's a good exercise!

Any system of linear equations, with any number of unknowns, has either one solution, an infinite number of solutions, or no solution at all.

Solving Equations by Using a Matrix

The first thing you learn in Linear Algebra is how to solve systems of linear equations by using a matrix³. The last example from above, in a matrix, looks like this:

⌈ 2 1 10 ⌉

⌊ 4 2 12 ⌋

All we've done is remove the extra symbols, leaving only numbers behind, and put the whole thing in brackets. The reason you put equations into a matrix is because it makes it very easy to work with systems of any size, and to keep track of exactly what you're doing. It also saves writing lots of '+' and '=' signs.

Subtracting the top row from the bottom row twice yields a matrix that looks like:

⌈ 2 1 10 ⌉

⌊ 0 0 -8 ⌋

and if you put the = sign back in this says that 0 = -8 which is not true, so there are no x and y that satisfy the two equations simultaneously.

However, if we consider the other equations from earlier, 5x = 45 and 2x - y = 7, this gives a matrix:

⌈ 5 0 45 ⌉

⌊ 2 -1 7 ⌋

which is, after dividing the first row by 5 and subtracting rows from each other, equivalent to:

⌈ 1 0 9 ⌉

⌊ 0 1 11 ⌋

and that tells us that x = 9 and y = 11 are the only values that satisfy both equations at the same time.

Just by analyzing the matrix that represents a system, you can determine whether it has any solutions, and if so, what they are. You can use matrices to analyze any system of linear equations - any number of equations with any number of unknowns.

Fun With Matrices

Once you start playing with matrices, you find that they don't have to represent systems of equations. They can represent lots of things, or they can just be played with on their own, and made to dance and whirl according to their own weird choreography. Matrices are a little bit like numbers. You can add them to each other (if they're the same size), and you can multiply them by numbers. If the sizes match up in the right way, you can even 'multiply' them by each other⁴. Matrix multiplication is not like anything you will have encountered in maths before linear algebra. The most strikingly weird thing about it is that it's non-commutative.

What does that mean? Well, multiplication of ordinary numbers is commutative, because 6 × 9 is always the same thing as 9 × 6 and so on. The order of the terms doesn't matter. If A and B are matrices, on the other hand, then AB is not usually the same as BA. Multiplying them in one order is different from multiplying them in a different order. Here's a fun example illustrating how something can be non-commutative. (This example will make a lot more sense if the reader actually gets a book or a CD or something and tries it):

Take a book (or a CD, or something), and hold it in both hands, with one hand on each side. Now let the letter 'A' represent the motion of rotating the book one quarter turn clockwise, while keeping the same side facing up. If you start from the standard position, holding the spine on the left with the title facing you, rightside-up, then A should change the book to a position where you are holding the bottom of the book in your left hand, with the spine at the top and the front cover still facing you.

The letter 'B' will represent a different motion of the book, namely flipping it over. B always means flipping it in the same way, so that the part that was furthest from you ends up towards you, and whichever cover was on the bottom ends up on top. If you start with the book in standard position, then you'll end up with the spine still on the left, but now you're looking at the back cover, and it's upside-down.

Now for the non-commutative part. Start with the book in standard position, then do A, then do B (rotate, then flip). You should end up with the back cover up, spine at the bottom. Now, starting again from standard position, do B, and then do A (flip, then rotate). Now the book is back cover up, spine at the top. Motions of a book are non-commutative!

Linear Transformations

The motions of the book described above are examples of
linear transformations. Other examples of linear transformations are certain things you can do to an image with a simple computer graphics program. You can rotate it (like the book), you can make it bigger or smaller, you can stretch it lengthwise, leaving the width the same, or you can shear it⁵ to one side. These are all linear transformations. Fancy transformations, like adding waves or curls, are non-linear.

Any of these linear transformations can be represented by a matrix. For example, the matrix for clockwise rotation through 90° (the motion A from above) looks like this and means that what was horizontal is now vertical and vice versa:

⌈  0  1 ⌉

⌊ -1  0 ⌋

The motion B is represented by another matrix, which essentially tells us that what was at the top is now at the bottom and vice versa, but the left and right sides stay where they are:

⌈  1  0 ⌉

⌊ 0  -1 ⌋

Then motion A followed by B is represented by:

⌈ 1 0 ⌉ ⌈ 0 1 ⌉ = ⌈ 1 × 0 + 0 × (-1) 0 × 0 + 1 × (-1) ⌉ = ⌈ 0 -1 ⌉

⌊ 0 -1 ⌋ ⌊ -1 0 ⌋       ⌊ 0 × 0 + (-1) × (-1) 0 × 1 + (-1) × 0 ⌋       ⌊ 1 0 ⌋

And motion B followed by A is represented by:

⌈ 0 1 ⌉ ⌈ 1 0 ⌉ = ⌈ 0 × 1 + 1 × 0 0 × 0 + 1 × (-1) ⌉ = ⌈ 0 -1 ⌉

⌊ -1 0 ⌋ ⌊ 0 -1 ⌋       ⌊ (-1) × 1 + 0 × 0 (-1) × 0 + 0 × (-1) ⌋       ⌊ -1 0 ⌋

So this calculation shows what we saw from manipulating the book - the motion B followed by A is different from the motion A followed by B because the matrices AB and BA are not the same. If the calculation doesn't make sense, don't worry. The important thing is that a matrix represents a transformation, and that doing one transformation, and then another, is represented by multiplying the matrices together - in the right order!

Further Concepts

There are an awful lot of matrices out there, and it can be difficult to keep track of everything about them. For this reason, you spend a lot of time in a linear algebra class looking at special types of matrices, which you can begin to understand more deeply. One way of reducing the confusion is to restrict the discussion to square matrices - matrices that have as many rows as they have columns. The matrix above, representing the transformation of rotating the book, is 2×2, so it's square.

There are some nice things you can do with square matrices to learn more about them. One thing is to calculate a number called the determinant of a matrix. Determinants are nice, because however big a matrix is, and however many numbers it contains, its determinant is always a single number that gives you some information about it.

What does a determinant tell you? Well, think about linear transformations, like in a photo-editing program. If you start out with a picture that has some area, in square centimetres, and then you apply some transformation to it, then the area of the new picture will equal the old area, multiplied by the determinant of the matrix you used. The determinant of A, the rotation matrix from the book example, is 1, because rotating something doesn't change its area.

There are other tools that can be used to examine the properties of square matrices, and a Linear Algebra course will cover them as well as techniques for working with other types of matrix. An important general concept that the course introduces students to is abstraction - starting with equations, the key information is captured by matrices, which can be studied in their own right and can arise in relation to many objects, not just equations.

If reading through these topics makes you hungry to know more, then you would probably enjoy taking a Linear Algebra course. Even if the subject doesn't appeal to you, we hope this Entry has given you a clearer idea of what it's all about, and what sort of ideas are dealt with in this area of mathematics.