Skip to main content

Discrete Mathematics for Computer Science: An open educational resource.

Ken Levasseur, Al Doerr, Michiel Smid, Oscar Levin, Charles M. Grinstead, J. Laurie Snell, Eric Lehman, F. Thomson Leighton, Albert R Meyer, Jeff Erickson, Kenneth P. Bogart, Carol Chritchlow, David Eck, OpenDSA Project, L.J. Miller

Section 11.2 Revisiting Matrix Arithmetic

In this section we will review and expand upon what we learned about matrices and matrix arithmetic in Chapter 7.

Subsection 11.2.1 Addition and Scalar Multiplication of Matrices

You have now solved systems of equations by writing them in terms of an augmented matrix and then doing row operations on this augmented matrix. It turns out such rectangular arrays of numbers are important from many other different points of view. Numbers are also called scalars. In this book, numbers will generally be either real or complex numbers. I will refer to the set of numbers as F sometimes when it is not important to worry about whether the number is real or complex. Thus F can be either the real numbers R or the complex numbers C. However, most of the algebraic considerations hold for more general fields of scalars.
A matrix is a rectangular array of numbers. Several of them are referred to as matrices. For example, here is a matrix.
\begin{equation*} \left( \begin{array}{c c c c} 1 & 2 & 3 & 4 \\ 5 & 2 & 8 & 7 \\ 6 & -9 & 1 & 2\\ \end{array} \right) \end{equation*}
The size or dimension of a matrix is defined as \(m \times n\) where \(m\) is the number of rows and \(n\) is the number of columns. The above matrix is a \(3 \times 4\) matrix because there are three rows and four columns. The first row is \((1 \, 2 \, 3 \, 4)\) , the second row is \((5 \, 2 \, 8 \, 7)\) and so forth. The first column is \(\left( \begin{array}{c} 1 \\ 5 \\ 6 \\ \end{array} \right) \text{.}\) When specifying the size of a matrix, you always list the number of rows before the number of columns. Also, you can remember the columns are like columns in a Greek temple. They stand upright while the rows just lie there like rows made by a tractor in a plowed field. Elements of the matrix are identified according to position in the matrix. For example, 8 is in position 2, 3 because it is in the second row and the third column. You might remember that you always list the rows before the columns by using the phrase Rowman Catholic. The symbol, \((a_{ij})\) refers to a matrix. The entry in the \(i^{th}\) row and the \(j^{th}\) column of this matrix is denoted by \(a_{ij}\) . Using this notation on the above matrix, \(a_{23} = 8, a_{32} = -9, a_{12} = 2,\) etc.
There are various operations which are done on matrices. Matrices can be added mul- tiplied by a scalar, and multiplied by other matrices. To illustrate scalar multiplication, consider the following example in which a matrix is being multiplied by the scalar 3.
\begin{equation*} 3 \left( \begin{array}{c c c c} 1 & 2 & 3 & 4 \\ 5 & 2 & 8 & 7 \\ 6 & -9 & 1 & 2\\ \end{array} \right) = \left( \begin{array}{c c c c} 3 & 6 & 9 & 12 \\ 15 & 6 & 24 & 21 \\ 18 & -27 & 3 & 6\\ \end{array} \right) \end{equation*}
The new matrix is obtained by multiplying every entry of the original matrix by the given scalar. If \(A\) is an \(m \times n\) matrix, \(-A\) is defined to equal \((-1) A\text{.}\)
Two matrices must be the same size to be added. The sum of two matrices is a matrix which is obtained by adding the corresponding entries. Thus
\begin{equation*} \left( \begin{array}{c c} 1 & 2 \\ 3 & 4 \\ 5 & 2\\ \end{array} \right) + \left(\begin{array}{c c} -1 & 4 \\ 2 & 8 \\ 6 & -4\\ \end{array} \right) = \left(\begin{array}{c c } 0 & 6 \\ 5 & 12 \\ 11 & -2\\ \end{array} \right) \end{equation*}
Two matrices are equal exactly when they are the same size and the corresponding entries are identical. Thus
\begin{equation*} \left( \begin{array}{c c} 0 & 0 \\ 0 & 0 \\ 0 & 0\\ \end{array} \right) \neq \left( \begin{array}{c c} 0 & 0 \\ 0 & 0 \\ \end{array} \right) \end{equation*}
because they are different sizes. As noted above, you write \((c_{ij})\) for the matrix \(C\) whose \(ij^{th}\) entry is \(c_{ij}\) . In doing arithmetic with matrices you must define what happens in terms of the \(c_{ij}\) sometimes called the entries of the matrix or the components of the matrix.
The above discussion stated for general matrices is given in the following definition.

Definition 11.2.1. Scalar Matrix Multiplication.

If \(A = (a_{ij})\) and \(k\) is a scalar, then \(kA = (ka_{ij})\text{.}\)

Example 11.2.2.

\begin{equation*} 7 \left(\begin{array}{c c} 2 & 0 \\ 1 & -4 \\ \end{array} \right) = \left( \begin{array}{c c} 14 & 0 \\ 7 & 28 \\ \end{array} \right)\text{.} \end{equation*}

Definition 11.2.3. Matrix Addition.

If \(A = (a_{ij})\) and \(B = (b_{ij})\) are two \(m \times n\) matrices. Then \(A+B = C\) where
\begin{equation*} C = (c_{ij}) \end{equation*}
for \(c_{ij} = a_{ij} + b_{ij}\) .

Example 11.2.4.

\begin{equation*} \left( \begin{array}{c c c} 1 & 2 & 3\\ 1 & 0 & 4 \\ \end{array} \right) + \left(\begin{array}{c c c} 5 & 2 & 3 \\ -6 & 2 & 1\\ \end{array} \right) = \left(\begin{array}{c c c} 6 & 4 & 6 \\ -5 & 2 & 5 \\ \end{array} \right) \end{equation*}
To save on notation, we will often use \(A_{ij}\) to refer to the \(ij^{th}\) entry of matrix \(A\text{.}\)

Definition 11.2.5. Zero Matrix.

The \(m \times n\) zero matrix is the \(m \times n\) matrix having every entry equal to zero. It is denoted by 0.

Example 11.2.6.

The \(2 \times 3\) zero matrix is \(\left( \begin{array}{c c c} 0 & 0 & 0\\ 0 & 0 & 0 \\ \end{array} \right)\text{.}\)

Definition 11.2.7. Matrix Equality.

Let \(A\) and \(B\) be two matrices. Then \(A = B\) means that the two matrices are of the same size and for \(A = (a_{ij})\) and \(B = (b_{ij})\) , \(a_{ij} = b_{ij} \)for all \(1 \leq i \leq m\) and \(1 \leq j \leq n\text{.}\)
The following properties of matrices can be easily verified. You should do so. These properties are called the vector space axioms.
  • Commutative Law Of Addition.
    \begin{equation*} A + B = B + A, \end{equation*}
  • Associative Law for Addition.
    \begin{equation*} (A + B) +C = A + (B +C) , \end{equation*}
  • Existence of an Additive Identity
    \begin{equation*} A + 0 = A, \end{equation*}
  • Existence of an Additive Inverse
    \begin{equation*} A + (-A) = 0. \end{equation*}
Also for \(\alpha, \beta\) scalars, the following additional properties hold.
  • Distributive law over Matrix Addition.
    \begin{equation*} \alpha (A + B) = \alpha A + \alpha B, \end{equation*}
  • Distributive law over Scalar Addition
    \begin{equation*} (\alpha + \beta ) A = \alpha A + \beta A, \end{equation*}
  • Associative law for Scalar Multiplication
    \begin{equation*} \alpha (\beta A) = \alpha\beta (A) , \end{equation*}
  • Rule for Multiplication by 1.
    \begin{equation*} 1A = A. \end{equation*}
As an example, consider the Commutative Law of Addition. Let \(A + B = C\) and \(B + A = D\text{.}\) Why is \(D = C\text{?}\)
\begin{equation*} C_{ij} = A_{ij} + B_{ij} = B_{ij} + A_{ij} = D_{ij} . \end{equation*}
Therefore, \(C = D\) because the \(ij^{th}\) entries are the same. Note that the conclusion follows from the commutative law of addition of numbers.

Subsection 11.2.2 Multiplication of Matrices

This is where things get interesting. Matrices can be thought of as a rule for making new vectors from old vectors.

Definition 11.2.8. Vectors.

Matrices which are \(n \times 1\) or \(1 \times n\) are called vectors and are often denoted by a bold letter. Thus the \(n \times 1\) matrix
\begin{equation*} \mathbf{x} = \left( \begin{array}{c} x_1 \\ \vdots \\ x_n \\ \end{array} \right) \end{equation*}
is also called a column vector. The \(1 \times n\) matrix
\begin{equation*} \mathbf{x} = \left( \begin{array}{c c c} x_1 & \dots & x_n \\ \end{array} \right) \end{equation*}
is called a row vector.
Although the following description of matrix multiplication may seem strange, it is in fact the most important and useful of the matrix operations. To begin with consider the case where a matrix is multiplied by a column vector. First consider a special case.
\begin{equation*} \left( \begin{array}{c c c} 1 & 2 & 3\\ 4 & 5 & 6 \\ \end{array} \right) \left( \begin{array}{c} 7 \\ 8 \\ 9\\ \end{array} \right) = \mathord{?} \end{equation*}
By definition, this equals
\begin{equation*} 7 \left( \begin{array}{c} 1\\ 4\\ \end{array} \right) + 8 \left( \begin{array}{c} 2\\ 5\\ \end{array} \right) + 9 \left( \begin{array}{c} 3\\ 6\\ \end{array} \right) = \left( \begin{array}{c} 50\\ 122\\ \end{array} \right). \end{equation*}
In more general terms,
\begin{align*} \begin{pmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ \end{pmatrix} \amp = x_1 \begin{pmatrix} a_{11}\\ a_{21}\\ \end{pmatrix} + x_2 \begin{pmatrix} a_{12}\\ a_{22} \\ \end{pmatrix} + x_3 \begin{pmatrix} a_{13}\\ a_{23}\\ \end{pmatrix} \\ \amp = \begin{pmatrix} a_{11}x_1 + a_{12}x_2 + a_{13}x_3\\ a_{21}x_1 + a_{22}x_2 + a_{23}x_3\\ \end{pmatrix}. \end{align*}
Thus you take \(x_1\) times the first column, add to \(x_2\) times the second column, and finally \(x_3\) times the third column. The above sum is called a linear combination of the given column vectors. These will be discussed more later. In general, a linear combination of vectors is just a sum consisting of scalars times vectors. When you multiply a matrix on the left by a vector on the right, the numbers making up the vector are just the scalars to be used in the linear combination of the columns as illustrated above.
More generally, here is the definition of how to multiply an \((m \times n)\) matrix times a \((n \times 1)\) matrix (column vector).

Definition 11.2.9.

Let \(A = A_{ij}\) be an \(m \times n\) matrix and \(\mathbf{v}\) be an \(n \times 1\) matrix,
\begin{equation*} \mathbf{v} = \begin{pmatrix} v_1 \\ \vdots\\ v_n\\ \end{pmatrix}, A = \begin{pmatrix} \mathbf{a}_1, \dots, \mathbf{a}_n \end{pmatrix} \end{equation*}
where \(\mathbf{a}_i\) is an \(m \times 1\) column vector. Then \(A\mathbf{v}\) written as
\begin{equation*} \begin{pmatrix} \mathbf{a}_1, \dots, \mathbf{a}_n \end{pmatrix} \begin{pmatrix} v_1 \\ \vdots\\ v_n\\ \end{pmatrix}, \end{equation*}
is the \(m \times 1\) column vector which equals the following linear combination of columns.
\begin{equation*} v_1 \mathbf{a}_1 + v_2 \mathbf{a}_2 + \dots + v_n\mathbf{a}_n \equiv \sum_{j=1}^n v_j\mathbf{a}_j \end{equation*}
If the \(j_{th}\) column of \(A\) is
\begin{equation*} \begin{pmatrix} A_{1j} \\ A_{2j} \\ \vdots\\ A_{mj}\\ \end{pmatrix}, \end{equation*}
then the sum above takes the form
\begin{equation*} v_1\begin{pmatrix} A_{11} \\ A_{21} \\ \vdots\\ A_{m1}\\ \end{pmatrix} + v_2 \begin{pmatrix} A_{12} \\ A_{22}\\ \vdots\\ A_{m2}\\ \end{pmatrix} + \dots + v_n\begin{pmatrix} A_{1n} \\ A_{2n}\\ \vdots\\ A_{mn}\\ \end{pmatrix} \end{equation*}
Thus the \(i_{th}\) entry of \(A\mathbf{v}\) is \(\sum_{j=1}^{n} A_{ij}v_j\text{.}\) Note that multiplication by an \(m \times n\) matrix takes an \(n \times 1\) matrix, and produces an \(m \times 1\) matrix (vector).
Here is another example.

Example 11.2.10.

Compute
\begin{equation*} \begin{pmatrix} 1 & 2 & 1 & 3\\ 0 & 2 & 1 & -2\\ 2 & 1 & 4 & 1\\ \end{pmatrix} \begin{pmatrix} 1 \\ 2\\ 0\\ 1\\ \end{pmatrix} \end{equation*}
First of all this is of the form \((3 \times 4) (4 \times 1)\) and so the result should be a \((3 \times 1)\) . Note how the inside numbers cancel. To get the element in the second row and first and only column, compute
\begin{align*} \sum_{k=1}^4 a_{2k}v_k \amp = a_{21}v_1 + a_{22}v_2 + a_{23}v_3 + a_{24}v_4 \\ \amp = 0 \times 1 + 2 \times 2 + 1 \times 0 + (-2) \times 1 = 2. \end{align*}
You should do the rest of the problem and verify
\begin{equation*} \begin{pmatrix} 1 & 2 & 1 & 3\\ 0 & 2 & 1 & -2\\ 2 & 1 & 4 & 1\\ \end{pmatrix} \begin{pmatrix} 1 \\ 2\\ 0\\ 1\\ \end{pmatrix} = \begin{pmatrix} 8 \\ 2\\ 5\\ \end{pmatrix} \end{equation*}
The next task is to multiply an \(m \times n\) matrix times an \(n \times p\) matrix. Before doing so the following may be helpful.
For \(A\) and \(B\) matrices, in order to form the product, \(AB\text{,}\) the number of columns of \(A\) must equal the number of rows of \(B\text{.}\) Thus the form of the product must be
\begin{equation*} (m \times n)(n \times p) = m \times p. \end{equation*}
Not the two outside numbers give the size of the product. Remember: If the two middle numbers don’t match, you can’t multiply the matrices!

Definition 11.2.11.

When the number of columns of \(A\) equals the number of rows of \(B\) the two matrices are said to be conformable and the product \(AB\) is obtained as follows. Let \(A\) be an \(m \times n\) matrix and let \(B\) be an \(n \times p\) matrix. Then \(B\) is of the form
\begin{equation*} B = (\mathbf{b}_1 , \dots , \mathbf{b}_p) \end{equation*}
where \(\mathbf{b}_k\) is an \(n \times 1\) matrix or column vector. Then the \(m \times p\) matrix \(AB\) is defined as follows:
\begin{equation*} AB \equiv (A\mathbf{b}_1, \dots , A\mathbf{b}_p) \end{equation*}
where \(A\mathbf{b}_k\) is an \(m \times 1\) matrix or column vector which gives the \(k^{th}\) column of \(AB\text{.}\)

Example 11.2.12.

Multiply the following:
\begin{equation*} \begin{pmatrix} 1 & 2 & 1 \\ 0 & 2 & 1 \\ \end{pmatrix} \begin{pmatrix} 1 & 2 & 0\\ 0 & 3 & 1\\ -2 & 1 & 1\\ \end{pmatrix} \end{equation*}
Solution.
The first thing you need to check before doing anything else is whether it is possible to do the multiplication. The first matrix on left is a \(2 \times 3\) and the second matrix on right is a \(3 \times 3\text{.}\) Therefore, is it possible to multiply these matrices. According to the above discussion it should be a \(2 \times 3\) matrix of the form
\begin{equation*} \begin{pmatrix} \overbrace{ \begin{pmatrix} 1 & 2 & 1\\ 0 & 2 & 1\\ \end{pmatrix} \begin{pmatrix} 1 \\ 0\\ -2\\ \end{pmatrix}}^{\text{First column}}, & \overbrace{ \begin{pmatrix} 1 & 2 & 1\\ 0 & 2 & 1\\ \end{pmatrix} \begin{pmatrix} 2 \\ 3\\ 1\\ \end{pmatrix}}^{\text{Second column}}, & \overbrace{ \begin{pmatrix} 1 & 2 & 1\\ 0 & 2 & 1\\ \end{pmatrix} \begin{pmatrix} 0 \\ 1\\ 1\\ \end{pmatrix}}^{\text{Third column}} \end{pmatrix} \end{equation*}
You know how to multiply a matrix times a vector and so you do so to obtain each of the three columns.
Answer.
\begin{equation*} \begin{pmatrix} 1 & 2 & 1\\ 0 & 2 & 1\\ \end{pmatrix} \begin{pmatrix} 1 & 2 & 0\\ 0 & 3 & 1\\ -2 & 1 & 1\\ \end{pmatrix} = \begin{pmatrix} -1 & 9 & 3 \\ -2 & 7 & 3 \\ \end{pmatrix} \end{equation*}

Example 11.2.13.

Multiply the following,
\begin{equation*} \begin{pmatrix} 1 & 2 & 0\\ 0 & 3 & 1\\ -2 & 1 & 1\\ \end{pmatrix} \begin{pmatrix} 1 & 2 & 1 \\ 0 & 2 & 1 \\ \end{pmatrix} \end{equation*}
Solution.
First check if it is possible. This is of the form \((3 \times 3) (2 \times 3) \text{.}\) The inside numbers do not match and so you can’t do this multiplication. This means that anything you write will be absolute nonsense because it is impossible to multiply these matrices in this order. Aren’t they the same two matrices considered in the previous example? Yes they are. It is just that here they are in a different order. This shows something you must always remember about matrix multiplication. Order Matters! Matrix Multiplication Is Not Commutative! This is very different than multiplication of numbers!

Subsection 11.2.3 The \(ij^{th}\) Entry of a Product

It is important to describe matrix multiplication in terms of entries of the matrices. What is the \(ij^{th}\) entry of \(AB\text{?}\) It would be the \(i^{th}\) entry of the \(j^{th}\) column of \(AB\text{.}\) Thus it would be the \(i^{th}\) entry of \(A\mathbf{b}_j\) . Now
\begin{equation*} \mathbf{b}_j = \begin{pmatrix} B_{1j} \\ \vdots\\ B_{nj}\\ \end{pmatrix} \end{equation*}
and from the above defintion, the \(i^{th}\) entry is
\begin{equation*} \sum_{k=1}^{n} A_{ik}B_{kj} \equiv A_{i1}B_{1j} + A_{i2}B_{2j} + \dots + A_{in}B_{nj} \end{equation*}
In terms of pictures of the matrix, you are doing
\begin{equation*} \begin{pmatrix} A_{11} & A_{12} & \dots & A_{1n} \\ A_{21} & A_{22} & \dots & A_{2n} \\ \vdots & \vdots & & \vdots\\ A_{m1} & A_{m2} & \dots & A_{mn}\\ \end{pmatrix} \begin{pmatrix} B_{11} & B_{12} & \dots & B_{1p} \\ B_{21} & B_{22} & \dots & B_{2p} \\ \vdots & \vdots & & \vdots\\ B_{n1} & B_{n2} & \dots & B_{np}\\ \end{pmatrix} \end{equation*}
Then as explained above, the \(j^{th}\) column is of the form
\begin{equation*} \begin{pmatrix} A_{11} & A_{12} & \dots & A_{1n} \\ A_{21} & A_{22} & \dots & A_{2n} \\ \vdots & \vdots & & \vdots\\ A_{m1} & A_{m2} & \dots & A_{mn}\\ \end{pmatrix} \begin{pmatrix} B_{1j}\\ B_{2j} \\ \vdots \\ B_{nj}\\ \end{pmatrix} \end{equation*}
which is an \(m \times 1\) matrix or column vector which equals
\begin{equation*} \begin{pmatrix} A_{11}\\ A_{21} \\ \vdots \\ A_{m1}\\ \end{pmatrix}B_{1j} + \begin{pmatrix} A_{12}\\ A_{22} \\ \vdots \\ A_{m2}\\ \end{pmatrix}B_{2j} + \dots + \begin{pmatrix} A_{1n}\\ A_{2n} \\ \vdots \\ A_{mn}\\ \end{pmatrix}B_{nj}. \end{equation*}
The second entry of this \(m \times 1\) matrix is
\begin{equation*} A_{21}B_{1j} + A_{22}B_{2j} + \dots + A_{2n}B_{nj} = \sum_{k=1}^{m} A_{2k}B_{kj}. \end{equation*}
Similarly, the \(i^{th}\) entry of this \(m \times 1\) matrix is
\begin{equation*} A_{i1}B_{1j} + A_{i2}B_{2j} + \dots + A_{in}B_{nj} = \sum_{k=1}^{m} A_{ik}B_{kj}. \end{equation*}
This shows the following definition for matrix multiplication in terms of the \(ij^{th}\) entries of the project coincides with Definition 11.2.11

Definition 11.2.14.

Let \(A = (A_{ij})\) be an \(m \times n\) matrix and let \(B = (B_{ij})\) be an \(n \times p\) matrix. Then \(AB\) is an \(m \times p\) matrix and
\begin{equation*} \left(AB\right)_{ij} = \sum_{k=1}^{n} A_{ik}B_{kj} \equiv A_{i1}B_{1j} + A_{i2}B_{2j} + \dots + A_{in}B_{nj} \end{equation*}
Another way to write this is
\begin{equation*} \left(AB\right)_{ij} = \overset{1 \times n} {\begin{pmatrix} A_{i1} & A_{i2} & \dots & A_{in}\end{pmatrix}} \overset{n \times 1}{ \begin{pmatrix} B_{1j} \\ B_{2j} \\ \vdots \\ B_{nj}\end{pmatrix}}. \end{equation*}
Note that to get \((AB)_{ij}\) you multiply the \(i^{th}\) row of \(A\) and the \(j^{th}\) column of \(B\text{.}\) In terms of the dot product from calculus, the \(ij^{th}\) entry of \(AB\) is the dot product of the \(i^{th}\) row of \(A\) with the \(j^{th}\) column of \(B\text{.}\)
I will summarize the above discussion in the following proposition which shows that the above definition delivers the earlier one in Definition 11.2.11 about \(AB = \begin{pmatrix} A\mathbf{b}_1 & \dots & A\mathbf{b}_p\end{pmatrix}\text{.}\) It is important to realize these two definitions are equivalent.

Proof.

From the definition of multiplication of matrices, \((AB)_{ik} = \sum_{r} A_{ir}B_{rk}\text{.}\) However,
\begin{equation*} \mathbf{b}_k = \begin{pmatrix} B_{1k} \\ \vdots \\ B_{nk}\end{pmatrix} \end{equation*}
and so, from the way we multiply a matrix times a vector,
\begin{equation*} (A\mathbf{b}_k)_i = \sum_r A_{ir}(\mathbf{b}_k)_r = \sum_r A_{ir}B_{rk}. \end{equation*}
Thus, the \(i^{th}\) entry from the top of \(A\mathbf{b}_k\) is the \(i^{th}\) entry in the \(k^{th}\) column of \(AB\) showing that indeed the claim is true. \(\blacksquare\)

Example 11.2.16.

Multiply if possible \(\begin{pmatrix} 1 & 2 \\ 3 & 1\\ 2 & 6\\ \end{pmatrix} \begin{pmatrix} 2 & 3 & 1\\ 7 & 6 & 2\\ \end{pmatrix}\text{.}\)
Solution.
First check to see if this is possible. It is of the form \((3 \times 2) (2 \times 3)\) and since the inside numbers match, the two matrices are conformable and it is possible to do the multiplication. The result should be a \(3 \times 3\) matrix. The answer is of the form
\begin{equation*} \begin{pmatrix} \begin{pmatrix} 1 & 2\\ 3 & 1\\ 2 & 6\\ \end{pmatrix} \begin{pmatrix} 2 \\ 7\\ \end{pmatrix}, & \begin{pmatrix} 1 & 2\\ 3 & 1\\ 2 & 6\\ \end{pmatrix} \begin{pmatrix} 3 \\ 6\\ \end{pmatrix}, & \begin{pmatrix} 1 & 2\\ 3 & 1\\ 2 & 6\\ \end{pmatrix} \begin{pmatrix} 1\\ 2\\ \end{pmatrix} \end{pmatrix} \end{equation*}
where the commas separate the columns in the resulting product. Thus the above product equals
\begin{equation*} \begin{pmatrix} 16 & 15 & 5\\ 13 & 15 & 5\\ 46 & 42 & 14\\ \end{pmatrix}, \end{equation*}
a \(3 \times 3\) matrix as desired. In terms of the \(ij^{th}\) entries and the above definition, the entry in the third row and second column of the product should equal
\begin{equation*} \sum_k a_{3k}b{k2} = a_{31}b{12} + a_{32}b{22} = 2 \times 3 + 6 \times 6 = 42. \end{equation*}
You should try a few more such examples to verify the above definition in terms of the \(ij^{th}\) entries works for other entries.

Example 11.2.17.

\(\begin{pmatrix} 1 & 2 \\ 3 & 1\\ 2 & 6\\ \end{pmatrix} \begin{pmatrix} 2 & 3 & 1\\ 7 & 6 & 2\\ 0 & 0 & 0\\ \end{pmatrix}\text{.}\)
Solution.
This is not possible because it is of the form \((3 \times 2)(3 \times 3)\) and the middle numbers don’t match. In other words the two matrices are not conformable in the indicated order.

Example 11.2.18.

\(\begin{pmatrix} 2 & 3 & 1\\ 7 & 6 & 2\\ 0 & 0 & 0\\ \end{pmatrix}\begin{pmatrix} 1 & 2 \\ 3 & 1\\ 2 & 6\\ \end{pmatrix} \text{.}\)
Solution.
This is possible because in this case it is of the form \((3 \times 3)(3 \times 2)\) and the middle numbers do match so the matrices are conformable. When the multiplication is done it equals
\begin{equation*} \begin{pmatrix} 13 & 13 \\ 29 & 32\\ 0 & 0\\ \end{pmatrix}. \end{equation*}
Check this and be sure you come up with the same answer.

Example 11.2.19.

\(\begin{pmatrix} 1\\ 2\\ 1\\ \end{pmatrix}\begin{pmatrix} 1 & 2 & 1 & 0\\ \end{pmatrix} \text{.}\)
Solution.
In this case it is of the form \((3 \times 1)(1 \times 4)\text{.}\) The middle numbers match so you can do it. Verify
\begin{equation*} \begin{pmatrix} 1\\ 2\\ 1\\ \end{pmatrix}\begin{pmatrix} 1 & 2 & 1 & 0\\ \end{pmatrix} = \begin{pmatrix} 1 & 2 & 1 & 0\\ 2 & 4 & 2 & 0\\ 1 & 2 & 1 & 0\\ \end{pmatrix}. \end{equation*}

Subsection 11.2.4 Properties of Matrix Multiplication

As pointed out above, sometimes it is possible to multiply matrices in one order but not in the other order. What if it makes sense to multiply them in either order? Will the two products be equal then?

Example 11.2.20.

Compare \(\begin{pmatrix} 1 & 2\\ 3 & 4\\ \end{pmatrix}\begin{pmatrix} 0 & 1 \\ 1 & 0\\ \end{pmatrix}\) and \(\begin{pmatrix} 0 & 1 \\ 1 & 0\\ \end{pmatrix}\begin{pmatrix} 1 & 2\\ 3 & 4\\ \end{pmatrix}\)
The first product is
\begin{equation*} \begin{pmatrix} 1 & 2\\ 3 & 4\\ \end{pmatrix}\begin{pmatrix} 0 & 1 \\ 1 & 0\\ \end{pmatrix} = \begin{pmatrix} 2 & 1 \\ 4 & 3\\ \end{pmatrix}. \end{equation*}
The second product is
\begin{equation*} \begin{pmatrix} 0 & 1 \\ 1 & 0\\ \end{pmatrix} \begin{pmatrix} 1 & 2\\ 3 & 4\\ \end{pmatrix} = \begin{pmatrix} 3 & 4 \\ 1 & 2\\ \end{pmatrix}. \end{equation*}
You see these are not equal. Again you cannot conclude that \(AB = BA\) for matrix multiplication even when multiplication is defined in both orders. However, there are some properties which do hold.

Proof.

Using Definition 11.2.14,
\begin{align*} (A(aB + bC))_{ij} \amp \sum_{k} A_{ik}(aB_{kj} + bC_{kj}) \\ \amp = a\sum_{k} A_{ik}B_{kj} + b\sum_{k} A_{ik}C_{kj}\\ \amp = a(AB)_{ij} + b(AC)_{ij}\\ \amp = (a(AB) + b(BC))_{ij}. \end{align*}
Thus \(A(aB + bC) = a(AB) + b(BC)\) as claimed. Formula (11.2.2) is entirely similar.
Formula (11.2.3) is the associative law of multiplication. Using definition Definition 11.2.14
\begin{align*} (A(BC))_{ij} \amp \sum_{k} A_{ik}(BC)_{kj} \\ \amp \sum_{k} A_{ik} \sum_{l} B_{kl}C_{lj}\\ \amp \sum_{l} (AB)_{il}C_{lj}\\ \amp ((AB)C)_{ij}. \end{align*}
This proves (11.2.3). \(\blacksquare\)

Subsection 11.2.5 Matrix Transpose

Another important operation on matrices is that of taking the transpose. The following example shows what is meant by this operation, denoted by placing a \(T\) as an exponent on the matrix.
\begin{equation*} \begin{pmatrix} 1 & 4\\ 3 & 1\\ 2 & 6\\ \end{pmatrix}^{T} = \begin{pmatrix} 1 & 3 & 2 \\ 4 & 1 & 6 \\ \end{pmatrix} \end{equation*}
What happened? The first column became the first row and the second column became the second row. Thus the \(3 \times 2\) matrix became a \(2 \times 3\) matrix. The number 3 was in the second row and the first column and it ended up in the first row and second column. Here is the definition.

Definition 11.2.22.

Let \(A\) be an \(m \times n\) matrix. Then \(A^T\) denotes the \(n \times n\) matrix which is defined as follows.
\begin{equation*} (A^T)_{ij} = A_{ji} \end{equation*}

Example 11.2.23.

\begin{equation*} \begin{pmatrix} 1 & 2 & -6 \\ 3 & 5 & 4 \\ \end{pmatrix}^{T} = \begin{pmatrix} 1 & 3\\ 2 & 5\\ -6 & 4\\ \end{pmatrix} \end{equation*}
The transpose of a matrix has the following important properties.

Proof.

From the definition,
\begin{equation*} \begin{pmatrix}(AB)^T \end{pmatrix}_{ij} = (AB)_{ji} = \sum_k (B^T)_{ik}(A^T)_{kj} = (B^T A^T)_{ij}. \end{equation*}
The proof of the second formula is left as an exercise.\(\blacksquare\)

Definition 11.2.25.

An \(n \times n\) matrix \(A\) is said to be symmetric if \(A = A^T\) . It is said to be skew symmetric if \(A = -A^T\) .

Example 11.2.26.

Let
\begin{equation*} A = \begin{pmatrix} 2 & 1 & 3\\ 1 & 5 & -3\\ 3 & -3 & 7.\\ \end{pmatrix} \end{equation*}
Then
\begin{equation*} A^T = \begin{pmatrix} 2 & 1 & 3\\ 1 & 5 & -3\\ 3 & -3 & 7.\\ \end{pmatrix} \end{equation*}
Therefore, \(A\) is symmetric.

Example 11.2.27.

Let
\begin{equation*} A = \begin{pmatrix} 0 & 1 & 3\\ -1 & 0 & 2\\ -3 & -2 & 0.\\ \end{pmatrix} \end{equation*}
Then
\begin{equation*} A^T = \begin{pmatrix} 0 & -1 & -3\\ 1 & 0 & -2\\ 3 & 2 & 0.\\ \end{pmatrix} \end{equation*}
Therefore, \(A\) is skew symmetric.

Subsection 11.2.6 Identity Matrix

There is a special matrix called \(I\) and referred to as the identity matrix. It is always a square matrix, meaning the number of rows equals the number of columns and it has the property that there are ones down the main diagonal and zeroes elsewhere. Here are some identity matrices of various sizes.
\begin{equation*} \begin{pmatrix} 1 \end{pmatrix}, \begin{pmatrix} 1 & 0\\ 0 & 1 \end{pmatrix}, \begin{pmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1\\ \end{pmatrix}, \begin{pmatrix} 1 & 0 & 0 & 0\\ 0 & 1 & 0 & 0\\ 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 1\\ \end{pmatrix} \end{equation*}
The first is the \(1 \times 1\) identity matrix, the second is the \(2 \times 2\) identity matrix, the third is the \(3 \times 3\) identity matrix, and the fourth is the \(4 \times 4\) identity matrix. By extension, you can likely see what the \(n \times n\) identity matrix would be. It is so important that there is a special symbol to denote the \(ij^{th}\) entry of the identity matrix \(I_{ij} = \delta_{ij}\) where \(\delta_{ij}\) is the Kronecker symbol defined by
\begin{equation*} \delta_{ij} = \begin{cases} 1 & \text{ if } i = j \\ 0 & \text{ if } i \neq j\\ \end{cases} \end{equation*}
It is called the identity matrix because it is a multiplicative identity in the following sense.

Proof.

\begin{equation*} (AI_n)_{ij} = \sum_{k} A_{ik}\delta_{kj} = A_{ij} \end{equation*}
and so \(AI_n = A\text{.}\) The other case is left as an exercise for you. \(\blacksquare\)

Exercises 11.2.7 Exercises

1.

Here are some matrices:
\begin{equation*} A = \begin{pmatrix} 1 & 2 & 3\\ 2 & 1 & 7 \end{pmatrix}, B = \begin{pmatrix} 3 & -1 & 2\\ -3 & 2 & 1 \end{pmatrix}, \end{equation*}
\begin{equation*} C = \begin{pmatrix} 1 & 2\\ 3 & 1 \end{pmatrix}, D = \begin{pmatrix} -1 & 2\\ 2 & -3 \end{pmatrix}, E = \begin{pmatrix} 2\\ 3 \end{pmatrix} \end{equation*}
Find if possible \(-3A, 3B - A, AC, CB, AE, EA.\) If it is not possible explain why.
Answer.
\begin{equation*} -3A = \begin{pmatrix} -3 & -6 & -9\\ -6 & -3 & -21 \end{pmatrix} \end{equation*}
\begin{equation*} 3B - A = \begin{pmatrix} 9 & -3 & 6\\ -9 & 6 & 3 \end{pmatrix} - \begin{pmatrix} 1 & 2 & 3\\ 2 & 1 & 7 \end{pmatrix} = \begin{pmatrix} 8 & -5 & 3\\ -11 & 5 & -4 \end{pmatrix} \end{equation*}
\begin{equation*} AC = \text{ not possible because } A \text{ has more rows than } C \text{ has columns.} \end{equation*}
\begin{equation*} CB = \begin{pmatrix} 1 & 2\\ 3 & 1 \end{pmatrix}\begin{pmatrix} 3 & -1 & 2\\ -3 & 2 & 1 \end{pmatrix} = \begin{pmatrix} -3 & 3 & 4\\ 6 & -1 & 7 \end{pmatrix} \end{equation*}
\begin{equation*} AE = \text{ not possible because } A \text{ has more rows than } E \text{ has columns.} \end{equation*}
\begin{equation*} EA = \text{ not possible because } E \text{ has less rows than } A \text{ has columns.} \end{equation*}

2.

Here are some matrices:
\begin{equation*} A = \begin{pmatrix} 1 & 2\\ 3 & 2 \\ 1 & -1 \end{pmatrix}, B = \begin{pmatrix} 2 & -5 & 2\\ -3 & 2 & 1 \end{pmatrix}, \end{equation*}
\begin{equation*} C = \begin{pmatrix} 1 & 2\\ 5 & 0 \end{pmatrix}, D = \begin{pmatrix} -1 & 1\\ 4 & -3 \end{pmatrix}, E = \begin{pmatrix} 1\\ 3 \end{pmatrix} \end{equation*}
Find if possible \(-3A, 3B - A, AC, CA, AE, EA, BE, DE.\) If it is not possible explain why.

3.

Here are some matrices:
\begin{equation*} A = \begin{pmatrix} 1 & 2 \\ 3 & 2 \\ 1 & -1 \end{pmatrix}, B = \begin{pmatrix} 2 & -5 & 2\\ -3 & 2 & 1 \end{pmatrix}, \end{equation*}
\begin{equation*} C = \begin{pmatrix} 1 & 2\\ 5 & 0 \end{pmatrix}, D = \begin{pmatrix} -1 & 1\\ 4 & -3 \end{pmatrix}, E = \begin{pmatrix} 1\\ 3 \end{pmatrix} \end{equation*}
Find if possible \(-3A^{T}, 3B - A^{T}, AC, CA, AE, E^{T}B, BE, DE, EE^{T}, E^{T}E.\) If it is not possible explain why.
Answer.
\begin{equation*} -3A^{T} = \begin{pmatrix} -3 & -6 \\ -9 & -6 \\ -3 & 3 \end{pmatrix}^{T} = \begin{pmatrix} -3 & -9 & -3\\ -6 & -6 & 3 \end{pmatrix} \end{equation*}
\begin{equation*} 3B - A^{T} = \begin{pmatrix} 6 & -15 & 6\\ -9 & 6 & 3 \end{pmatrix} - \begin{pmatrix} 1 & 3 & 1\\ 2 & 2 & -1 \end{pmatrix} = \begin{pmatrix} 8 & -6 & 5\\ -7 & 4 & 4 \end{pmatrix} \end{equation*}
\begin{equation*} AC = \ \begin{pmatrix} 1 & 2 \\ 3 & 2 \\ 1 & -1 \end{pmatrix}\begin{pmatrix} 1 & 2\\ 5 & 0 \end{pmatrix} = \ \begin{pmatrix} 11 & 2 \\ 13 & 6 \\ -4 & 2 \end{pmatrix} \end{equation*}
\begin{equation*} CA = \text{ not possible because } C \text{ has less columns than } A \text{ has rows.} \end{equation*}
\begin{equation*} AE = \begin{pmatrix} 1 & 2 \\ 3 & 2 \\ 1 & -1 \end{pmatrix}\begin{pmatrix} 1\\ 3 \end{pmatrix} = \begin{pmatrix} 7\\ 9\\ -2 \end{pmatrix} \end{equation*}
\begin{equation*} E^{T}B = \begin{pmatrix} 1 &3 \end{pmatrix} \begin{pmatrix} 2 & -5 & 2\\ -3 & 2 & 1 \end{pmatrix} = \begin{pmatrix} -7 & 1 & 5 \end{pmatrix} \end{equation*}
\begin{equation*} BE = \text{ not possible because } E \text{ has less rows than } B \text{ has columns.} \end{equation*}
\begin{equation*} DE = \begin{pmatrix} -1 & 1\\ 4 & -3 \end{pmatrix}\begin{pmatrix} 1\\ 3 \end{pmatrix} = \begin{pmatrix} 2\\ -5 \end{pmatrix} \end{equation*}
\begin{equation*} EE^{T} = \begin{pmatrix} 1\\ 3 \end{pmatrix} \begin{pmatrix} 1 & 3 \end{pmatrix} = \begin{pmatrix} 1 & 3\\ 3 & 9 \end{pmatrix} \end{equation*}
\begin{equation*} E^{T}E = \begin{pmatrix} 1 & 3 \end{pmatrix} \begin{pmatrix} 1\\ 3 \end{pmatrix} = \begin{pmatrix} 10 \end{pmatrix} \end{equation*}

4.

Here are some matrices:
\begin{equation*} A = \begin{pmatrix} 1 & 2\\ 3 & 2 \\ 1 & -1 \end{pmatrix}, B = \begin{pmatrix} 2 & -5 & 2\\ -3 & 2 & 1 \end{pmatrix}, \end{equation*}
\begin{equation*} C = \begin{pmatrix} 1 & 2\\ 5 & 0 \end{pmatrix}, D = \begin{pmatrix} -1\\ 4 \end{pmatrix}, E = \begin{pmatrix} 1\\ 3 \end{pmatrix} \end{equation*}
Find if possible \(AD, DA, D^{T}B, D^{T}BE, E^{T}D, DE^{T} .\) If it is not possible explain why.

5.

Let \(A = \begin{pmatrix} 1 & 1 \\ -2 & -1 \\ 1 & 2 \end{pmatrix}, B = \begin{pmatrix} 1 & -1 & -2\\ 2 & 1 & -2 \end{pmatrix},\) and \(C = \begin{pmatrix} 1 & 1 & -3\\ -1 & 2 & 0 \\ -3 & -1 & 0 \end{pmatrix}\)
Find if possible:
  1. \(\displaystyle AB\)
  2. \(\displaystyle BA\)
  3. \(\displaystyle AC\)
  4. \(\displaystyle CA\)
  5. \(\displaystyle CB\)
  6. \(\displaystyle BC\)
If not possible, explain why.
Answer.
  1. \begin{equation*} AB = \begin{pmatrix} 1 & 1 \\ -2 & -1 \\ 1 & 2 \end{pmatrix} \begin{pmatrix} 1 & -1 & -2\\ 2 & 1 & -2 \end{pmatrix} = \begin{pmatrix} 3 & 0 & -4 \\ -4 & 1 & 6 \\ 5 & 1 & -6 \end{pmatrix} \end{equation*}
  2. \begin{equation*} BA = \begin{pmatrix} 1 & -1 & -2\\ 2 & 1 & -2 \end{pmatrix} \begin{pmatrix} 1 & 1 \\ -2 & -1 \\ 1 & 2 \end{pmatrix} = \begin{pmatrix} -11 & -8 \\ 4 & -3 \end{pmatrix} \end{equation*}
  3. \(AC \) is not possible, the number of columns of \(A\) do not match the number of rows of \(C\text{.}\)
  4. \begin{equation*} CA= \begin{pmatrix} 1 & 1 & -3\\ -1 & 2 & 0 \\ -3 & -1 & 0 \end{pmatrix}\begin{pmatrix} 1 & 1 \\ -2 & -1 \\ 1 & 2 \end{pmatrix} = \begin{pmatrix} -4 & -4 \\ -5 & 1 \\ -1 & -4 \end{pmatrix} \end{equation*}
  5. \(CB\) is not possible because the number of columns of \(C\) are not the same as the number of rows of \(B\text{.}\)
  6. \begin{equation*} BC = \begin{pmatrix} 1 & -1 & -2\\ 2 & 1 & -2 \end{pmatrix} \begin{pmatrix} 1 & 1 & -3\\ -1 & 2 & 0 \\ -3 & -1 & 0 \end{pmatrix} = \begin{pmatrix} 8 & 1 & -3\\ 7 & 6 & -6 \end{pmatrix} \end{equation*}

6.

Suppose \(A\) and \(B\) are square matrices of the same size. Which of the following are correct?
  1. \(\displaystyle (A - B)^2 = A^2 - 2AB + B^2\)
  2. \(\displaystyle (AB)^2 = A^2 B^2\)
  3. \(\displaystyle (A + B)^2 = A^2 + 2AB + B^2\)
  4. \(\displaystyle (A + B)^2 = A^2 + AB + BA + B^2\)
  5. \(\displaystyle A^2 B^2 = A (AB) B\)
  6. \(\displaystyle (A + B)3 = A^3 + 3A^2 B + 3AB^2 + B^3\)
  7. \(\displaystyle (A + B) (A - B) = A^2 - B^2\)

7.

Let \(A = \begin{pmatrix} -1 & -1 \\ 3 & 3 \end{pmatrix}.\) Find all \(2 \times 2\) matrices, \(B\) such that \(AB = 0\text{.}\)
Answer.
If \(B\) is all zeros then \(AB = 0\text{:}\)
\begin{equation*} \begin{pmatrix} -1 & -1 \\ 3 & 3 \end{pmatrix}\begin{pmatrix} 0 & 0 \\ 0 & 0 \end{pmatrix} = \begin{pmatrix} 0 & 0 \\ 0 & 0 \end{pmatrix} \end{equation*}
Otherwise let \(B = \begin{pmatrix} x & y \\ z & w \end{pmatrix}\text{,}\) then \(AB = 0\) whenever \(z = -x \) and \(w = -y\text{.}\)

8.

Let \(\mathbf{x} = (-1, -1, 1)\) and \(\mathbf{y} = (0, 1, 2)\) . Find \(\mathbf{x^{T}y}\) and \(\mathbf{xy^{T}}\) if possible.

9.

Let \(A = \begin{pmatrix}1 & 2 \\ 3 & 4 \end{pmatrix}, B = \begin{pmatrix} 1 & 2 \\ 3 & k \end{pmatrix} \text{.}\) Is it possible to choose \(k\) such that \(AB = BA\text{?}\) If so, what should \(k\) equal?
Answer.
Yes, if \(k = 4\text{,}\) the matrices are identical so the order of multiplication doesn’t matter, the outcome is the same.

10.

Let \(A = \begin{pmatrix}1 & 2 \\ 3 & 4 \end{pmatrix}, B = \begin{pmatrix} 1 & 2 \\ 1 & k \end{pmatrix} \text{.}\) Is it possible to choose \(k\) such that \(AB = BA\text{?}\) If so, what should \(k\) equal?

11.

Prove that \(I_{m}A = A\) where \(A\) is an \(m \times n\) matrix.
Answer.
In matrix multiplication, the \(i^{th}\) row of the first matrix affects only the \(i^{th}\) row of the output. The \(i^{th}\) row of \(I_m\) is all 0s except for a 1 in the \(i^{th}\) column. Calculating the \(i,k^{th}\) element in the output matrix will be a sum of products of row \(i\) of \(I_m\) and column \(k\) of \(A\text{.}\) By definition these row and column vectors must be the same length. The products will all 0s, except for the \(i^{th}\) term, which will be \(1 \cdot A_{ik}\text{.}\) Therefore the \(i,k^{th}\) element in the output matrix will be identical to \(A_{ik}\text{,}\) so \(I_{m}A = A. \blacksquare\)

12.

Give an example of matrices, \(A, B, C \) such that \(B \neq C, A \neq 0,\) and yet \(AB = AC\text{.}\)
PrevTopNext