WEBVTT
1
00:00:00.000 --> 00:00:15.800
Okay. This lecture is mostly about the idea of similar matrices. I'm going to tell you
2
00:00:15.800 --> 00:00:23.800
what that word similar means and in what way two matrices are called similar. But before
3
00:00:23.800 --> 00:00:30.800
I do that, I have a little more to say about positive definite matrices. You can tell this
4
00:00:30.800 --> 00:00:39.040
is a subject I think is really important and I told you what positive definite meant.
5
00:00:39.040 --> 00:00:48.320
It means that this expression, this quadratic form, x transpose, a x is always positive. But
6
00:00:48.320 --> 00:00:56.800
the direct way to test it was with eigenvalues or pivots or determinants. So we know what
7
00:00:56.800 --> 00:01:03.160
it means. We know how to test it. But I didn't really say where positive definite matrices
8
00:01:03.160 --> 00:01:13.000
come from. And so one thing I want to say is that they come from least square and all
9
00:01:13.000 --> 00:01:19.960
sorts of physical problems start with a rectangular matrix. You remember in these squares the
10
00:01:19.960 --> 00:01:26.840
crucial combination was a transpose a. So I want to show that that's a positive definite
11
00:01:26.840 --> 00:01:34.760
matrix. So I'm going to speak a little more about positive definite matrices. Just recapping.
12
00:01:34.760 --> 00:01:45.360
So let me ask a question. It may be on the homework. Suppose a matrix a is positive definite.
13
00:01:45.360 --> 00:01:52.880
I mean by that it's all I'm assuming it's symmetric. That's always built into the definition.
14
00:01:52.880 --> 00:01:59.920
So we have a symmetric positive definite matrix. What about it's inverse? Is the inverse
15
00:01:59.920 --> 00:02:07.360
of a symmetric positive definite matrix also symmetric positive definite? So you quickly
16
00:02:07.360 --> 00:02:18.000
think, okay, what do I know about the pivots of the inverse matrix not much? What do I know
17
00:02:18.000 --> 00:02:26.680
about the eigenvalues of the inverse matrix? Everything, right? The eigenvalues of the inverse
18
00:02:26.680 --> 00:02:34.760
are one over the eigenvalues of the matrix. So if my matrix starts out positive definite
19
00:02:34.760 --> 00:02:41.440
then right away I know that it's inverse is positive definite because those positive
20
00:02:41.440 --> 00:02:50.400
eigenvalues then one over the eigenvalues also positive. What if I know that a matrix
21
00:02:50.400 --> 00:02:58.480
a and a matrix b are both positive definite? But let me ask you this. Suppose if a and
22
00:02:58.480 --> 00:03:12.080
b are positive definite, what about a plus b? In some way you hope that that would be true.
23
00:03:12.080 --> 00:03:19.080
It's positive definite. For a matrix it's kind of like positive for a real number. But
24
00:03:19.080 --> 00:03:25.400
we don't know the eigenvalues of a plus b. We don't know the pivots of a plus b. So we
25
00:03:25.400 --> 00:03:31.760
just have to go down this list of all right, which approach to positive definite can we
26
00:03:31.760 --> 00:03:40.040
get a handle on? And this is a good one. This is a good one. How would we decide that
27
00:03:40.040 --> 00:03:48.760
if a was like this and if b was like this then we would look at x transpose a plus b x.
28
00:03:48.760 --> 00:03:55.760
I'm sure this is in the homework. Now so we have x transpose a x but bigger than zero,
29
00:03:55.760 --> 00:04:05.640
x transpose b x positive for all x. So now I ask you about this guy. And of course you
30
00:04:05.640 --> 00:04:13.640
just add that and that and we get what we want. If a and b are positive definite so is
31
00:04:13.640 --> 00:04:25.840
a plus b. So that's what I've shown. So is a plus b. Just be sort of ready for all
32
00:04:25.840 --> 00:04:34.120
the approaches through eigenvalues and through this expression. And now finally one more
33
00:04:34.120 --> 00:04:41.920
thought about positive definite is this combination that came up in the squares. Can I do that?
34
00:04:41.920 --> 00:04:59.280
So now suppose a is rectangular m by n. So I'm sorry that I've used the same letter a for
35
00:04:59.280 --> 00:05:08.200
the positive definite matrices and the eigenvalue chapter that I used back in earlier chapters
36
00:05:08.200 --> 00:05:14.960
when the matrix was rectangular. Now that matrix, a rectangular matrix, no way it's positive
37
00:05:14.960 --> 00:05:23.000
definite. It's not symmetric. It's not even square in general. But you remember that the
38
00:05:23.000 --> 00:05:39.920
key for these rectangular ones was a transpose x. That's square. That's symmetric. Those
39
00:05:39.920 --> 00:05:49.280
are things we knew back when we met this thing in the least square stuff, in the projection
40
00:05:49.280 --> 00:05:56.720
stuff. But now we know something more we can ask a more important question, a deeper
41
00:05:56.720 --> 00:06:06.960
question. Is it positive definite? And we sort of hope so. Like we might in analogy
42
00:06:06.960 --> 00:06:16.920
with numbers it's like sort of like the square of a number and that's positive. So now
43
00:06:16.920 --> 00:06:26.280
I want to ask the matrix question. It is a transpose a positive definite. Okay, now it's
44
00:06:26.280 --> 00:06:32.480
so again. It's a rectangular a that I'm starting with but it's the combination a transpose
45
00:06:32.480 --> 00:06:39.480
a that's the square, symmetric and hopefully positive definite matrix. So how do I see
46
00:06:39.480 --> 00:06:49.120
that it is? Positive definite or at least positive semi definite. You'll see that. Well,
47
00:06:49.120 --> 00:06:57.880
I don't know the eigenvalues of this product. I don't want to work with the pivot. The right
48
00:06:57.880 --> 00:07:06.360
thing, the right quantity to look at is this x transpose a x, x transpose times my matrix
49
00:07:06.360 --> 00:07:15.200
times x. I'd like to see that this thing that that expression is always positive. I'm
50
00:07:15.200 --> 00:07:22.920
not doing it with numbers. I'm doing with symbols. Do you see how do I see that that expression
51
00:07:22.920 --> 00:07:31.680
comes out positive? I'm taking a rectangular matrix a and a transpose that gives me something
52
00:07:31.680 --> 00:07:37.120
square symmetric but now I want to see that if I multiply that if I do this, I form this
53
00:07:37.120 --> 00:07:43.880
quadratic expression that I get this positive thing that goes upwards when I graph it. How
54
00:07:43.880 --> 00:07:50.520
do I see that that's positive? Or absolutely. It isn't negative anyway. We will have to
55
00:07:50.520 --> 00:07:56.960
spend a minute on the question. Could it be zero? But it can't be negative. Why can't
56
00:07:56.960 --> 00:08:09.640
this never be negative? The argument is like the one key idea in so many steps in linear
57
00:08:09.640 --> 00:08:19.320
algebra put those parentheses in a good way. Put the parentheses around a x and what's the
58
00:08:19.320 --> 00:08:32.920
first part? What's this x transpose a transpose? That is a x transpose. So what do we have?
59
00:08:32.920 --> 00:08:41.960
We have the length squared of a x. That's the column vector a x. That's the row vector
60
00:08:41.960 --> 00:08:53.600
a x. It's length squared. Length squared, certainly greater than or possibly equal to zero.
61
00:08:53.600 --> 00:08:59.400
So we have to deal with this little possibility. Could it be equal? When could the length squared
62
00:08:59.400 --> 00:09:10.720
be zero? Only if the vector is zero. That's the only vector that has length squared zero.
63
00:09:10.720 --> 00:09:20.680
So we would like to get that possibility out of there. So I want to have a x never be
64
00:09:20.680 --> 00:09:27.200
zero except of course for the zero vector. How do I assure that a x is never zero? In other
65
00:09:27.200 --> 00:09:38.280
words, how do I assure that there's no null space of a? The rank should be, so now remember
66
00:09:38.280 --> 00:09:46.240
what's the rank when there's no null space? By no null space you know what I mean only
67
00:09:46.240 --> 00:09:56.560
to zero vector in the null space. So if I have an 11 by 5 matrix, so it's got 11 rows,
68
00:09:56.560 --> 00:10:04.520
5 columns, when is there no null space? So the columns should be independent. What's
69
00:10:04.520 --> 00:10:18.400
the rank? N, 5. Rank N. Independent columns. So if I, then I conclude yes positive definite.
70
00:10:18.400 --> 00:10:27.040
And this was the assumption that then a transpose a is invertible. The, the, the, these squares
71
00:10:27.040 --> 00:10:36.520
equations all work fine. And more than that, the matrix is even positive definite. And
72
00:10:36.520 --> 00:10:43.680
I just just say one comment about numerical things. With a positive definite matrix, you
73
00:10:43.680 --> 00:10:52.800
never have to do row exchanges. You never run into unsuitably small numbers or zeros in
74
00:10:52.800 --> 00:10:58.120
the pivot position. They're the right, they're the great matrices to compute with and they're
75
00:10:58.120 --> 00:11:08.360
the great matrices to study. So that's, I wanted to take this first 10 minutes of, grab
76
00:11:08.360 --> 00:11:16.520
the first 10 minutes away from similar matrices and continue this much more with positive
77
00:11:16.520 --> 00:11:27.240
definite. I'm really at this point now coming close to the end of the heart of the niragra.
78
00:11:27.240 --> 00:11:34.120
Positive definiteness brought everything together. Similar matrices, which is coming the rest
79
00:11:34.120 --> 00:11:42.720
of this hour is a key topic. And please come on Monday. Monday is about what's called
80
00:11:42.720 --> 00:11:53.280
the SVD singular values. It's, the, has become a central, a fact in a central part of linear
81
00:11:53.280 --> 00:12:05.520
algebra. I mean, you could come after Monday also. But Monday is, that singular value thing
82
00:12:05.520 --> 00:12:11.720
is, has made it into this course. 10 years ago, five years ago, it wasn't in the course.
83
00:12:11.720 --> 00:12:19.800
Now it has to be. Okay. So, can I begin today's lecture proper with this idea of similar
84
00:12:19.800 --> 00:12:27.480
matrices? This is what similar matrices mean. So, let's start again. I'll write it again.
85
00:12:27.480 --> 00:12:37.320
So a and b are similar. A and b are, now I'm, these matrices, I'm no longer talking
86
00:12:37.320 --> 00:12:45.960
about symmetric matrices. In, at least no longer expecting symmetric matrices. I'm talking
87
00:12:45.960 --> 00:12:57.760
about two square matrices n by n. A and b, they're, they're n by n, n by n matrices. And
88
00:12:57.760 --> 00:13:06.680
I'm introducing this word similar. So, I'm going to say, what does it mean? It means
89
00:13:06.680 --> 00:13:14.200
that they're connected in the way, well, in the way I've written here. So, let me rewrite
90
00:13:14.200 --> 00:13:27.000
it. That means, that means, means, that for some matrix m, which has to be invertible,
91
00:13:27.000 --> 00:13:34.320
because you'll see that. This, one matrix is, take the other matrix, multiply on the
92
00:13:34.320 --> 00:13:46.800
right by m and on the left by m inverse. So, the question is, why that combination? But
93
00:13:46.800 --> 00:13:53.160
part of the answer you know already. You remember, we've done this, we've taken a matrix
94
00:13:53.160 --> 00:14:08.680
A. So, let's do an example of similar. Suppose, suppose a, the matrix A, suppose it has
95
00:14:08.680 --> 00:14:16.680
a full set of eigenvectors. They go in this eigenvector matrix S. Then what was the
96
00:14:16.680 --> 00:14:24.480
main point of the whole, the main calculation of the whole chapter was, is use that eigenvector
97
00:14:24.480 --> 00:14:36.800
matrix S. And its inverse comes over there to produce the nicest possible matrix, lambda,
98
00:14:36.800 --> 00:14:49.640
nicest possible because it's diagonal. So, in our new language, this is saying A is similar
99
00:14:49.640 --> 00:15:02.120
to lambda. A is similar to lambda. Because there is a matrix, and this particular, there
100
00:15:02.120 --> 00:15:09.880
is an m, and this particular m is this important guy, this eigenvector matrix. But if I take
101
00:15:09.880 --> 00:15:18.840
a different matrix, m, and I look at m inverse am, the result won't come out diagonal, but
102
00:15:18.840 --> 00:15:26.240
it'll come out a matrix B that's similar to A. Do you see that I'm, what I'm doing is,
103
00:15:26.240 --> 00:15:36.640
like, I'm putting these matrices into families. All of matrices in the family are similar
104
00:15:36.640 --> 00:15:42.760
to each other. They're all, each one in this family is connected to each other one by
105
00:15:42.760 --> 00:15:51.160
some matrix m, and like the outstanding member of the family is the diagonal guy. I mean,
106
00:15:51.160 --> 00:15:57.840
that's the simplest, neatest matrix in this family of all the matrices that are similar
107
00:15:57.840 --> 00:16:04.800
to A. The best one is lambda. But there are lots of others because I can take different
108
00:16:04.800 --> 00:16:11.960
instead of S. I can take any old matrix m, any old invertible matrix, and do it. I better
109
00:16:11.960 --> 00:16:26.680
do an example. Suppose I take A as the matrix, 2112. Do you know the eigenvalue matrix for
110
00:16:26.680 --> 00:16:38.840
that? The eigenvalues of that matrix are, well, 3 and 1. So that and the eigenvectors
111
00:16:38.840 --> 00:16:50.560
would be easy to find. So this matrix is similar to this one. But my point is I can also take
112
00:16:50.560 --> 00:16:57.200
my matrix 2112. I can multiply it by, let's see, what I'm just going to cook off a matrix
113
00:16:57.200 --> 00:17:06.920
m here. Let me just invent one for one zero. And over here, I'll put m inverse, and because
114
00:17:06.920 --> 00:17:14.600
I happen to make that triangular, I know that it's inverse is that. So there's m inverse
115
00:17:14.600 --> 00:17:21.400
a m. That's going to produce some matrix. Well, I got to do the multiplication. So hang
116
00:17:21.400 --> 00:17:29.520
on a second. I'll just copy that 1 minus 401 and multiply these guys. So I'm getting
117
00:17:29.520 --> 00:17:43.680
2, 9, 1 and 6, I think. Can you check it as I go? Because you see, I'm just, so that's
118
00:17:43.680 --> 00:17:52.160
2 minus 4. I'm getting a minus 2. 9 minus 24 is a minus 15. My God, how did I get this?
119
00:17:52.160 --> 00:18:04.440
And that's probably 1 and 6. So there's my matrix B. And there's my matrix lambda. There's
120
00:18:04.440 --> 00:18:11.920
my matrix A, and my point is these are all similar matrices. They all have something
121
00:18:11.920 --> 00:18:25.040
in common, besides being just 2 by 2. They have something in common. And what is it?
122
00:18:25.040 --> 00:18:33.280
What's the point about two matrices that are built out of, the B is built out of m
123
00:18:33.280 --> 00:18:42.280
inverse a m. What is it that a and b have in common? That's the, now I'm telling you the
124
00:18:42.280 --> 00:18:49.480
main fact about similar matrices. They have the same eigenvalues. This is this chapter
125
00:18:49.480 --> 00:18:54.880
is about eigenvalues. And that's why we're interested in this family of matrices that
126
00:18:54.880 --> 00:19:02.160
have the same eigenvalues. What are the eigenvalues in this example? Lambda. The eigenvalues
127
00:19:02.160 --> 00:19:10.920
of that I could compute. The eigenvalues of that I can compute really fast. So the eigenvalues
128
00:19:10.920 --> 00:19:18.840
are 3 and 1 for this for sure. Now, did we, do you see why the eigenvalues are 3 and
129
00:19:18.840 --> 00:19:26.640
1 for that one? If I tell you the eigenvalues are 3 and 1, you, you, you prick, quickly process
130
00:19:26.640 --> 00:19:35.720
the trace, which is then 4, agrees with 4, and you process the determinant 3 times 1.
131
00:19:35.720 --> 00:19:42.040
The determinant is 3 and you say yes, it's right. Now, I am hoping that the eigenvalues
132
00:19:42.040 --> 00:19:49.360
of this thing are 3 and 1. May I process the trace under determinant for that one? What's
133
00:19:49.360 --> 00:19:57.160
the trace here? The trace of this matrix is 4 minus 2 and 6, and that's what it should
134
00:19:57.160 --> 00:20:04.920
be. What's the determinant minus 12 plus 15 is 3? The determinant is 3. The eigenvalues
135
00:20:04.920 --> 00:20:11.960
of that matrix are also 3 and 1. And you see, I created this matrix just like I just took
136
00:20:11.960 --> 00:20:19.840
any M, like when the popped into my head and computed M inverse AM, got that matrix. It
137
00:20:19.840 --> 00:20:29.120
didn't look anything special, but it's, it's like A itself. It has those eigenvalues
138
00:20:29.120 --> 00:20:36.920
3 and 1. So that's the main fact, and let me write it down. Similar matrices.
139
00:20:36.920 --> 00:21:00.640
Have the same eigenvalues. So I'll just put that as an important point and think why.
140
00:21:00.640 --> 00:21:06.600
Why is that? So that's what that family of matrices is. The matrices that are similar
141
00:21:06.600 --> 00:21:15.240
to this A here are all the matrices with eigenvalues 3 and 1. Every matrix with eigenvalues
142
00:21:15.240 --> 00:21:24.120
3 and 1, there's some M that connects this guy to the one you think of. And then, of
143
00:21:24.120 --> 00:21:30.240
course, the most special guy in the whole family is the diagonal one with eigenvalues 3
144
00:21:30.240 --> 00:21:36.280
and 1 sitting there on the diagonal. But also, I could find, tell me just a couple more
145
00:21:36.280 --> 00:21:43.360
members of the family. Another, tell me another matrix that has eigenvalues 3 and 1.
146
00:21:43.360 --> 00:21:55.200
Well, let's see. Oh, I'll just make it triangular. That's in the family. There is some M that
147
00:21:55.200 --> 00:22:05.400
connects to this one. And, and, and also this. There's some matrix M so that M inverse
148
00:22:05.400 --> 00:22:15.760
A M comes out to be that. There's a whole family here. And they all share the same eigenvalues.
149
00:22:15.760 --> 00:22:26.800
So why is that? Okay. I'm going to start the only possibility is to start with A x equal
150
00:22:26.800 --> 00:22:36.880
to M to X. Okay. So suppose A has the eigenvalue lambda. Now, I want to get B in the picture
151
00:22:36.880 --> 00:22:46.000
here somehow. Remember B is M inverse A M. Let's just remember that over here. B is M inverse
152
00:22:46.000 --> 00:22:56.720
A M. And I want to see its eigenvalues. How am I going to get M inverse A M into this equation?
153
00:22:56.720 --> 00:23:04.280
Let me just sort of do it. I'll put it in M times an M inverse in there. Right? That
154
00:23:04.280 --> 00:23:14.040
was, I haven't changed the left hand side. So I better not change the right hand side. So
155
00:23:14.040 --> 00:23:19.360
everybody's okay so far. I just put an M. See, I want to get it. So now I multiply on
156
00:23:19.360 --> 00:23:26.520
the left by M inverse. I have to do the same to this side. And that number lambda is just
157
00:23:26.520 --> 00:23:36.800
a number. So it factors out in the front. So what I had here is this was safe. I did
158
00:23:36.800 --> 00:23:45.680
the same thing to both sides. And now I've got B. There's B. That's B times this vector
159
00:23:45.680 --> 00:23:54.320
M inverse X is equal to lambda times this vector M inverse X. So what have I learned? I've
160
00:23:54.320 --> 00:24:00.880
learned that B times some vector is lambda times that vector. I've learned that lambda
161
00:24:00.880 --> 00:24:08.280
is an eigenvalue of B also. So this is if, so this is if lambda is an eigenvalue of A,
162
00:24:08.280 --> 00:24:14.720
then I can write it this way and I discover that lambda is an eigenvalue of B. That's
163
00:24:14.720 --> 00:24:22.120
the end of the proof. The eigenvector didn't say the same. Of course, I don't expect the
164
00:24:22.120 --> 00:24:26.480
eigenvectors to say the same. If all the eigenvalues are the same and all the eigenvectors
165
00:24:26.480 --> 00:24:34.120
are the same, then probably the matrix is the same. Here, the eigenvector is changed.
166
00:24:34.120 --> 00:24:47.640
So the eigenvector, so the point is then the eigenvector of B is M inverse times the
167
00:24:47.640 --> 00:25:00.040
eigenvector of A. Okay. That's all that this says here. The eigenvector of A was X.
168
00:25:00.040 --> 00:25:07.400
And so the M inverse, similar matrices then have the same eigenvalues and their eigenvectors
169
00:25:07.400 --> 00:25:14.600
are just moved around. Of course, that's what we, that's what happened way back in the
170
00:25:14.600 --> 00:25:22.680
most important similar matrices are to diagonalize. So what was the point when we diagonalized?
171
00:25:22.680 --> 00:25:29.680
The eigenvalues stayed the same, of course, three and one. What about the eigenvectors?
172
00:25:29.680 --> 00:25:35.640
The eigenvectors were whatever they were for the matrix A, but then what were the eigenvectors
173
00:25:35.640 --> 00:25:42.480
for the diagonal matrix? They're just, what are the eigenvectors of a diagonal matrix?
174
00:25:42.480 --> 00:25:49.200
They're just one zero and zero one. So this step made the eigenvector's nice, didn't
175
00:25:49.200 --> 00:25:57.880
change the eigenvalues. And every time we don't change the eigenvalues, same eigenvalues.
176
00:25:57.880 --> 00:26:08.880
Okay. Now, so I've got all these matrices, I've got this family of matrices with eigenvalues
177
00:26:08.880 --> 00:26:18.440
three and one. Fine. That's a nice family. It's nice because those two eigenvalues are
178
00:26:18.440 --> 00:26:30.840
different. I now have to get into the, into the less happy possibility that the two eigenvalues
179
00:26:30.840 --> 00:26:38.720
could be the same. And then it's a little trickier because you remember when two eigenvalues
180
00:26:38.720 --> 00:26:47.240
are the same, what's the bad possibility? That there might not be enough, a full set
181
00:26:47.240 --> 00:26:53.560
of eigenvectors and we might not be able to diagonalize. So I need to discuss the bad
182
00:26:53.560 --> 00:27:08.400
case. So the bad, can I just say bad, if lambda one equals lambda two, then the matrix
183
00:27:08.400 --> 00:27:20.480
might not be diagonalizable. Suppose lambda one equals lambda two equals four, say. Now,
184
00:27:20.480 --> 00:27:29.240
if I look at the family of matrices with eigenvalues four and four, well, one possibility
185
00:27:29.240 --> 00:27:43.080
occurs to me, one family with eigenvalues four and four has this matrix in it, four
186
00:27:43.080 --> 00:27:52.280
times the identity. Then another, but now I want to ask also about the matrix four,
187
00:27:52.280 --> 00:28:02.240
one zero. And my point here, here's the whole point of this, of this bad stuff is that
188
00:28:02.240 --> 00:28:09.720
this guy is not in the same family with that one. The family of matrices that have eigenvalues
189
00:28:09.720 --> 00:28:20.960
four and four is two families. There's this total loner here who's in a family of,
190
00:28:20.960 --> 00:28:35.280
right, just by himself. And all the others are in with this guy. So the big family includes
191
00:28:35.280 --> 00:28:45.680
this one. And it includes a whole lot of other matrices, all, in fact, in this two by
192
00:28:45.680 --> 00:28:54.480
two case, you see, what do I mean? What am I using this word family? In a family, I
193
00:28:54.480 --> 00:29:01.680
mean they're similar. So my point is that the only matrix that's similar to this is
194
00:29:01.680 --> 00:29:08.720
itself. The only matrix that's similar to four times the identity is four times the identity.
195
00:29:08.720 --> 00:29:18.080
It's off by itself. Why is that? If this is my matrix four times the identity and I take
196
00:29:18.080 --> 00:29:25.320
it, I multiply on the right by any matrix M, I multiply on the left by M inverse, what do
197
00:29:25.320 --> 00:29:36.240
I get? This is any M, but what's the result? Well, factoring out of four, that's just the
198
00:29:36.240 --> 00:29:43.840
identity matrix in there. So then the M inverse cancel the M. So I've just got this matrix
199
00:29:43.840 --> 00:29:53.000
back again. So whatever the M is, I'm not getting any more members of the family. So this
200
00:29:53.000 --> 00:30:04.960
is one small family, because it only has one person, one matrix. Excuse me. I think
201
00:30:04.960 --> 00:30:13.440
of these matrices as people by this point in 1806. Okay, the other family includes all
202
00:30:13.440 --> 00:30:22.960
the rest, all other matrices that have eigenvalues four and four. This is somehow the best one
203
00:30:22.960 --> 00:30:30.360
in that family. See, I can't make a diagonal. If it's diagonal, it's this one. It's in
204
00:30:30.360 --> 00:30:37.920
its own by itself. So I have to think, okay, what's the nearest I can get to diagonal?
205
00:30:37.920 --> 00:30:44.880
But it will not be diagonalizable. Do you know that that matrix is not diagonalizable?
206
00:30:44.880 --> 00:30:51.440
Of course, because if it was diagonalizable, it would be similar to that, which it isn't.
207
00:30:51.440 --> 00:30:58.320
The eigenvalues of this are four and four, but what's the catch with that matrix? It's
208
00:30:58.320 --> 00:31:06.360
only got one eigenvector. That's a non-diagonizable matrix, only one eigenvector. And somehow,
209
00:31:06.360 --> 00:31:14.800
if I made that one into a 10 or into a million, I could find an M. It's in the family. It's
210
00:31:14.800 --> 00:31:21.360
similar. But the best. So the best guy in this family is this one. And this is called
211
00:31:21.360 --> 00:31:33.400
the fjordon. So this guy, Jordan, picked out. So he like studied these families of matrices.
212
00:31:33.400 --> 00:31:42.280
And in each family, he picked out the nicest, most diagonal one. But not completely diagonal,
213
00:31:42.280 --> 00:31:46.680
because there's nobody, there isn't a diagonal matrix in this family. So there's a one
214
00:31:46.680 --> 00:31:55.200
up there in the Jordan uniform. Okay. I think we've got to see some more matrices in that
215
00:31:55.200 --> 00:32:02.440
family. So let me just think of some other matrices, whose eigenvalues are four and four,
216
00:32:02.440 --> 00:32:11.480
but they're not four times the identity. So, and I believe that all the examples we pick
217
00:32:11.480 --> 00:32:23.000
up will be similar to each other. And you see why. So with in this topic of similar matrices,
218
00:32:23.000 --> 00:32:30.340
the climax is the Jordan form. So it says that every matrix, I'll write down what the
219
00:32:30.340 --> 00:32:40.880
Jordan form, what Jordan discovered. He found the best looking matrix in each family.
220
00:32:40.880 --> 00:32:48.680
And that's, then we've got, then we've covered all matrices, including the non diagonalizable
221
00:32:48.680 --> 00:32:55.800
one. That's the point. That in some way, Jordan completed the diagonalization by coming
222
00:32:55.800 --> 00:33:01.840
as near as he could, which is his, his Jordan form. And therefore, if you want to cover
223
00:33:01.840 --> 00:33:08.600
all matrices, you've got to get him in the picture. It used to be, when I took 1806,
224
00:33:08.600 --> 00:33:14.960
that was the climax of the course, this Jordan form stuff. I think it's not the climax
225
00:33:14.960 --> 00:33:27.120
of linear algebra anymore. Because it's not easy to find this Jordan form for a general
226
00:33:27.120 --> 00:33:34.040
matrix, because it depends on these eigenvalues being exactly the same. You'd have to know
227
00:33:34.040 --> 00:33:41.320
exactly the eigenvalues. And you'd have to know exactly the rank. And the slightest change
228
00:33:41.320 --> 00:33:47.840
in numbers will change those eigenvalues, change the rank. And therefore, the whole thing
229
00:33:47.840 --> 00:33:57.560
is numerically not a good thing. But for algebra, it's the right thing to understand this
230
00:33:57.560 --> 00:34:12.200
family. So just tell me another matrix, a few more matrices. So more members of the family.
231
00:34:12.200 --> 00:34:20.440
Let me put down, again, what the best one is. Okay. All right. Some more matrices, let's
232
00:34:20.440 --> 00:34:29.160
see. What am I looking for? I'm looking for matrices whose trace is what? So if I'm looking
233
00:34:29.160 --> 00:34:34.320
for more matrices in the family, they'll all have the same eigenvalues, 4 and 4. So their
234
00:34:34.320 --> 00:34:42.360
trace will be 8. So why don't I just take like 5 and 3? I've got the trace right. Now
235
00:34:42.360 --> 00:34:50.280
the determinant should be what? 16. So I just fix this up. Shall I put maybe a 1 and a
236
00:34:50.280 --> 00:34:59.080
minus 1 there? Okay. There's a matrix with eigenvalues, 4 and 4, because the trace is 8
237
00:34:59.080 --> 00:35:07.320
and the determinant is 16. And I don't think it's diagonalizable. Do you know why it's
238
00:35:07.320 --> 00:35:18.440
not diagonalizable? Because if it was diagonalizable, the diagonal form would have to be this.
239
00:35:18.440 --> 00:35:23.200
But I can't get to that form because whatever I do with any M inverse and M I stay with
240
00:35:23.200 --> 00:35:28.880
that form, I could never get connect those. So I can put down more members. Here's another
241
00:35:28.880 --> 00:35:37.960
easy one. I could put the 4 and the 4 and a 17 down there. All these matrices are similar.
242
00:35:37.960 --> 00:35:44.600
I could find an M that would show that that one is similar to that one. And you can see
243
00:35:44.600 --> 00:35:51.520
the general picture is I can take any A and any 8 minus A here and any, oh I don't know
244
00:35:51.520 --> 00:35:59.280
whatever you put it. Anyway, you can see I can fill this in, fill this in to make the trace
245
00:35:59.280 --> 00:36:08.200
equal 8. The determinant equal 16. I get all that family of matrices and they're all
246
00:36:08.200 --> 00:36:17.360
similar. So we see what eigenvalues do. They're all similar and they all have only one
247
00:36:17.360 --> 00:36:27.840
eigenvector. So if you allow me to add to this picture, they have the same lambdas and
248
00:36:27.840 --> 00:36:37.320
they also have the same number of independent eigenvectors. Suppose if I get an eigenvector
249
00:36:37.320 --> 00:36:49.800
for X, I get one for A, I get one for B also. So N, same number of eigenvectors. But even
250
00:36:49.800 --> 00:36:56.080
more than that, even more than that, I mean it's not enough just to count eigenvectors.
251
00:36:56.080 --> 00:37:04.960
Yeah, let me give you an example why it's not even enough to count eigenvectors. So another
252
00:37:04.960 --> 00:37:15.320
example. So here are some matrices. Oh, let me make them 4 by 4. Okay, here's a matrix.
253
00:37:15.320 --> 00:37:23.280
I mean, like if you want nightmares, think about matrices like these. So a 1 of the diagonal,
254
00:37:23.280 --> 00:37:36.960
a 1 there. How many, what are the eigenvalues of that matrix? Oh, I mean, okay. What are the
255
00:37:36.960 --> 00:37:49.200
eigenvalues of that matrix? Please. 4 is arrows, right? So we're really getting bad
256
00:37:49.200 --> 00:37:58.240
matrices now. So I mean, this is like Jordan was a good guy, but he had to think about
257
00:37:58.240 --> 00:38:06.480
matrices that had like an eigenvalue repeated four times. How many eigenvectors does that
258
00:38:06.480 --> 00:38:17.360
matrix have? Well, I'm, eigenvectors will be, since the eigenvalue is 0, eigenvectors
259
00:38:17.360 --> 00:38:24.480
are B in the null space, right? And eigenvectors are got to be a x equals 0 x. So what's the
260
00:38:24.480 --> 00:38:33.400
dimension of the null space? 2. Somebody said 2. And that's right, hot, why? Because you
261
00:38:33.400 --> 00:38:40.120
ask what's the rank of that matrix? The rank is obviously 2. The number of independent
262
00:38:40.120 --> 00:38:46.680
rows is 2. The number of independent columns is 2. The rank is 2. So the null, the dimension
263
00:38:46.680 --> 00:38:54.520
of the null space is 4 minus 2. So it's got 2 eigenvectors. 2 eigenvectors. 2 independent
264
00:38:54.520 --> 00:39:07.960
eigenvectors. All right. The dimension of the null space is 2. Now, suppose I change
265
00:39:07.960 --> 00:39:21.920
this 0 to a 7. The eigenvectors are all still 0. How, what about how many eigenvectors?
266
00:39:21.920 --> 00:39:28.000
What's the dimension of the, what's the rank of this matrix now? Still 2, right? So it's
267
00:39:28.000 --> 00:39:34.440
okay. And actually, this would be similar to the one that had a 0 in there. But it's
268
00:39:34.440 --> 00:39:43.200
not as beautiful. Jordan picked this one. He picked. He put 1's. We have a 1 on the, above
269
00:39:43.200 --> 00:39:48.680
the diagonal for every missing eigenvector. And here we're missing 2 because we've got
270
00:39:48.680 --> 00:40:02.240
2. So we've got 2 eigenvectors and 2 are missing. Because it's a 4 by 4 matrix. Okay. Now,
271
00:40:02.240 --> 00:40:16.160
I was going to give you this second example. 0, 1, 0, 0. Let me just move the 1. Oh, not
272
00:40:16.160 --> 00:40:29.320
there. Off the diagonal and 0, 0, 0, 0. Okay. So now, tell me about this matrix. It's
273
00:40:29.320 --> 00:40:43.360
eigenvalues are 4, 0s again. Its rank is 2 again. So it has 2 eigenvectors and 2 missing.
274
00:40:43.360 --> 00:40:50.520
But the darn thing is not similar to that one. Account of eigenvectors looks like these
275
00:40:50.520 --> 00:40:59.240
could be similar, but they're not. Jordan, see, this is like a little, little 3 by 3 block
276
00:40:59.240 --> 00:41:06.440
and a little 1 by 1 block. And this one is like a 2 by 2 block and a 2 by 2 block. And those
277
00:41:06.440 --> 00:41:19.200
blocks are called Jordan blocks. So let me say what is a Jordan block? J, J block number
278
00:41:19.200 --> 00:41:31.600
i. So a Jordan block has a repeated eigenvalue, lambda i, lambda i on the diagonal. Zeroes
279
00:41:31.600 --> 00:41:43.840
below and 1's above. So there's a block with this guy repeated, but it only has 1 eigenvector.
280
00:41:43.840 --> 00:41:54.400
So a Jordan block has 1 eigenvector only. This one has 1 eigenvector. This block has 1 eigenvector
281
00:41:54.400 --> 00:42:00.520
and we get 2. This block has 1 eigenvector and that block has 1 eigenvector and we get
282
00:42:00.520 --> 00:42:12.640
2. So, but the blocks are different sizes. And that, it turns out Jordan worked out. Then
283
00:42:12.640 --> 00:42:29.000
this is not similar, not similar to this one. So I'm like giving you the whole story,
284
00:42:29.000 --> 00:42:36.920
well not the whole story, but the main themes of this story is, is here's Jordan's theorem.
285
00:42:36.920 --> 00:42:56.800
So, the first three square matrix A is similar to a Jordan matrix J. And what's a Jordan
286
00:42:56.800 --> 00:43:06.120
matrix J? It's a matrix with these blocks. Block, Jordan block number 1, Jordan block
287
00:43:06.120 --> 00:43:20.160
number 2 and so on. And let's say Jordan block number D. And there was Jordan blocks
288
00:43:20.160 --> 00:43:25.680
look like that. So the eigenvalues are sitting on the diagonal, but we've got some of these
289
00:43:25.680 --> 00:43:36.520
ones above the diagonal. We've got the number of blocks. The number of blocks is the number
290
00:43:36.520 --> 00:43:51.980
of eigenvectors because we get 1 eigenvector per block. So what I'm, so if I summarize
291
00:43:51.980 --> 00:44:00.560
Jordan's idea, start with any A. If it's eigenvalues are distinct, then what's it
292
00:44:00.560 --> 00:44:06.520
similar to? This is the good case. If I start with a matrix A and it has different
293
00:44:06.520 --> 00:44:14.440
eigenvalues, it's n eigenvalues, none of them are repeated. Then that's a diagonal,
294
00:44:14.440 --> 00:44:22.960
diagonalizable matrix. The Jordan block is, has the Jordan matrix is diagonal. It's lambda.
295
00:44:22.960 --> 00:44:38.400
So the good case, the good case J is lambda. All, there are D equals n. There are n eigenvector,
296
00:44:38.400 --> 00:44:48.760
n blocks, diagonal, everything great. But Jordan covered all cases by including these cases
297
00:44:48.760 --> 00:44:57.880
of repeated eigenvalues and missing eigenvectors. Okay, that's a description of Jordan. That's,
298
00:44:57.880 --> 00:45:05.920
I haven't told you how to compute this thing. And it isn't easy. Whereas the good case,
299
00:45:05.920 --> 00:45:15.880
the good case is what 1806 is about. This case is what 1806 was about 20 years ago. So
300
00:45:15.880 --> 00:45:22.320
you can see you probably won't have on the final exam the computation of a Jordan matrix
301
00:45:22.320 --> 00:45:33.880
for some horrible thing with four repeated eigenvalues. I'm not that crazy about the Jordan
302
00:45:33.880 --> 00:45:45.080
matrix. But I'm very positive about positive definite matrices and about the idea that's
303
00:45:45.080 --> 00:45:50.640
coming Monday, the singular value decomposition. So I'll see you on Monday and have a great
304
00:45:50.640 --> 00:46:05.640
weekend. Bye.