Build Your First Machine Learning Algorithm

0:00Hi, Sean here, and welcome to this skill where we're going to start our PCEI

0:04journey by taking a look at a simple vanilla Python example that will introduce us to some

0:11of the most important AI and machine learning concepts. So, you know, if you've looked at

0:17Python examples in the past, you're probably aware of the fact that there are lots and lots

0:22and lots of libraries out there for helping you do machine learning and AI. And while those are

0:28wonderful tools, once you've actually learned the basics, they're also a great way, as I've

0:34heard someone say, I don't remember who it was, to do machine learning and AI in Python without

0:39knowing anything about machine learning and AI in Python. And I'm assuming that is not your goal

0:44here, is to just get by by running the right lines of code and, you know, copying and pasting

0:51the right functions. So what we're going to do here is we're going to take a look at the

0:57tremendously popular and also, I would venture to say, quite fun k-nearest neighbor algorithm,

1:04right? KNN is typically the abbreviation for that. And how to implement it using just plain old

1:11Python. All right. So that brings us to this thing that I have on my screen. And this is two graphs

1:19displaying data from a so-called synthetic data set that I created, right? This is not a real

1:27data set gathered from real people. It's just one that I generated using Python. And the idea here

1:34is that this data set contains the shirt sizes worn by people with different measurements, right?

1:41So we have the person's chest size and waist size, as well as their height and weight. And because

1:46there's four features here, we'll talk about that word in a little bit more detail later,

1:51because there's four features or dimensions here, we have to display that on two separate graphs

1:56in order to see the real picture, okay? So the idea here, and this is something that I'm sure

2:03many people out there can sympathize with me on, is that there's not really a hard line between

2:10any of these measurements for shirt sizes. And so it can often be very difficult when buying a new

2:15shirt to know which size to buy. So what we're going to do here is we're going to use Python and

2:21this KNN, right? K nearest neighbor algorithm, to write a program that will allow us to enter our

2:28measurements and tell us what the most likely correct shirt size will be for us based on this

2:36data. All right, this sort of use case is very common in machine learning, because basically

2:42it involves us making decisions not based on any hard and fast rules, right? Like if there were just

2:48hard and fast rules, like if, you know, shirt companies just gave us the exact measurements

2:54that they used to draw the line between small and medium and large, well, that would be really easy

2:59to write a program for, right? We would just be able to say, you know, if chest size is greater

3:04than this number, if waist size is greater than this number, then, you know, assign the following

3:11size to that, right? But as it happens, that's not the case. And so that's where machine learning

3:18algorithms can really start to come in handy. So anyway, coming back to this data set that we have

3:23here, let's just do a quick visual inspection of this to see, you know, what we're looking at,

3:30right? So as we can see, the, you know, the dividing lines between these different shirt sizes,

3:38given, you know, the different measurements that we have here, are fairly clear. They're not,

3:43you know, they're not always foolproof, right? For example, we see that there's this green one

3:47here where we might expect to see an orange one. And over here, we see that there's, you know,

3:52green where we might expect red or green where we might expect orange. But for the most part,

3:58our data is clustered in a way that's going to make it fairly straightforward for us to use this

4:03KNN algorithm to solve it. All right. So here's what we want to do, right? Here's the ultimate

4:09goal of what we're going to be doing here. We already know all of these data points and

4:17their corresponding sizes, right? And that is in, if you open up the CSV file that I've included

4:24with this skill, right? Here it is right here. This just contains, you know, an ID. You can

4:29pretty much just ignore that. That's just a unique identifier for each of these measurements.

4:33We have height in centimeters, weight in kilograms, chest in centimeters, and waist

4:38in centimeters, as well as the corresponding size for each of the data points or rows in this file.

4:46Okay? So the key point here is that we already know what shirt sizes these people with these

4:52measurements wear. But what we want to be able to do is create a Python program that based on

4:59this data will be able to predict where a new data point would fall, right? So if we have a

5:06new person whose measurements fall around here on these two dimensions and around here on these two

5:13dimensions, right? Which shirt size would they be most likely to need? Well, the first and most

5:19important thing to understand when getting started with machine learning with problems like this is

5:24that there's generally not a single so-called best approach for any one problem, right? There

5:32might be certain approaches that are better than others. And certainly there are approaches that

5:36just don't make sense for certain situations. But even a problem that's as straightforward as

5:41this one has a variety of different approaches, right? A variety of different algorithms or

5:46strategies that we could use to try and answer questions like this one, right? Like which is

5:52what shirt size would be best for this person here based on these measurements?

5:57So going back to the KNN or K nearest neighbor algorithm, I'm just going to say KNN from now on

6:04because I'm tired of having to edit out myself saying K nearest neighbor and, you know, butchering

6:11it, right? All right. But KNN, the way that this works or the solution to this problem that KNN

6:18proposes is to take this unknown data point and find which data points are closest to it.

6:27And then simply take the average of what those data points are, right? So let's just take a

6:33simple look at what this would look like if we wanted to make the decision for this one based

6:38on what the three closest points are. So let's say that these three are the closest points,

6:44orange, green, and green. And I'm just going to do this in two dimensions here because it's

6:48kind of hard to tell what's closest to what in four dimensions. But let's just imagine that

6:53these are the three closest points, green, green, and orange. Well, in this case, we would say that

6:59this should be green or large because there are more large data points that are closer to this one

7:07than medium. OK, so this is a very simple heuristic, a very simple algorithm for using

7:14data to make decisions. And the nice part about this is that it's also fairly straightforward to

7:19implement in Python, which is what we're going to do, as I said, in the rest of this skill.

7:24So anyway, with that in mind, in the next video, we're going to start implementing this algorithm

7:30so that we can use K nearest neighbor in order to assign shirt sizes to unknown data points.

0:00All right. Well, now that we've discussed the problem at hand and we know the algorithm that

0:05we're going to be using in order to help solve it, let's get started by implementing this

0:10algorithm. And once again, we're just going to be using plain old vanilla Python in order to do

0:14this because, you know, it'll really help us to focus in on some concepts and terminology,

0:20and also just use our problem solving skills to implement a basic machine learning algorithm.

0:25So here's what we're going to do. We're going to start off by creating a new

0:28file. And again, I'm just doing this in a simple directory. So you're just going to want to have

0:34Python installed if you're going to want to follow along. So I'm going to create a new

0:37file and we'll call this something like knnbasics.py. And the first thing that we're going

0:43to need to do here, and for now I'm just going to close these two images. The first thing we're

0:48going to do here is we're going to need to load this CSV file into our file, right? There's not

0:53really a whole lot we can do with our data until we have it in our program where we can manipulate

0:59it and, you know, work with it in different ways. So here's what this is going to look like. We're

1:03going to say import CSV, and this is a built-in module in Python, so this is still technically

1:10vanilla Python, right? We're going to say with open, and then we're going to open our file here,

1:16which is called shirt-sizes.csv, right? And you can download this from above in this skill if

1:23you want to follow along. All right, and then we're going to say as file, and all we really

1:29need to do to load this in is read in each thing as a line and then convert it to a list, right?

1:36It's going to be as simple as that. So here's what this is going to look like. We're going to say

1:39reader equals csv.reader file, and then we're just going to say rows equals list reader, okay? So

1:49that's all we need to do there, and now we should be able to print out our rows to the console by

1:56saying print rows, and if we run our program here, which we can do by saying, let's just wait for

2:03that there. There we go. We can do that by saying python3 knnbasics.py, your exact command for

2:09running Python may vary, but I'm going to be using Python 3 here for this course, and sure enough,

2:13we see that this has successfully loaded all of that data into our Python program. Now, notice

2:20that the first line here contains all of the column headers, and so there's probably a better

2:26way that we can do this by using csv.dictionary reader instead of csv.reader. So let's try that,

2:35and that should improve things a little bit for us. It's always a good idea to,

2:39you know, actually take a look at the data that you've loaded before you start working with it,

2:43which is, you know, hopefully what that little demo showed you, and sure enough, we see that

2:48this has id, height centimeters, weight kilograms, chest centimeters, and waist centimeters, as well

2:54as the size all in their own dictionaries, right? So each one of these dictionaries represents a

3:00separate data point that, you know, forms part of our data set. So it's at this point that I'd like

3:07to just pause for a moment and talk about a little bit of terminology that is going to be very

3:12relevant to us going forward. The first thing is you've probably already heard me say half a dozen

3:17times the word data set, right? Now, this one's pretty self-explanatory. It basically just refers

3:23to a large collection of data, right? So in our case, this is the shirt size data that we've been

3:30working with that we're planning on using for a specific purpose, okay? Now, inside that data set,

3:38we have a number of what are known as data points, all right? And a data point just refers to an

3:44individual collection of, you know, values in all of the columns that this data set contains,

3:53right? So if our data set were, I don't know, geographical data, and we had the latitude,

3:59the longitude, and the elevation of different points on Earth, then a single data point would

4:08consist of a latitude value, a longitude value, and an elevation value, right? So that would be

4:14one data point, okay? All right, so that's data sets and data points. And this brings us to two

4:22other words that you've already heard me use, at least I think so. And those are labels and

4:30features, right? So features in a data set basically just refer to the columns, right? Or

4:36the individual types of data that each data point can have. So in our case here, with this one that

4:43I just drew out, latitude, longitude, and elevation would each be features of the data set. And in our

4:51shirt size, right? Let's just open that back up here. We'll open up shirtsize.csv. The features

4:56in this data set are the person's height, weight, chest size, and waist size. Now, what about this

5:03size thing, right? Is this a feature? Well, this is actually what's referred to in many cases as a

5:10label because this is the thing that we're trying to predict using our machine learning algorithm,

5:18right? So in our case, we're trying to use these features, right? The height, weight,

5:23chest size, and waist size to predict what the shirt size should be for a given person.

5:30So that's the difference in machine learning between labels and features. The features are

5:36typically the things that we're going to use, that we're going to feed into the algorithm.

5:40And the labels are typically the things that we're going to be predicting with the algorithm,

5:46all right? That's just kind of a rough definition there, but that's the one we're going to use for

5:50now. So anyway, just to kind of draw that back on here, let me just hide all these things and bring

5:56my drawing back. Each data point has features, all right? Let's just write that here. And labels or,

6:06you know, a label could be one or multiple labels depending on how many pieces of information are

6:12contained in there. And in case you need a little way to help remember this, features are sort of

6:18like the clues, whereas labels are like the answers in whatever problem it is we're trying to solve,

6:26right? And, you know, these terms will become a little bit more useful as we go along, but I just

6:30wanted to get those out of the way right now, just while we're in the process of loading our data.

6:37All right. So at this point, we have our data set successfully loaded into our Python program.

6:43And so here's the next thing that we're going to do. We're going to create a function that will

6:48tell us the distance between any two points in multiple dimensions, right? And specifically

6:55with the four features that our data set here has. So, you know, this is going to be used in our

7:02KNN algorithm because when we have many points like this, right, when we have points here,

7:08and then we have a new point that we want to know the label for, we're going to need to figure out

7:13which of these points are closest to this point by basically just calculating the distance between

7:19this point and all of the other points in our data set. Now, this might sound computationally

7:26expensive, and it certainly can be for larger data sets, but we'll take a look at that in a

7:31little bit more detail, what some things that can be done to improve this are. But, you know,

7:37anyway, we're going to create a function that will just tell us for any two points, right,

7:41for example, this point and this point, how far away they are. All right. Now, the good news here

7:47is that there's a very commonly known equation for doing this, and this is called the Pythagorean

7:52theorem. I'm not going to try spelling that out because frankly, I'm not quite sure that I could

7:56do it right now, but the good news is that while the Pythagorean theorem is typically taught in

8:02two dimensions, right, a squared plus b squared equals c squared, it actually works for higher

8:06dimensions as well. So if we want to find the closest points in four dimensions, which our

8:12data set does have four dimensions, then we can generalize it to a squared plus b squared plus c

8:18squared plus d squared equals e squared, right? It works in higher dimensions as well. So here's

8:23what this is going to look like. We're going to create a function in Python called get distance,

8:31and this is going to take two points, right? We're going to call those point one and point two,

8:37and it's going to return the distance between them by basically just using the Pythagorean

8:41theorem. So here's what that's going to look like. First of all, we're going to need to access the

8:45values on each of these points. So let's just open up the output from our previous,

8:50uh, you know, from the previous time we ran the program. So we're going to want to get the height

8:55in centimeters. In fact, what I'm going to do is I'm just going to copy and paste these things here

8:59so that I won't have to type all of them out. All right, we'll copy those like so, and then I'm going

9:04to just paste that right here, and we're just going to use the, uh, keys there. All right, so we'll say,

9:11uh, height centimeter, weight kilograms, chest centimeter, and waist centimeters, and those are

9:18going to be the four dimensions that we're going to access on each point. So here's what this is

9:23going to look like is we're going to say dimensions. In fact, we'll call this something like

9:28dimension keys equals, and we're just going to put all of those in a list because that'll make it a

9:33little bit easier here. So let's just add commas to those and square brackets, and now we're just

9:38going to say four key in dimension keys. All we're doing here is we're just saying, okay, we have two

9:47points here, right, point one and point two. All right, um, how far are they in the x direction,

9:54and how far apart are they in the y direction, and we're just generalizing this to more than

10:00two dimensions. Okay, so anyway, here's what that's going to look like. We're going to say

10:04something like total equals, and then we're just going to say zero, and now we're going to say

10:11total plus equals, and then we're going to say p1, and then we'll get the key like so minus p2

10:21key, and we're going to square that, and you can square things in Python just by putting

10:26parentheses around these, and then using the double asterisk operator and saying two, right,

10:32so that's going to give us the square of the distance between these two points in that single

10:37dimension, and now we just need to return the square root of that total, right, that's how

10:44you solve the equation a squared plus b squared equals c squared is you can just say that c

10:52is equal to the square root of a squared plus b squared. All right, if this is giving you

10:58nightmares by the way that I'm writing algebraic equations out here, then please feel free to

11:03ignore them. I'm just using that to give a little bit more explanation behind what I'm doing here.

11:09All you really need to know though is that this function is going to do basically what it says

11:14in the name, right, it's just going to give us the distance between any two points in our data set as

11:20well as other points that we might want to label, all right, so this is going to be very important

11:25for implementing this algorithm, and so the last thing here we just we're going to say return math

11:30dot square root, and we're going to return the square root of the total, and so here let's just

11:36say import math up at the top, and that should be all we need to do. All right, so let's just test

11:41this out. In order to make sure that this is working, we're just going to get the distance

11:44between the first two points in our data set, and so here's what that'll look like. We're going to

11:50say print, and then we'll say get distance, and then we're going to use rows index zero and rows

11:59index one as our data points. Now let's just take a look at what those values are to get some idea

12:06of what those should be. So this is the first row here, and this is the second row here. All right,

12:12so as we can see they're fairly close to each other, and you can do the calculation by hand

12:17if you really want to know, but let's just run our application here, or run our script rather,

12:22by saying python3 knnbasics.py, and uh-oh, this is why we run our script to make sure that,

12:29you know, to make sure that it's working so far, because it's way easier to fix when it's nice and

12:35small like this than when you've tried to implement the entire algorithm and left a few things like

12:39this out. So you may have noticed previously, and here let's just go back to where we were before,

12:47let's just print out our rows again. You may or may not have noticed that the measurements here,

12:54even though we know that they're supposed to be numbers, Python has actually interpreted these

12:59as strings, right? So chest centimeter here is 100.8 as a string, which is a very different

13:06thing in Python from 100.8 as a number, and this is a very important thing to pay attention to

13:12when doing machine learning in Python, is you need to make sure you're getting the right

13:17data types. All right, so here's what we're going to do. We're going to do our first example of

13:21data cleaning, right? Data cleaning is where you take data that's not quite in the format that you

13:26need, or that has some other sorts of problems, and there's lots of problems that data can have,

13:31as you'll see later on in the course, and we're going to convert it into data that fits our

13:37purposes here for our program. So here's what this is going to look like. We're going to say,

13:41well, first of all, we'll create one called rows here, and we'll rename this one to rows raw,

13:47right? Basically meaning that that's the raw data, which is not quite in the format that we need it

13:51in, and then we're going to say rows empty list, and we'll say for raw row in rows raw,

14:00we're going to say that we want to append a new dictionary to rows, so we'll say rows.append,

14:05and here's what this is going to look like. We're just going to take height centimeters,

14:09and we're going to set that to, we'll parse this as a float here. We'll say float raw row,

14:17and then we'll say height centimeters there, and then we're just going to do that same thing with

14:22the other columns, right? So we're going to copy that, and I'll just paste that underneath like so,

14:28and we'll just change this one to weight in kilograms. In fact, I'll just copy this

14:33and paste it, all right? There we go. That'll help us avoid typos, and then we're going to do

14:38chest centimeters. We're going to replace that here like so, and then we'll do waist centimeters

14:45and replace this one here, all right? Now, there probably is a slightly more concise way that we

14:50could have done this here, but well, we'll just leave it the way it is. So anyway, oh, here,

14:55let's change that to row.append. I don't know why I wrote appends, and now let's print out our rows,

15:01and they should now be numbers, so let's just try that again, and sure enough, we see that these are

15:06now numbers, which means that we should be able to go back to testing our get distance function,

15:12so let's uncomment that here. We're going to run our code, and sure enough, that gives us the

15:18distance between those two points. Now again, I'll leave it up to you to figure out whether that is

15:23the right answer or not, but if you want to take my word for it, that is the right answer. So anyway,

15:29what we've done here is we've implemented a very important function that will tell us the distance

15:34between any two points, and so the next thing we're going to do is we're going to see how we

15:39can use this function to actually put the KNN algorithm together and make a decision for a new

15:47point.

0:00All right. Well, at this point, we have this get distance function, which will tell us how far

0:04apart any two points are in our data set. And we also are loading in all of our data and converting

0:11it to the right type so that these end up being numbers instead of strings as they were before.

0:17So really, all that's left for us to do here is write some logic that will use this get distance

0:23function to find the closest points for some unknown point that we want to add a label to.

0:32All right. So here's what this is going to look like. The first thing we're going to need to do

0:35in order to get the closest points is we're going to need to basically just create a function called

0:41get closest points. OK, so we'll say get closest points. And what this one's going to do is this

0:48is going to take an unlabeled point. All right. Let me just spell that like that. There we go.

0:55And it's going to take all of the points that we want to basically measure this unlabeled point

1:01against. Right. So we'll say something like labeled points for that argument. And the last

1:08thing that we're going to do is we're going to add one more parameter that will be the K. And

1:14in case you were wondering what the K nearest neighbors are, what the K was, that is in K

1:19nearest neighbors, K is basically any integer number that refers to how many of the nearest

1:25neighbors we're going to find that, you know, we'll get the labels from. Right. So K could be

1:31three. K could be five. Right. Typically, it's an odd number or even a prime number. Anything that,

1:39you know, we can use to avoid a tie. OK, so anyway, we're just going to you know,

1:44we'll probably use three just for our demo, but it's nice to design this function from the outset

1:50to allow us to pass in different values there. So here's what this is going to look like.

1:55We're going to loop through our labeled points and calculate the distance between each one of

2:01those and our new unlabeled point. So here's what this is going to look like. We're going to say

2:06for point in labeled points. All right. We're going to say in here what we're going to do is

2:13we're going to say distances and create a list there. What we're going to do is we're going to

2:19say distance equals get distance. All right. And then we're going to pass the point in as well as

2:28the or sorry here, we're going to pass the unlabeled point in first rather. And then we're

2:34going to pass in the new point, which is the, you know, labeled point that we're currently looking

2:39at. Awesome. So now that we have the distance, here's what we're going to do. We're going to

2:45append this onto the distances list by saying distances dot append. And then instead of just

2:51storing the number, we're also going to need to know what label is associated with each distance.

2:56We'll use that in order to, you know, return the appropriate label here in just a minute.

3:02And so we're going to say distances dot append, and we're actually going to

3:05add a dictionary onto this that will have the distance. All right. As one of the entries. So

3:11distance, distance, and then the label for that distance is going to be the label from the point

3:18here. And we can get that by saying point. And then we're going to get the size entry from that.

3:26All right. So that's what's going to happen there. So let's just test this out right now.

3:30We're going to say here, we'll just say print distances after we're done with that. And what

3:37we hope is that we'll have a list of dictionaries, each containing a distance value and the associated

3:45label for that value, because what we'll do after that is we'll actually sort this and get the

3:50closest points. So here's what that'll look like. Let's just run this, but instead of calling print

3:56get distance, we're just going to call, get closest points. And what we're going to do is

4:01we're going to create a new unlabeled point. Well, actually, um, one thing that's nice to do to test

4:08this algorithm out is because we already know what the label is for the first one here. We're just

4:14going to use the first point as our quote unquote unlabeled point. And then we'll use the rest of

4:19them in order to calculate what the label for this should be. Now, it's not always going to return

4:24the, the label that we know should be on this, just because this point might be in a weird

4:31position, right? It might be in the middle of a bunch of other points that have a different label,

4:35but nevertheless, it just makes it easy so that we don't have to come up with our own numbers here

4:40for each of these, uh, you know, for each of these features. So here's what this is going to look

4:45like. We're going to say, get closest points. We're going to say, um, here, up here, we have

4:49rows is what we want. So we're going to say rows index zero for the unlabeled point. And then for

4:55the labeled points, we're going to say rows index, and then we'll use a slice to get the rest of

5:00them. And then we'll say maybe three. It doesn't really matter what we pass in there at this point.

5:05Um, just matters that we pass in something. So let's give this a try. We're going to run

5:09our, uh, code and oops. Uh, the reason we're getting this is because we actually inadvertently

5:15removed the size, uh, from our rows. So we're just going to say size here. All right. And that's

5:22just going to be, um, raw row size, right? So we don't need to convert that to a number. In fact,

5:27we don't want to convert that to a number because it's not a number, right? It's just a size label.

5:33So let's try this again. And sure enough, what we'll see is that we end up with a list of distances

5:40and labels associated with those distances. So let's finish this thing up here. What we're going

5:46to do is we're going to sort them and we can do that by saying sorted. And then we're going to

5:52pass in the distances that we have, but, um, we're going to want to sort those in ascending order by

6:01their distance key, right? We're going to want to sort them in ascending order by, oops,

6:06let's just scroll up here by this thing here so that we get the smallest distances first,

6:11and we can take a slice. So here's what we're going to need to do for that. We're going to need

6:14to say, um, key equals, and then we're going to need to use a Lambda expression here so that we

6:21can say D and then we'll say D, uh, and that'll be distance. Okay. So that's just going to cause

6:27it to sort by the distance. That's all that is awesome. So now that we have that, we're just

6:32going to say, um, uh, sorted distances. There we go. Equals. And then that's going to be sorted

6:39distances, key blah, blah, blah, a little bit repetitive there, but you know, it just prevents

6:43us from having to type this out again if we need it. And, uh, anyway, now that we've done that,

6:49we're going to return what the, uh, K closest points are. And here's what that's going to look

6:56like. We're going to say sorted distances, and we're just going to say, oops, here, we'll say

7:01return sort of distances. And then we're going to say, uh, slice, and we're going to get zero

7:07through K like that. All right. So that should give us the closest points. So we're going to

7:13print those out and see what those look like. And here, just to make it a little bit easier,

7:16we'll start by getting the 10 closest points, just so that we can see what they look like.

7:21So let's run this again. And sure enough, what we see is that we start off with the closest

7:28distances, right? So four points, something, four points, something, four points, something.

7:32And then we go up to some of the further distances, like six points, something,

7:36seven points, something, and so on. All right. And so the last thing that we're going to need to do

7:40here is use these closest distances to sort of like vote on what the label for this point should

7:51be. So here's what that's going to look like. We're actually just going to use the Python

7:56collections module. So we're going to say from collections, there we go. Uh, we're going to

8:02import counter. And now if we scroll down here, what we're going to do is we're going to say,

8:08um, here, we'll just say, uh, first of all, we're going to get the closest points. So we'll say

8:13closest equals. And then what we're going to do is we're going to get the, uh, labels here. So

8:21we'll say closest labels. All right. And then we're just going to use a list comprehension here.

8:26We're just going to say C and we'll get the label, uh, entry from that for C in closest.

8:34And that should give us the, uh, labels. So now in order to basically take a vote on those,

8:40we can just use that counter thing. Um, right. This thing up here in order to get the most

8:45common element from that list. So here's what that's going to look like. We're going to say

8:50counter, and then we're going to paste those labels in there. Right. And then we're going

8:55to say dot most common, and we're going to get the first most common one there. And then we just

9:00have to say zero, zero at the end there. If you don't know how counter works, right. Or counter

9:04most common, don't worry too much about it. Just know that this will get us the most common one.

9:08This is just the most sort of Pythonic clean way of doing it. All right. So let's get the closest

9:16label here. Okay. And what that's going to look like, well, let's just print it out. So we'll say

9:21closest label like so. All right. If we run this, what we'll see is that the closest label is in

9:28fact extra large. Okay. And as we remember from the data set itself, that was the correct label

9:35for this one. That's not always going to work out that way, but it does seem to have worked out that

9:39way in this case. All right. So now all we have left to do is rewrite this into sort of like a,

9:44an all encompassing function that we can use to label pretty much any, uh, new person, right. Or

9:51suggest a shirt size for any new person based on their measurements. So here's what this is going

9:55to look like. We're just going to say, um, define, and we'll say something like, uh, label shirt size,

10:03or you know what? We'll say something like predict shirt size. We'll be a little bit better there.

10:08And then what we're going to do is we're going to take the new measurements here. All right. So what

10:14that's going to look like is we'll say, uh, measurements. All right. And now we're going to

10:21basically just put all of this inside that function. All right. And then let's just go

10:27through and clean this up a little bit. So we already have get distance inside here. We could

10:32make those, you know, we might want to move those outside of this function here. In fact, here,

10:37let's do that just to make it a little bit cleaner here. So we'll move, uh, get distance and get

10:42closest points outside of there. All right. And those are pretty self-contained functions. So

10:46those shouldn't be, uh, there shouldn't be any problems there. And now that we've done that,

10:50what we're going to do is we're going to just say that we want to return the closest label.

10:58And there we go. We'll also change this to some other value besides 10. Maybe we'll do like

11:04the seven nearest neighbors there. And well, what we're going to do is we're going to say,

11:10predict shirt size. All right. And we are going to actually define a new set of measurements here.

11:16Let's just look through our shirt sizes here and just pick something that's kind of close to one

11:20of these. So, um, maybe we'll do for the, Oh, here, first of all, let me just copy these here.

11:27And then we'll put those into here and we'll use those as the keys like, so that'll just make it a

11:32little bit easier there. Cause we won't have to type each of them out. All right. And now that

11:37we have those, here's what this is going to look like. We're going to say that the height in

11:40centimeters, let's just, um, pick something kind of roughly in this range here. So we'll say,

11:46uh, here we'll pick something between two of the mediums so that we know that it

11:49should probably be medium. So we'll do like 177 for height in centimeters, 177.0 for weight in

11:57kilograms. Here's what this is going to look like. We're going to go and take a look between some of

12:02the mediums here again. So we'll pick maybe a 79 for the weight. All right. And then for chest

12:10centimeters, we're going to use another value from that or sort of like an in-between value here.

12:16So we're going to pick, uh, let's see here. We'll do 98 perhaps. So we'll say 98.0 and I'm just

12:23doing 0.0 for the sake of simplicity here. And then for waist centimeters, what we're going to

12:27do is we're going to say, uh, let's just go back here and take a look at some of those. Uh, we'll

12:32do maybe a 90. Okay. For that. So we'll say 90.0. And now that we've done that, let's see what this

12:39one predicts. All right. And here we need to actually print that out. All right. And if we

12:45run this now, what we're going to see is, Oh, it looks like it gave us XL. Oh. And the reason for

12:50that is that we forgot to actually pass in these new measurements here instead of just picking the

12:57first one there. So here's what this is going to look like. We're going to swap this out. So

13:01instead of rows zero, we're going to say measurements. All right. And then what we're

13:06going to do is we're going to swap that out with just plain old rows and that should be all we need

13:10to do there. So let's try running this again. And sure enough, that now gives us medium, which is

13:16our expected value. So anyway, that is the basics of implementing a K nearest neighbor algorithm.

13:23So in the next video, I'm going to give you a challenge that will basically require you to

13:28take this a little bit further and play around with some of the different pieces and the available

13:33alternatives to them.

0:00All right, well, now that we've finished

0:02our simple implementation of the

0:04k-nearest-neighbors algorithm,

0:06it's time for you to do a challenge.

0:08And in this challenge, you're gonna be creating

0:10an alternative distance function

0:13for calculating the distance between two points.

0:16Now, Sean, what on earth do you mean

0:18alternative distance function?

0:21Well, the distance function that we just used

0:24in this example is what's often referred to

0:26as the Euclidean distance, right?

0:29This is the, you know, Pythagorean theorem at work,

0:32basically, where if you have a point here

0:35and a point here, the actual distance between them,

0:38the diagonal distance, if you wanna call it that,

0:41is the square root of the sum of the squares

0:46of both those sides.

0:47And that might sound confusing,

0:48but it's just the good old A squared plus B squared

0:50equals C squared thing at work, right?

0:53Which, as we've learned in this skill,

0:55applies to higher dimensions as well.

0:57However, this way of calculating distances

1:01does have some important issues,

1:03and we'll talk about those in a little bit more detail

1:05later in the course, but one of those issues

1:09is that because it involves squares,

1:12the Euclidean distance, right, spelled like this

1:15after this guy named Euclid,

1:18but the Euclidean distance overemphasizes

1:21occasional large variations, right?

1:24So because we're doing like A squared and B squared,

1:27if, oops, I wrote B as the exponent there,

1:30B squared is what I meant to say there,

1:32because we're doing A squared and B squared,

1:35when we have the occasional value that's like,

1:37you know, way out of range, right,

1:40or way out of the normal range,

1:41and A or B happens to be very large,

1:45then squaring that gives an unusual amount of weight

1:49to that particular dimension,

1:52and that's often not what we want.

1:54Now, if this is not making sense to you,

1:55don't worry too much about the exact details.

1:57Just know that there are certain situations

1:58where the Euclidean distance is not desirable,

2:02and so in place of this,

2:04there are other distance functions,

2:07and the one that you're gonna be working with

2:08in this challenge is known as the Manhattan distance.

2:13All right, now this is, I think,

2:14a somewhat humorously named distance,

2:17because as we just saw, the Euclidean distance

2:21is basically the shortest distance,

2:24the length of the shortest line between two points in space,

2:29but the Manhattan distance is literally just A plus B,

2:36right, now why is this?

2:37Well, think about Manhattan.

2:39If you've ever been to Manhattan,

2:40you know that the entire place

2:42is laid out like a grid, right?

2:44So if you're trying to get from one place

2:47to another in Manhattan,

2:49you have to travel along the roads,

2:51which go like this, right?

2:52They're like a grid,

2:53and so the shortest distance between two points

2:56in Manhattan is not as the crow flies, right,

3:00unless you have a helicopter,

3:01and you know, some people do,

3:03but for most of us,

3:04the shortest distance between two points in Manhattan

3:07is going along the streets, right,

3:10or the sidewalks or whatever,

3:12and so that's where this Manhattan distance idea comes from.

3:16Now this distance, it might seem kind of arbitrary

3:19to just do A plus B instead of the square root

3:21of A squared plus B squared,

3:23but this actually has some very nice properties

3:27in certain situations.

3:28I'm not gonna go into much detail on that right now,

3:30but anyway, this is your challenge

3:33is you're going to implement a Manhattan distance function

3:38and use that to replace

3:40our regular Euclidean distance function

3:43in the algorithm that we just implemented here.

3:46So that's your challenge,

3:47and this should take you maybe

3:48about five to 10 minutes to complete.

3:51So once you've given it a try,

3:52you can feel free to move on to the next video

3:54where I'll walk you through the challenge.

3:55Oh, and one last thing that is probably relevant here

3:59is that just like how the Euclidean distance

4:01generalizes to multiple dimensions, right,

4:03so you can do A squared plus B squared,

4:06but you can also do A squared plus B squared

4:07plus C squared plus D squared

4:09plus as many dimensions as you want,

4:11long as you take the square root of those, of course,

4:13you can do the same thing with the Manhattan distance, right?

4:16So just like how the Manhattan distance

4:17between two points is just A plus B,

4:21well, in two dimensions, that is,

4:23the distance between two points in three dimensions

4:26is A plus B plus C,

4:28with C being the distance between them

4:30in the third dimension,

4:31and if you're in four dimensions,

4:33that would be the distance between them

4:34in the fourth dimension.

4:35So anyway, best of luck on your challenge,

4:37and I'll see you in the next video.

0:00All right, well, hopefully you gave this challenge a try,

0:02so let's take a look at the solution.

0:04So all you really had to do for this challenge

0:07was define a new function called get Manhattan distance.

0:12Probably the hardest part of this challenge

0:13was spelling Manhattan.

0:15So we're gonna say get Manhattan distance,

0:17and from point one to point two,

0:20the Manhattan distance was just the sum

0:23of the distances between each point.

0:26So, you know, another hard aspect of this too

0:30is that sometimes this could turn out to be negative.

0:33So you did actually have to add an absolute value into here.

0:37If you didn't get that, don't worry too much about it.

0:39I didn't mention that in the challenge video.

0:41So anyway, just make sure to fix that going forward

0:44because we are gonna be reusing this code

0:46in the following skills

0:48to learn more about machine learning.

0:51But here's what we're gonna do.

0:52We're just going to reuse a lot of this code

0:54from the get distance function,

0:57which we'll also rename in Euclid's honor

0:59to get Euclidean distance.

1:03And now for get Manhattan distance,

1:06we're gonna say total equals zero,

1:08and then we're gonna say total plus equals,

1:10and we're literally just gonna get the difference there

1:12between those two,

1:14and then we're just gonna get the absolute value

1:15by saying abs and wrapping that there.

1:19And then we don't need to take the square root either,

1:22right, it's just the total of all of those.

1:25So that should be all we really need to do there.

1:29Let's just swap out the get distance function down here

1:32with the get Manhattan distance function.

1:35And oops, sorry, that should be inside get closest points.

1:39So we're gonna swap this one out.

1:41There isn't even a get distance function anymore.

1:43So we're gonna replace that with get Manhattan distance,

1:47and that should be all we really need to do there.

1:49So let's just save this and try running it again.

1:52And what we should see

1:53is that we'll still get similar results.

1:55Typically the Manhattan distance

1:57and the Euclidean distance

1:59don't give significantly different results

2:01for fairly normal situations,

2:06like the one that we saw here,

2:07where there's not really any outliers.

2:09The place where you start to see,

2:11or the situations rather,

2:13where you start to see some major differences

2:14is if we had someone like way down here

2:17who wore extra large clothing, right,

2:19that might skew things a little bit, right?

2:21Or if we had someone like way over here,

2:24or someone way over here,

2:26if we had some unexpectedly large distances,

2:29that's when the Manhattan distance

2:30would tend to give more accurate answers

2:33than the Euclidean distance.

2:35It's not always better,

2:36but it is better in certain cases like that.