Our project was to try to create algorithms to classify handwritten numbers by
accessing training sets from the MNIST database. Through trial and error, we used
many different machine learning techniques to do try to accomplish this.
The ability to classify using machine learning is useful because it can used for
different tasks like speech recognition and disaster resilience.
In class,
our professor showed us this website
which provides a nice model for ,
but even so there were many times in which the website was unable
to produce the correct number from the input.
This issue is often considered the equivalent to "Hello World" in machine learning, and its what we attempted to improve upon in our group project.
Computer's don't really "see" images the way we see them displayed.
To computer's, images are just a square grid of numbers, such as the picture on the right, which we can call a matrix.
Each number represents the color/brightness of a pixel, or a little square light on your computer's screen.
For our images, these numbers were in between 0 and 256, where each extreme represented a white or black pixel and any number between was a shade of gray.
So how can we get the computer to distinguish between what each hand written number means?
Especially because they're handwritten and no two of them will have the exact same pixel numbers?
This type of machine learning program is called classification. This means that what we're trying to do is get the computer to be able to differntiate
what image means what number. Or. slightly more formally, we're trying to get the computer to label our data. The obvious answer to this problem would be a neural network,
but let's build our way up by talking about other algorithms, or ways to code solutions. Just to give ourselves more options for approaching future problems.
Now that's quite the vocabulary word isn't it? Don't let this word scare you though. It just means the approach that we're going to take in order to solve this
problem. It's important to note that the algorithms we will be implimenting are supervised learning algorithms. This means that the information that we give
the computer will come with labels, so that it can know why it's wrong after it tries to classify a number.
There are quite a few kinds of classification algorithms we can apply here, but first I want to take a bit of a tangent and talk about a regression.
Regression refers to a different type of problem than classification. Instead of trying to program the computer to differentiate between ideas, regression is trying to
program the computer to try and predict what an output will be given the inputs. An example of this would be trying to predict the price of a house given how old it
is, the amount of rooms it has, etc. In a way, this is like finding the line of best fit on a scatter plot, except you might have more than one x and sometimes you
need the "parabola of best fit" or the "curve(s) of best fit" rather than a line. However, if your dots on your scatterplot don't really curve in much, you might actually use that line of best fit.
Using a line for these problems would mean applying a Linear Regression Algorithm, which means we'd program the computer to try a bunch of lines until it
found the line where the points were closest to it, or rather least far from it.
And voila! We have this beautiful thing:
That leads us into our first classification algorithm: Logistic Regression! Yeah, it has regression in the name of it, and that's because its also regression at the same time. It's easier if we look at the graph. This type of regression is non linear, meaning it's one of the "curves of best fit" types of regression. This curve is a fancy one called a sigmoid. . Which only has outputs between 0 and 1. Therefore, once we find the best sigmoid to represent our data, we can classify by assigning a label to numbers closer to 1 and the other those closer to 0. Here's a 2d sigmoid and a representation of the algorithm in 3d:
The next classification algorithm is K-Nearest Neighbor. This requires an already clustered, or labeled, data set. Again, it makes more sense if you refer to the graph at the bottom as you read. This algorithm then labels a unlabed data point by looking at it's "nearest neighbors", or the data points closest to it, and seeing which is the most common label amongst its neighbors. K refers to the fact that we have to decide how many neighbors to look at, and that this number can be any number we want it to be as long as it's not a large percentage of the dataset. In the example graph below, the clusters are labeled by color, and we know our unclassified point should be labeled as orange by the algorithm because its neighbors are manly orange.
)