Support vector machines

A support vector machine (SVM) is a supervised machine learning algorithm that can be used for both classification and regression. SVMs are more commonly used for classification.

Given some data points, each belonging to one of the two binary classes, the goal is to decide which class a new data point will be in. We need to visualize the data point as a -dimensional vector, and we need to determine whether we can separate two such data points with a () dimensional hyperplane.

There may be many hyper planes that separate such data points, and this algorithm will help us to arrive at the best hyperplane that provides the largest separation. This hyperplane is called the maximum-margin hyperplane, and the classifier is called the maximum-margin classifier. We can extend the concept of a separating hyperplane to develop a hyperplane that almost separates the classes, using a so-called soft margin. The generalization of the maximal margin classifier to the non-separable case is known as the support vector classifier.

Let's take the first example. In this, there is one hyperplane that separates the red dots and the blue dots:

But imagine that the points were distributed as follows—how will we identify the hyperplane that separates the red dots and the blue dots:

The solution is to identify the hyperplane with SVM. It can execute transformations to identify the hyperplane that separates the two for classification. It will introduce a new feature, z, which is z=x^2+y^2. Let's plot the graph with the x and z axes, and identify the hyperplane for classification:

Now that we understand the basics of SVM, let's look at the areas where it can be applied:

Face detection
Image classification
Bioinformatics
Geological and environmental sciences
Genetics
Protein studies
Handwriting recognition