In predictive modeling, boosting is an iterative ensemble method that starts out by applying a classification algorithm. The boosted classifications are then assessed, and a second round of model-fitting occurs in which the records classified incorrectly in the first round are given a higher weight in the second round. This boosting procedure is repeated a number of times, and the final classifier results from a merger of the various iterations, with lesser weights typically accorded to the very last rounds. The idea is to concentrate the iterative learning process on the hard-to-classify cases.
How Does Boosting Work?
Boosting works by starting with a classification algorithm and then assessing the results. The next step involves fitting a model again, but this time giving more weight to those cases that were classified incorrectly in the first round. This process repeats multiple times until finally, a merged classifier with different weights is created based on how well each case was classified.
What is a Classification Algorithm?
A classification algorithm is a function that takes input data (such as images, text, signals etc.) and assigns each item to one of two classes (e.g., cats vs dogs, spam vs non-spam). It does this by weighing the features of the input data, such as color, shape, size etc., so that the output separates one class into positive values and the other into negative values.
For example, if we have a dataset containing images of cats and dogs, then a classification algorithm will take each image as an input feature and assign it to either “cat” or “dog” based on the characteristics of the image.
Classification algorithms can be used on both structured data (like numerical data) and unstructured data (like images). For example, if we have a dataset with customer information like age and income level, then a classification algorithm can be used to segment customers into two different classes (e.g., high-income vs low-income). Similarly, if we have an image dataset with pictures of cats and dogs then again a classification algorithm can be used to separate out cats from dogs.
In addition to being used for categorization purposes, classification algorithms can also be used for prediction tasks such as predicting whether or not an email is spam or predicting whether or not someone will default on their loan payments.
Boosting involves a repeated, or iterative, classification procedure.