Ensemble Techniques

The basic principle that two heads are better than one applies also in Machine Learning. In many circumstances, if a classifier has a particular level of performance on a problem, a committee of such classifiers will have a better performance on that problem -- provided the committee is constructed appropriately and the `decision making' of the committee is managed properly. The objective of this paper is to explain why such ensembles of classifiers are effective and to provide an overview of the three most popular approaches to ensemble construction: Bagging, Boosting and Random Forests.