Combination of classifiers A combination or an ensemble of classifiers is a set of classifiers are combined to classify new examples. A combination of classifiers is often more accurate than the individual classifiers that make them up. This is Chapter 9: Combination of classifiers. | Chapter 9 Combination of classifiers Assoc. Prof. Dr. Duong Tuan Anh Faculty of Computer Science and Engineering, HCMC Univ. of Technology 3/2015 Outline 1 Introduction 2 Bagging 3 Boosting 4 ROC curves Introduction A combination or an ensemble of classifiers is a set of classifiers are combined to classify new examples. A combination of classifiers is often more accurate than the individual classifiers that make them up. A combination of classifiers is a way of compensating for imperfect classifiers. Each classifier is also called an expert. Bagging and boosting are general techniques that can be applied to classification as well as prediction problems. The methods for constructing ensembles of classifiers include Sub-sampling the training set Bagging and boosting each generate a set of classification models, M1, M2, ,Mk. Voting strategies are used to combine the classifications for a given unknown sample. Figure Bagging Given a set D of d tuples, bagging works as . | Chapter 9 Combination of classifiers Assoc. Prof. Dr. Duong Tuan Anh Faculty of Computer Science and Engineering, HCMC Univ. of Technology 3/2015 Outline 1 Introduction 2 Bagging 3 Boosting 4 ROC curves Introduction A combination or an ensemble of classifiers is a set of classifiers are combined to classify new examples. A combination of classifiers is often more accurate than the individual classifiers that make them up. A combination of classifiers is a way of compensating for imperfect classifiers. Each classifier is also called an expert. Bagging and boosting are general techniques that can be applied to classification as well as prediction problems. The methods for constructing ensembles of classifiers include Sub-sampling the training set Bagging and boosting each generate a set of classification models, M1, M2, ,Mk. Voting strategies are used to combine the classifications for a given unknown sample. Figure Bagging Given a set D of d tuples, bagging works as follows. For iterations i (i = 1, 2, k), a training set Di of d tuples is sampled with replacement from the original set of tuples, D. Note that the term bagging stands for boostrap aggregation. Each training set is a boostrap sample. Because sampling with replacement is used, some of the original tuples of D may not be included in Di, whereas others may occur more than once. A classifier model, Mi, is learned for each training set, Di. To classify an unknown tuple, X, each classifier, Mi, returns its class prediction, which counts as one vote. The bagged classifier, M*, counts the votes and assigns the class with the most votes to X. Algorithm: Bagging 1. for i = 1 to k do // create k models 2. Create bootstrap sample, Di, by sampling D with replacement; 3. Use Di to derive the model Mi; 4. endfor To use the composite model to classify tuple, X: 1. Let each of the k models classify X and return the majority vote; D: a set of training tuples k: the number of models in the ensemble. .
