Bài giảng Khai phá dữ liệu (Data mining): Ensemble models, chương này trình bày những nội dung về: introduction; voting; bagging; boosting; stacking and blending; learning ensembles; methods of constructing ensembles; bias-variance tradeoff; simple ensemble techniques; . Mời các bạn cùng tham khảo chi tiết nội dung bài giảng! | Trịnh Tấn Đạt Khoa CNTT Đại Học Sài Gòn Email trinhtandat@ Website https site ttdat88 Contents Introduction Voting Bagging Boosting Stacking and Blending Introduction Definition An ensemble of classifiers is a set of classifiers whose individual decisions are combined in some way typically by weighted or un-weighted voting to classify new examples Ensembles are often much more accurate than the individual classifiers that make them up. Learning Ensembles Learn multiple alternative definitions of a concept using different training data or different learning algorithms. Combine decisions of multiple definitions . using voting. Training Data Data 1 Data 2 Data K Learner 1 Learner 2 Learner K Model 1 Model 2 Model K Model Combiner Final Model Necessary and Sufficient Condition For the idea to work the classifiers should be Accurate Diverse Accurate Has an error rate better than random guessing on new instances Diverse They make different errors on new data points Why they Work Suppose there are 25 base classifiers Each classifier has an error rate Assume classifiers are independent Probability that the ensemble classifier makes a wrong prediction 25 25 i i i 13 1 25 i Marquis de Condorcet 1785 Majority vote is wrong with probability Value of Ensembles When combing multiple independent and diverse decisions each of which is at least more accurate than random guessing random errors cancel each other out correct decisions are reinforced. Human ensembles are demonstrably better How many jelly beans in the jar Individual estimates vs. group average. A Motivating Example Suppose that you are a patient with a set of symptoms Instead of taking opinion of just one doctor classifier you decide to take opinion of a few doctors Is this a good idea Indeed it is. Consult many doctors and then based on their diagnosis you can get a fairly accurate idea of the diagnosis. The Wisdom of Crowds The collective knowledge of a diverse and independent
