Study of Single and Ensemble Classifiers of Classification Tree and Support Vector Machine

Abstract

A classifier is such a rule that can be used to group an object into predetermined group or classs based on its attributes. There are two types of approach to develop a classifier rules are a parametric and a nonparametric. Parametric method requires certain assumptions to obtain the best classification but not all assumptions are met so that makes it difficult for researchers. The violation of the assumptions might lead to the lack of the effectiveness and the validity results. Recently, people pay more attention to non parametric classifiers such as Support Vector Machine (SVM) and Classification Tree (CT) to overcome the violation of the assumptions of parametric method. Some resent research figured out that an ensemble of classifiers could be an effective way to improve the classification accuracy and reduce the prediction variation of a single classifier (Valentini dan Dietterich 2000). The ensemble method is combining the class predictions resulted by a set of single classifiers into a single prediction by applying a majority vote rule. Among some popular techniques a method of bagging (bootstrap agregating) by Breiman (1996) is the simplest but powerful technique. The data used in this research are simulation data and real-life data. Simulation data are used to assess and compare the performance of single and ensemble classifiers of classification tree and SVM in three different data structures: (1) a situation where the members of different classes are perfectly linear separable, (2) a situation where the members of different classes are linerseparable but not perfect and (3) a situation where the members of different classes could not be separated by a linear function. Single and ensemble classifiers of classification trees and SVM will be applied to classify the successful study of postgraduate IPB students in Statistics department enrollment 2000-2010. Our research revealed that SVM resulted better classifier compared to Classification Tree. It is valid for all three data structure under consideration. Moreover, ensemble treatment to the classifier succeeded in improving the classification performance, especiality when radial kernel function is embedded in the procedure. Ensemble SVM in real-life data with a radial kernel function has the best performance compared to other methods and is the most appropriate method to classify the successful study of postgraduate IPB students in Statistics department enrollment 2000-2010.