How Robust Are Robust Regression Methods Really With Respect To Outliers and Influential Points?

Abstract

Since the first time Least Squares Regression was developed, many researchers use this method to predict and describe the relationship between explanatory variable and response variable. The basic concept of classical least squares regression is minimizing the sum of the squared residuals. This concept has weaknesses whenever there are outliers and/or influential points. So many researchers have produced more robust versions of this estimator. Huber M Regression and Least Median of Square methods are some robust methods. In order to know when we have to use robust methods, this paper attempts to study robustness of M Regression and Least Median squares (LMS) toward extreme outliers and influential points with regard to accuracy of the estimator utility data simulation. This research is focused on simple linear regression with four data cases. The cases are data sets with influential points, position of outliers, proportion of outliers, and combination of influential point and outlier. It seems that Huber M Regression call handle influential points. and outliers in small proportions while Least Median Square is better to be used against outliers in large proportions. Parameter bias, R square and, standard error were used to compare the methods.