A Study of Rule Extraction and Association Rules for an Interpretable Random Forest

Abstract

Random forest is one of the analytical methods for classification and regression derived from a set of decision trees. It is the most popular ensemble method with many advantages, such as applying to various prediction problems, producing competitive accuracy, adjusting parameters, and handling small sample sizes in high-dimensional feature spaces. However, the random forest is often criticized as a "black box" model because random forests are difficult to interpret. In contrast, interpretation is much needed to understand the relationship between the predictor and target variables. This study discusses the interpretation of a random forest using an association rule technique using rules extracted from each decision tree in the random forest model. The association rule technique is an "if-then" statement, a pattern mining technique that aims to obtain frequently occurring item combinations in large data sets and can also be used to find association rules among item combinations. In this study, the itemset refers to the rules (order of branches) formed by each decision tree in the random forest model. Based on the simulation results obtained, the application of rule extraction techniques and association rules in the random forest model can determine thresholds for each factor that affects the target variable. It can also determine interacting factors that have an association relationship with the target variable so that is obtained a more interpretable random forest model. The results obtained in the simulation are then applied to answer empirical questions regarding poverty. Poverty is a complex problem in Indonesia and has always been an important indicator in measuring the success of a region's development. According to the Central Bureau of Statistics of Indonesia (BPS) report in 2022, West Java Province is the province that occupies the second position with the most significant number of poor people in Indonesia, with a poverty percentage of 8.06%. Tasikmalaya City has the largest percentage of poverty in West Java for 14 consecutive years from 2008 to 2021. According to the results of the empirical study, the factors that most influence the status of poor households in West Java Province are households with at least five members and using firewood as the primary fuel for cooking. In Tasikmalaya City, the factors influencing the status of poor households in Tasikmalaya City, the most dominating variables in describing the characteristics are the number of household members of more than four people, the primary fuel for cooking using firewood, and not having their land.