Tree oblique for regression with weighted support vector machine

Carta, Andrea

First

;Frigau, Luca

Second

2025-01-01

---

Abstract

This work presents a new approach to learning oblique decision trees for regression tasks. Oblique decision trees are a type of supervised statistical learning technique in which a linear combination of a set of predictors is used to find the hyperplane that partitions the features’ space at each node. Our novel algorithm, called Tree Oblique for Regression with weighted Support vector machine (TORS), at each node, first applies a feature selection method based on the predictors’ correlation with the dependent variable, and then dichotomizes the continuous dependent variable and applies a weighted support vector machine classifier with linear kernel to discover the oblique hyperplane that minimizes the deviance. We evaluate the performance of TORS on a set of different types of simulated data, and we find out that TORS performs well in any type of dataset. Moreover, we assess its performance by comparing the prediction power in terms of root mean squared error with respect to that obtained by other oblique tree models and standard decision tree, using both simulated and real data. Based on empirical evidence, TORS outperforms the other oblique decision trees and has the additional advantage of being easier to interpret.