Ordinal Trees and Random Forests: Score-Free Recursive Partitioning and Improved Ensembles

Existing ordinal trees and random forests typically use scores that are assigned to the ordered categories, which implies that a higher scale level is used. Versions of ordinal trees are proposed that take the scale level seriously and avoid the assignment of artificial scores. The construction principle is based on an investigation of the binary models that are implicitly used in parametric ordinal regression. These building blocks can be fitted by trees and combined in a similar way as in parametric models. The obtained trees use the ordinal scale level only. Since binary trees and random forests are constituent elements of the proposed trees, one can exploit the wide range of binary trees that have already been developed. A further topic is the potentially poor performance of random forests, which seems to have been neglected in the literature. Ensembles that include parametric models are proposed to obtain prediction methods that tend to perform well in a wide range of settings. The performance of the methods is evaluated empirically by using several data sets.

Model-based random forests for ordinal regression

We study and compare several variants of random forests tailored to prognostic models for ordinal outcomes. Models of the conditional odds function are employed to understand the various random forest flavours. Existing random forest variants for ordinal outcomes, such as Ordinal Forests and Conditional Inference Forests, are evaluated in the presence of a non-proportional odds impact of prognostic variables. We propose two novel random forest variants in the model-based transformation forest family, only one of which explicitly assumes proportional odds. These two novel transformation forests differ in the specification of the split procedures for the underlying ordinal trees. One of these split criteria is able to detect changes in non-proportional odds situations and the other one focuses on finding proportional-odds signals. We empirically evaluate the performance of the existing and proposed methods using a simulation study and illustrate the practical aspects of the procedures by a re-analysis of the respiratory sub-item in functional rating scales of patients suffering from Amyotrophic Lateral Sclerosis (ALS).