Learning to Enforce Monotone Constraints in Regression and Classification Trees
A better method to enforce monotonic constraints in regression and classification trees
In this report we present two new ways of enforcing monotone constraints in regression and classification trees.We also propose a heuristic that takes into account that greedily splitting a tree by choosing a monotone split with respect to its immediate gain is far from optimal.Then, we compare the results with the current implementation of the monotone constraints in the lightgbm library, using the well known adult public dataset.The best method we propose (a smarter way to split the tree coupled to a penalization of monotone splits)consistently beats the current implementation of lightgbm.With small or average trees, the loss reduction can be as high as 1% in the early stages of training and decreases to around 0.1% at the loss peak for the adult dataset.
