Description
I have been playing around with the quantile loss implementation in GBRT. I have had somewhat mixed results. In the plot below are the differences between the .05 and .95 quantiles and between the .2 and .8 quantiles. The jagged plots are with using the default algorithm. The smooth plots just replace the {,_}update_terminal_region{s,}() functions with the ones in the LeastSquaresError class. I'm getting better results with the new quantiles i.e. ~90% of my predictions fall within the .05-.95 quantiles and ~60% fall within the .2-.8 quantiles. With the default terminal updates, with my dataset I'm getting about 5-10% lower.
I'm wondering if it makes sense to make this an option, i.e. a boolean to just use the MSE estimates of the leaves in the tree to update the prediction values. In other words if the more sophisticated steps don't happen to work as well as the default one.
I realize this isn't probably a very strong argument yet but I wanted to see how others felt.
This may be related to #4210?