Description
Describe the workflow you want to enable
Currently the only OOB-related performance metric we store on GradientBoostingClassifier is oob_improvement_, which is an array of OOB loss decreases per iteration. However, it would also be useful to track the actual OOB loss values for each iteration. This can be used as an estimate of the generalization error, and might bypass the need for cross validation in some cases. This would also help it integrate into my #23391 framework.
Describe your proposed solution
I propose we add a new attribute: oob_score_ (or alternatively oob_loss_) to GradientBoostingClassifier. This would only be set in cases where subsample < 1. It would be updated in each iteration. Currently we are already calculating this, we just throw it away:
scikit-learn/sklearn/ensemble/_gb.py
Lines 758 to 768 in 32f9dea
Describe alternatives you've considered, if relevant
You might think that the cumsum of the oob_improvement_ would give us the loss values, and this is almost true, except for the fact that we need the OOB loss for the first iteration, which isn't stored anywhere. So this doesn't solve the issue.
Additional context
No response