HistGradientBoostingClassifier slow in prediction mode #16429
Comments
|
Indeed the prediction latency was optimized for Just to clarify, do you predict on a batch of 1000 samples in a single numpy array or do you call 1000 times the predict method with 1 sample at a time? |
|
We need to do a profiling session with |
|
ping @jeremiedbb |
|
Thanks for the report @SebastianBr . Do you observe a similar difference when predicting for 1 million samples instead? The prediction of the HistGB is multi-threaded, but the regular GB isn't. With 1000 samples, the thread spawning might be too much of an overhead. If you have time, I'm interested to see what it does if you set @ogrisel , just a side note, we parallelize the _raw_predict (i.e. decision_function), but not the
Curious, what are you referring to? |
I tested it again on larger datasets. Starting from 100k GB and HGB are indeed about equally fast. It seems there is some overhead that is probably not relevant in most cases. But since I often have smaller batches of 10 to 1000 examples and need in predictions in real-time HGB isn't in good choice for me currently. Do you see any chance for an improvement? |
I agree.
I remember that @pprett spent a lot of time profiling the predict method of |
|
Oh I see I haven't tried OMP_NUM_THREADS. It isn't a parameter, so where can I set it? |
|
|
It's an environment variable, you can do
Yes. The first obvious thing is to check whether this is indeed a thread overhead issue, and if so, only parallelize the code if the number of samples is high enough. The second one is to pack the tree structure so that it optimizes cache hits and stuff. |
|
Thanks for the info. I benchmarked the prediction with So it seems the overhead lies in the method itself. |
|
interesting, thanks I'll run a few benchmarks on my side. From what I remember, we're consistently faster than LightGBM and XGBoost on prediction (but I haven't tested against the regular GB estimators) |
|
there is a recent article (april 2023) comparing catboost / lightbm / xgboost / sklearn Gradient Boosting / Sklearn Hierarchical GB
|
|
and a study published on arxiv in may 2023 https://arxiv.org/pdf/2305.17094.pdf apparently xgboost can use GPU, but it doesn't mean it's better than sklearn. |
|
Let's keep this particular issue focused on the topic of the prediction speed and open different issues to track various aspects of training speed instead. Those have different performance bottlenecks that can be investigated and fixed quite independently of one another. For instance, for training there are already the following open issues: |
|
To start working on improving the prediction throughput and or latency of HGBDT we first need to collect detailed benchmarks and profiling results. |
While HistGradientBoostingClassifier is 100 faster than GradientBoostingClassifier when fitting the model, I found it to be very slow in case of predicting the class probabilities, in my case about 100 times slower :-(
For example:
GradientBoostingClassifier: 3.2 min for training for 1 million examples. 32 ms for 1000 predictions.
HistGradientBoostingClassifier: 7s for training. 1s for 1000 predictions
The text was updated successfully, but these errors were encountered: