This dashboard changes software and hardware at once, to answer questions like "is cuML on this GPU faster than scikit-learn-intelex on my server?". A speed-upBaseline time divided by the compared time: 2x is twice as fast as the baseline, 0.5x twice as slow. In the detailed results, a noisy speed-up is shown as ~1.5x (5-15% spread over the repeats) or as a range like 1.2-1.9x (more than 15%). here mixes both effects. To separate them, see the software and hardware dashboards.
How to read
Pick a baselineThe reference that speed-ups are computed against.: one software on one machine. Then compare it with either one software on every machine (one color per machine), or one machine with every software (one color per software). The tabs below describe the baseline and each compared machine and software environmentOne Pixi environment of this repository: a fixed set of package versions and builds..
The plots read like in the software/implementations dashboard: each point is one benchmark case, and points above the dashed 1x line are faster than the baseline. Marker shapes mean the same.
Random forests and extra trees often have a different forest size on each machine: their times are rescaled to the same size, and their metrics aren't compared when the sizes differ. KMeans is capped at 64 threads on machines with more than 64 CPUs, to avoid an OpenBLAS crash. The last row compares the time per fit in hyper-parameter searches, where many fits run in parallel.
Findings
On the benchmarked cases:
- Software and hardware gains can compound. Random forests fit ~30x faster with scikit-learn-intelex on the Intel Xeon server than with scikit-learn on the Intel Core Ultra laptop, while the software alone gives ~2x to 3x and the server alone ~3x.
- cuML on the NVIDIA L4 fits LogisticRegression ~3.5x and RandomForestClassifier ~2.5x faster than scikit-learn on the laptop. RandomForestRegressor and Ridge are often slower.
- A big CPU server can beat that GPU: scikit-learn-intelex on the Intel Xeon server fits random forests ~20x to 35x faster than cuML on the L4, and ~1.7x to 2.5x faster on the Intel Core Ultra laptop, with 256 bins on both sides.
- Array APIStandard API shared by array libraries (NumPy, PyTorch, CuPy, dpnp). With it, some scikit-learn estimators run directly on these libraries' arrays, including on GPU. on a GPU is mostly a LogisticRegression story: PyTorchDeep learning library. scikit-learn can take its tensors as Array API inputs, on CPU or GPU. More on the L4 or the Arc B580 fits it ~10x to 20x faster than scikit-learn on the laptop, but Ridge is slower.