← All dashboards

Builds comparison

This dashboard compares buildsOne packaging of the same scikit-learn version. Here, builds differ by their BLAS and OpenMP libraries. of plain scikit-learn. The code is the same, only the BLAS/OpenMP runtimeThe BLAS and OpenMP libraries loaded at run time. They do the multithreaded number crunching for scikit-learn. changes. The PyPI wheelThe Python Package Index, where pip installs packages from. Its scikit-learn wheels bundle their own OpenMP runtime, and NumPy and SciPy bundle OpenBLAS. is the baselineThe reference that speed-ups are computed against.. The conda-forgeCommunity-maintained channel of conda packages. Its scikit-learn can be installed with different BLAS and OpenMP libraries. build names encode two choices:

  • libgomp or libomp: the OpenMPStandard for running the loops of compiled code on several threads. scikit-learn's Cython code (HistGradientBoosting, KMeans, ...) uses it. runtime of scikit-learn's own loops (libgompGCC's OpenMP runtime, used by the PyPI scikit-learn wheels on Linux. or libompLLVM's OpenMP runtime, used by the PyPI scikit-learn wheels on macOS.).
  • openblas or openblas-omp: how OpenBLASOpen-source BLAS implementation. The NumPy and SciPy wheels on PyPI each ship their own copy of it. runs its threads, with pthreads (the conda-forge default) or with OpenMP. In libomp-openblas-omp, OpenBLAS and scikit-learn share the same libomp threads. mkl is MKLIntel's Math Kernel Library (oneMKL): a BLAS implementation tuned for Intel CPUs, available as a conda-forge build option., on libomp too.

sklearn-cf-default installs conda-forge's scikit-learn without pinning either, which gives the same libraries as sklearn-cf-libgomp-openblas.

The "Threadpools" and "OpenMP" lines of each environment tab show what was loaded at run time. sklearn-pypi lists two libscipy_openblas: the NumPy and SciPy wheels each ship their own copy of OpenBLAS, and both get loaded, often in different versions. NumPy's matrix products run on NumPy's copy, SciPy's linear algebra (used by some solvers) on SciPy's. On conda-forge, NumPy and SciPy share a single OpenBLAS.

How to read

Read each cell like in the software/implementations dashboard: fitTraining a model on data. More in the scikit-learn glossary or predictMaking predictions with a fitted model. More in the scikit-learn glossary speed-upBaseline time divided by the compared time: 2x is twice as fast as the baseline, 0.5x twice as slow. In the detailed results, a noisy speed-up is shown as ~1.5x (5-15% spread over the repeats) or as a range like 1.2-1.9x (more than 15%). (log scale) per estimator category, one line per build. Marker shapes mean the same: a square (■) flags a setup difference, an open diamond (◇) different metricsScores of the fitted model on held-out data (ROC AUC, R2 score). They check that the compared implementations learn equivalent models. More in the scikit-learn glossary.

Findings
  • MKLIntel's Math Kernel Library (oneMKL): a BLAS implementation tuned for Intel CPUs, available as a conda-forge build option. is a good pick for linear models: it speeds up BLASBasic Linear Algebra Subprograms: the library that runs the matrix and vector products of NumPy and SciPy. OpenBLAS and MKL are two implementations of it.-bound linear model fits, commonly by 1.3x to 1.5x.
  • On the laptop, every conda-forgeCommunity-maintained channel of conda packages. Its scikit-learn can be installed with different BLAS and OpenMP libraries. build fits LogisticRegression much faster than the PyPIThe Python Package Index, where pip installs packages from. Its scikit-learn wheels bundle their own OpenMP runtime, and NumPy and SciPy bundle OpenBLAS. one, up to ~10x on some datasets. This comes from the OpenBLASOpen-source BLAS implementation. The NumPy and SciPy wheels on PyPI each ship their own copy of it. shipped in scipy and is going to be fixed in scipy soon, see scipy#26193.
  • HistGradientBoosting varies more between builds. On laptops, this comes from differences in active waitIdle OpenMP threads spin on the CPU for a while before sleeping, so they start the next parallel loop faster, at the cost of CPU time. (how long idle OpenMP threadsStandard for running the loops of compiled code on several threads. scikit-learn's Cython code (HistGradientBoosting, KMeans, ...) uses it. spin before sleeping), which conda-forge disables by default. Without active wait, HGB fits on small and medium datasets can be much slower. On the high-end server, the LLVM/Intel OpenMPLLVM's OpenMP runtime (libomp) and Intel's (libiomp5), which share most of their code. runtimes seem faster than libgompGCC's OpenMP runtime, used by the PyPI scikit-learn wheels on Linux.. See scikit-learn#34764 for the analysis, and scikit-learn#34935 for a fix in progress, based on the insights from the HistGradientBoosting thread-scalability breakdown plots.
  • ExtraTrees fits are ~25% slower on conda-forge builds. scikit-learn#34876 fixes it and will land in the next release.