← All dashboards

HistGradientBoosting thread-scalability breakdown

A threadUnits of execution that share the memory of their process. Each one can run on its own CPU core. scaling curve shows whether a fitTraining a model on data. More in the scikit-learn glossary got faster, but not why. This dashboard breaks HistGradientBoosting's fit time into its phases (binningMapping each feature to a small number of integer bins (a few hundred at most) before building trees., histogram computationFor each tree node, summing the gradients of the samples in each bin of each feature., split findingScanning the histograms to find the best feature and threshold to split each node.) to see which ones scale.

How to read

There is one tab per machine, buildOne packaging of the same scikit-learn version. Here, builds differ by their BLAS and OpenMP libraries. and thread affinityPinning threads to given CPU cores, so the operating system doesn't move them around. setting, and one stacked bar per workload. The x-axis is the requested thread count (OMP_NUM_THREADSEnvironment variable setting how many threads OpenMP code uses.). If a bar stops shrinking or grows as threads are added, the extra threads bring nothing or cost time, and the segments show which phase is responsible.

Findings

On the benchmarked cases, adding more threads can be highly counter-productive:

  • On the high-end server, the best thread count grows with the workload. The smallest workloads are fastest on 1 thread, medium ones stop scaling after 4 to 8 threads, and the largest after 16 to 64. Past the best thread count, fits get slower, sometimes dramatically: every workload is 2x to 30x slower at 128 or 172 threads than at its best thread count.
  • On the laptop, active waitIdle OpenMP threads spin on the CPU for a while before sleeping, so they start the next parallel loop faster, at the cost of CPU time. matters for small and medium workloads. Without active wait (the conda-forgeCommunity-maintained channel of conda packages. Its scikit-learn can be installed with different BLAS and OpenMP libraries. build), they get several times slower at 8 and 16 threads: for instance, covtype goes from 11s on 4 threads to 38s on 8. With it (PyPIThe Python Package Index, where pip installs packages from. Its scikit-learn wheels bundle their own OpenMP runtime, and NumPy and SciPy bundle OpenBLAS.), they stay roughly flat or keep improving up to 8 threads. The largest workloads scale up to 16 threads either way. For details about active wait, see scikit-learn#34764.
  • We're working on fixing this. With scikit-learn#34935, using all the threads is rarely much slower than the best thread count.

Where these results come from (1 source)
RecordedHardwareRecordsSourceEnvsConfig
2026-09-23 Intel Core Ultra laptop 75 CI run · PR #111 · f7aa3cb4 by github-actions[bot] sklearn-cf-default configs/hgb_scalability.py

Hardware

CPU

Intel(R) Core(TM) Ultra X7 358H

X86_64, 16 physical cores, 16 logical CPUs

31 GB RAM

$1,299, released 2026

GPU(s)

  • level_zero:gpu:0: Intel(R) Arc(TM) B390 GPU (29 GB, integrated, priced with the CPU)

sklearn

Python 3.12.14

Packages

  • scikit-learn 1.9.1 (conda-forge)
  • numpy 2.5.3 (conda-forge)
  • scipy 1.18.0 (conda-forge)
  • pandas 2.3.3 (conda-forge)

Threadpools

  • libopenblas 0.3.34 (pthreads, 16 threads)
  • libgomp (16 threads)

OpenMP

  • GNU libgomp (OpenMP 5.2)
  • GOMP_SPINCOUNT: 1

Full environment

view pixi env JSON

bin fitbin transformother / unmeasuredapply splitfind splitcompute hist
x-axis: requested threads (OMP_NUM_THREADS)

XS (1,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

S (1,000 x 100)

n_iter=100, max_leaf_nodes=31, max_features=0.5

ames_housing (1,168 x 79)

n_iter=200, max_leaf_nodes=15, 43 categorical, min_samples_leaf=10

S-thin (5,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

M (20,000 x 50)

n_iter=100, max_leaf_nodes=31, max_features=0.5

amazon_employee_access (26,215 x 9)

n_iter=150, max_leaf_nodes=31, 9 categorical, min_samples_leaf=20

kddcup09_churn (40,000 x 207)

n_iter=150, max_leaf_nodes=15, 34 categorical, min_samples_leaf=50

M-very-thin (50,000 x 5)

n_iter=100, max_leaf_nodes=31, max_features=1.0

L (100,000 x 100)

n_iter=100, max_leaf_nodes=31, max_features=0.5

L-leaf255 (100,000 x 100)

n_iter=20, max_leaf_nodes=255, max_features=0.5

year_prediction_msd (463,810 x 90)

n_iter=100, max_leaf_nodes=127, min_samples_leaf=5

covtype (464,809 x 12)

n_iter=100, max_leaf_nodes=127, 2 categorical, min_samples_leaf=5

L-thin (500,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

L-stumps (1,000,000 x 100)

n_iter=100, max_leaf_nodes=2, max_features=0.5

susy (4,500,000 x 18)

n_iter=70, max_leaf_nodes=127, min_samples_leaf=5

Where these results come from (1 source)
RecordedHardwareRecordsSourceEnvsConfig
2026-09-23 Intel Core Ultra laptop 75 CI run · PR #111 · f7aa3cb4 by github-actions[bot] sklearn-pypi configs/hgb_scalability.py

Hardware

CPU

Intel(R) Core(TM) Ultra X7 358H

X86_64, 16 physical cores, 16 logical CPUs

31 GB RAM

$1,299, released 2026

GPU(s)

  • level_zero:gpu:0: Intel(R) Arc(TM) B390 GPU (29 GB, integrated, priced with the CPU)

sklearn

Python 3.12.14

Packages

  • scikit-learn 1.9.1 (pypi)
  • numpy 2.5.3 (pypi)
  • scipy 1.18.1 (pypi)
  • pandas 2.3.3 (pypi)

Threadpools

  • libscipy_openblas 0.3.31.dev (pthreads, 16 threads)
  • libscipy_openblas 0.3.34.106.0 (pthreads, 16 threads)
  • libgomp (16 threads)

OpenMP

  • GNU libgomp (OpenMP 4.5)
  • GOMP_SPINCOUNT: 300000

Full environment

view pixi env JSON

bin fitbin transformother / unmeasuredapply splitfind splitcompute hist
x-axis: requested threads (OMP_NUM_THREADS)

XS (1,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

S (1,000 x 100)

n_iter=100, max_leaf_nodes=31, max_features=0.5

ames_housing (1,168 x 79)

n_iter=200, max_leaf_nodes=15, 43 categorical, min_samples_leaf=10

S-thin (5,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

M (20,000 x 50)

n_iter=100, max_leaf_nodes=31, max_features=0.5

amazon_employee_access (26,215 x 9)

n_iter=150, max_leaf_nodes=31, 9 categorical, min_samples_leaf=20

kddcup09_churn (40,000 x 207)

n_iter=150, max_leaf_nodes=15, 34 categorical, min_samples_leaf=50

M-very-thin (50,000 x 5)

n_iter=100, max_leaf_nodes=31, max_features=1.0

L (100,000 x 100)

n_iter=100, max_leaf_nodes=31, max_features=0.5

L-leaf255 (100,000 x 100)

n_iter=20, max_leaf_nodes=255, max_features=0.5

year_prediction_msd (463,810 x 90)

n_iter=100, max_leaf_nodes=127, min_samples_leaf=5

covtype (464,809 x 12)

n_iter=100, max_leaf_nodes=127, 2 categorical, min_samples_leaf=5

L-thin (500,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

L-stumps (1,000,000 x 100)

n_iter=100, max_leaf_nodes=2, max_features=0.5

susy (4,500,000 x 18)

n_iter=70, max_leaf_nodes=127, min_samples_leaf=5

Where these results come from (1 source)
RecordedHardwareRecordsSourceEnvsConfig
2026-11-03 Intel Core Ultra desktop 90 Manual run · ec8228b4 by Arthur Lacote sklearn-cf-default configs/hgb_scalability.py

Hardware

CPU

Intel(R) Core(TM) Ultra 9 285

X86_64, 24 physical cores, 24 logical CPUs

61 GB RAM

GPU(s)

  • level_zero:gpu:0: Intel(R) Arc(TM) B580 Graphics (12 GB)

sklearn

Python 3.14.8

Packages

  • scikit-learn 1.9.1 (conda-forge)
  • numpy 2.5.3 (conda-forge)
  • scipy 1.18.1 (conda-forge)
  • pandas 2.3.3 (conda-forge)

Threadpools

  • libopenblas 0.3.34 (pthreads, 24 threads)
  • libgomp (24 threads)

OpenMP

  • GNU libgomp (OpenMP 5.2)
  • GOMP_SPINCOUNT: 1

Full environment

view pixi env JSON

bin fitbin transformother / unmeasuredapply splitfind splitcompute hist
x-axis: requested threads (OMP_NUM_THREADS)

XS (1,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

S (1,000 x 100)

n_iter=100, max_leaf_nodes=31, max_features=0.5

ames_housing (1,168 x 79)

n_iter=200, max_leaf_nodes=15, 43 categorical, min_samples_leaf=10

S-thin (5,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

M (20,000 x 50)

n_iter=100, max_leaf_nodes=31, max_features=0.5

amazon_employee_access (26,215 x 9)

n_iter=150, max_leaf_nodes=31, 9 categorical, min_samples_leaf=20

kddcup09_churn (40,000 x 207)

n_iter=150, max_leaf_nodes=15, 34 categorical, min_samples_leaf=50

M-very-thin (50,000 x 5)

n_iter=100, max_leaf_nodes=31, max_features=1.0

L (100,000 x 100)

n_iter=100, max_leaf_nodes=31, max_features=0.5

L-leaf255 (100,000 x 100)

n_iter=20, max_leaf_nodes=255, max_features=0.5

year_prediction_msd (463,810 x 90)

n_iter=100, max_leaf_nodes=127, min_samples_leaf=5

covtype (464,809 x 12)

n_iter=100, max_leaf_nodes=127, 2 categorical, min_samples_leaf=5

L-thin (500,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

L-stumps (1,000,000 x 100)

n_iter=100, max_leaf_nodes=2, max_features=0.5

susy (4,500,000 x 18)

n_iter=70, max_leaf_nodes=127, min_samples_leaf=5

Where these results come from (1 source)
RecordedHardwareRecordsSourceEnvsConfig
2026-11-03 Intel Core Ultra desktop 90 Manual run · ec8228b4 by Arthur Lacote sklearn-pypi configs/hgb_scalability.py

Hardware

CPU

Intel(R) Core(TM) Ultra 9 285

X86_64, 24 physical cores, 24 logical CPUs

61 GB RAM

GPU(s)

  • level_zero:gpu:0: Intel(R) Arc(TM) B580 Graphics (12 GB)

sklearn

Python 3.14.8

Packages

  • scikit-learn 1.9.1 (pypi)
  • numpy 2.5.3 (pypi)
  • scipy 1.18.1 (pypi)
  • pandas 2.3.3 (pypi)

Threadpools

  • libscipy_openblas 0.3.31.dev (pthreads, 24 threads)
  • libscipy_openblas 0.3.34.106.0 (pthreads, 24 threads)
  • libgomp (24 threads)

OpenMP

  • GNU libgomp (OpenMP 4.5)
  • GOMP_SPINCOUNT: 300000

Full environment

view pixi env JSON

bin fitbin transformother / unmeasuredapply splitfind splitcompute hist
x-axis: requested threads (OMP_NUM_THREADS)

XS (1,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

S (1,000 x 100)

n_iter=100, max_leaf_nodes=31, max_features=0.5

ames_housing (1,168 x 79)

n_iter=200, max_leaf_nodes=15, 43 categorical, min_samples_leaf=10

S-thin (5,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

M (20,000 x 50)

n_iter=100, max_leaf_nodes=31, max_features=0.5

amazon_employee_access (26,215 x 9)

n_iter=150, max_leaf_nodes=31, 9 categorical, min_samples_leaf=20

kddcup09_churn (40,000 x 207)

n_iter=150, max_leaf_nodes=15, 34 categorical, min_samples_leaf=50

M-very-thin (50,000 x 5)

n_iter=100, max_leaf_nodes=31, max_features=1.0

L (100,000 x 100)

n_iter=100, max_leaf_nodes=31, max_features=0.5

L-leaf255 (100,000 x 100)

n_iter=20, max_leaf_nodes=255, max_features=0.5

year_prediction_msd (463,810 x 90)

n_iter=100, max_leaf_nodes=127, min_samples_leaf=5

covtype (464,809 x 12)

n_iter=100, max_leaf_nodes=127, 2 categorical, min_samples_leaf=5

L-thin (500,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

L-stumps (1,000,000 x 100)

n_iter=100, max_leaf_nodes=2, max_features=0.5

susy (4,500,000 x 18)

n_iter=70, max_leaf_nodes=127, min_samples_leaf=5

Where these results come from (1 source)
RecordedHardwareRecordsSourceEnvsConfig
2026-09-18 AMD Workstation 60 Manual run · PR #108 · b12d271e by jeremiedbb sklearn-cf-libomp-openblas-omp configs/hgb_scalability.py

Hardware

CPU

AMD Ryzen 7 3700X 8-Core Processor

X86_64, 8 physical cores, 16 logical CPUs

16 GB RAM

$1,100, released 2019

GPU(s)

  • cuda:0: NVIDIA GeForce RTX 2060 (6 GB, priced with the CPU)

sklearn

Python 3.12.14

Packages

  • scikit-learn 1.9.1 (conda-forge)
  • numpy 2.5.3 (conda-forge)
  • scipy 1.18.0 (conda-forge)
  • pandas 2.3.3 (conda-forge)

Threadpools

  • libomp (16 threads)
  • vcomp (16 threads)

OpenMP

  • LLVM OpenMP (spec 201611)
  • KMP_BLOCKTIME: 200ms

Full environment

view pixi env JSON

bin fitbin transformother / unmeasuredapply splitfind splitcompute hist
x-axis: requested threads (OMP_NUM_THREADS)

XS (1,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

S (1,000 x 100)

n_iter=100, max_leaf_nodes=31, max_features=0.5

ames_housing (1,168 x 79)

n_iter=200, max_leaf_nodes=15, 43 categorical, min_samples_leaf=10

S-thin (5,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

M (20,000 x 50)

n_iter=100, max_leaf_nodes=31, max_features=0.5

amazon_employee_access (26,215 x 9)

n_iter=150, max_leaf_nodes=31, 9 categorical, min_samples_leaf=20

kddcup09_churn (40,000 x 207)

n_iter=150, max_leaf_nodes=15, 34 categorical, min_samples_leaf=50

M-very-thin (50,000 x 5)

n_iter=100, max_leaf_nodes=31, max_features=1.0

L (100,000 x 100)

n_iter=100, max_leaf_nodes=31, max_features=0.5

L-leaf255 (100,000 x 100)

n_iter=20, max_leaf_nodes=255, max_features=0.5

year_prediction_msd (463,810 x 90)

n_iter=100, max_leaf_nodes=127, min_samples_leaf=5

covtype (464,809 x 12)

n_iter=100, max_leaf_nodes=127, 2 categorical, min_samples_leaf=5

L-thin (500,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

L-stumps (1,000,000 x 100)

n_iter=100, max_leaf_nodes=2, max_features=0.5

susy (4,500,000 x 18)

n_iter=70, max_leaf_nodes=127, min_samples_leaf=5

Where these results come from (1 source)
RecordedHardwareRecordsSourceEnvsConfig
2026-09-18 AMD Workstation 60 Manual run · PR #108 · b12d271e by jeremiedbb sklearn-pypi configs/hgb_scalability.py

Hardware

CPU

AMD Ryzen 7 3700X 8-Core Processor

X86_64, 8 physical cores, 16 logical CPUs

16 GB RAM

$1,100, released 2019

GPU(s)

  • cuda:0: NVIDIA GeForce RTX 2060 (6 GB, priced with the CPU)

sklearn

Python 3.12.14

Packages

  • scikit-learn 1.9.1 (pypi)
  • numpy 2.5.3 (pypi)
  • scipy 1.18.1 (pypi)
  • pandas 2.3.3 (pypi)

Threadpools

  • libscipy_openblas 0.3.31.dev (pthreads, 16 threads)
  • libscipy_openblas 0.3.34.106.0 (pthreads, 16 threads)
  • vcomp (16 threads)

Full environment

view pixi env JSON

bin fitbin transformother / unmeasuredapply splitfind splitcompute hist
x-axis: requested threads (OMP_NUM_THREADS)

XS (1,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

S (1,000 x 100)

n_iter=100, max_leaf_nodes=31, max_features=0.5

ames_housing (1,168 x 79)

n_iter=200, max_leaf_nodes=15, 43 categorical, min_samples_leaf=10

S-thin (5,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

M (20,000 x 50)

n_iter=100, max_leaf_nodes=31, max_features=0.5

amazon_employee_access (26,215 x 9)

n_iter=150, max_leaf_nodes=31, 9 categorical, min_samples_leaf=20

kddcup09_churn (40,000 x 207)

n_iter=150, max_leaf_nodes=15, 34 categorical, min_samples_leaf=50

M-very-thin (50,000 x 5)

n_iter=100, max_leaf_nodes=31, max_features=1.0

L (100,000 x 100)

n_iter=100, max_leaf_nodes=31, max_features=0.5

L-leaf255 (100,000 x 100)

n_iter=20, max_leaf_nodes=255, max_features=0.5

year_prediction_msd (463,810 x 90)

n_iter=100, max_leaf_nodes=127, min_samples_leaf=5

covtype (464,809 x 12)

n_iter=100, max_leaf_nodes=127, 2 categorical, min_samples_leaf=5

L-thin (500,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

L-stumps (1,000,000 x 100)

n_iter=100, max_leaf_nodes=2, max_features=0.5

susy (4,500,000 x 18)

n_iter=70, max_leaf_nodes=127, min_samples_leaf=5

Where these results come from (1 source)
RecordedHardwareRecordsSourceEnvsConfig
2026-09-23 High-end Intel Xeon server 135 CI run · PR #112 · afa49ecc by github-actions[bot] sklearn-cf-default configs/hgb_scalability.py

Hardware

CPU

Intel(R) Xeon(R) 6787P

X86_64, 172 physical cores, 344 logical CPUs

499 GB RAM

$11,648, released 2025

GPU(s)

No GPU detected.

sklearn

Python 3.12.14

Packages

  • scikit-learn 1.9.1 (conda-forge)
  • numpy 2.5.3 (conda-forge)
  • scipy 1.18.0 (conda-forge)
  • pandas 2.3.3 (conda-forge)

Threadpools

  • libopenblas 0.3.34 (pthreads, 128 threads)
  • libgomp (344 threads)

OpenMP

  • GNU libgomp (OpenMP 5.2)
  • GOMP_SPINCOUNT: 300000

Full environment

view pixi env JSON

bin fitbin transformother / unmeasuredapply splitfind splitcompute hist
x-axis: requested threads (OMP_NUM_THREADS)

XS (1,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

S (1,000 x 100)

n_iter=100, max_leaf_nodes=31, max_features=0.5

ames_housing (1,168 x 79)

n_iter=200, max_leaf_nodes=15, 43 categorical, min_samples_leaf=10

S-thin (5,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

M (20,000 x 50)

n_iter=100, max_leaf_nodes=31, max_features=0.5

amazon_employee_access (26,215 x 9)

n_iter=150, max_leaf_nodes=31, 9 categorical, min_samples_leaf=20

kddcup09_churn (40,000 x 207)

n_iter=150, max_leaf_nodes=15, 34 categorical, min_samples_leaf=50

M-very-thin (50,000 x 5)

n_iter=100, max_leaf_nodes=31, max_features=1.0

L-leaf255 (100,000 x 100)

n_iter=20, max_leaf_nodes=255, max_features=0.5

L (100,000 x 100)

n_iter=100, max_leaf_nodes=31, max_features=0.5

year_prediction_msd (463,810 x 90)

n_iter=100, max_leaf_nodes=127, min_samples_leaf=5

covtype (464,809 x 12)

n_iter=100, max_leaf_nodes=127, 2 categorical, min_samples_leaf=5

L-thin (500,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

L-stumps (1,000,000 x 100)

n_iter=100, max_leaf_nodes=2, max_features=0.5

susy (4,500,000 x 18)

n_iter=70, max_leaf_nodes=127, min_samples_leaf=5

Where these results come from (1 source)
RecordedHardwareRecordsSourceEnvsConfig
2026-09-23 High-end Intel Xeon server 135 CI run · PR #112 · afa49ecc by github-actions[bot] sklearn-pypi configs/hgb_scalability.py

Hardware

CPU

Intel(R) Xeon(R) 6787P

X86_64, 172 physical cores, 344 logical CPUs

499 GB RAM

$11,648, released 2025

GPU(s)

No GPU detected.

sklearn

Python 3.12.14

Packages

  • scikit-learn 1.9.1 (pypi)
  • numpy 2.5.3 (pypi)
  • scipy 1.18.1 (pypi)
  • pandas 2.3.3 (pypi)

Threadpools

  • libscipy_openblas 0.3.31.dev (pthreads, 64 threads)
  • libscipy_openblas 0.3.34.106.0 (pthreads, 64 threads)
  • libgomp (344 threads)

OpenMP

  • GNU libgomp (OpenMP 4.5)
  • GOMP_SPINCOUNT: 300000

Full environment

view pixi env JSON

bin fitbin transformother / unmeasuredapply splitfind splitcompute hist
x-axis: requested threads (OMP_NUM_THREADS)

XS (1,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

S (1,000 x 100)

n_iter=100, max_leaf_nodes=31, max_features=0.5

ames_housing (1,168 x 79)

n_iter=200, max_leaf_nodes=15, 43 categorical, min_samples_leaf=10

S-thin (5,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

M (20,000 x 50)

n_iter=100, max_leaf_nodes=31, max_features=0.5

amazon_employee_access (26,215 x 9)

n_iter=150, max_leaf_nodes=31, 9 categorical, min_samples_leaf=20

kddcup09_churn (40,000 x 207)

n_iter=150, max_leaf_nodes=15, 34 categorical, min_samples_leaf=50

M-very-thin (50,000 x 5)

n_iter=100, max_leaf_nodes=31, max_features=1.0

L-leaf255 (100,000 x 100)

n_iter=20, max_leaf_nodes=255, max_features=0.5

L (100,000 x 100)

n_iter=100, max_leaf_nodes=31, max_features=0.5

year_prediction_msd (463,810 x 90)

n_iter=100, max_leaf_nodes=127, min_samples_leaf=5

covtype (464,809 x 12)

n_iter=100, max_leaf_nodes=127, 2 categorical, min_samples_leaf=5

L-thin (500,000 x 10)

n_iter=100, max_leaf_nodes=31, max_features=1.0

L-stumps (1,000,000 x 100)

n_iter=100, max_leaf_nodes=2, max_features=0.5

susy (4,500,000 x 18)

n_iter=70, max_leaf_nodes=127, min_samples_leaf=5