Bazel does not use all cores on Windows Server Multi-Socket machine

[peakschris]

Description of the bug:

We have a Windows Server 2022 machine with 2 sockets, each containing an AMD EPYC 7443 24-Core Processor (each processor has 48 logical cpus). Task manager reports:

Base speed: 2.85Ghz
Sockets: 2
Cores: 48
Logical processors: 96
Virtualization: Enabled

We are seeing bazel builds on this machine use 48 actions, so it only sees half the available logical cpus.

It is possible to use --local_resources=cpu=96 --jobs=96 to force bazel to schedule 96 actions, but overall cpu usage remains around 50%.

The bazel code uses JVM's availableProcessors() to determine cpu count. Windows Server uses 'Processor Groups' concept, and each socket is a ProcessorGroup. JVM has for a while had a limitation that it is only able to access compute within a single ProcessorGroup. But this limitation has been lifted by Microsoft in Windows Server 2022 and Windows 11, and a JVM change (https://bugs.openjdk.org/browse/JDK-6942632) allows all sockets to be used by a single JVM.

This JVM change appears to have been backported to JDK21. Would it be possible to use such a JVM version in bazel so that it can work across multiple sockets?

Thanks!

Which category does this issue belong to?

Core

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Use windows machine with 2 sockets. Run bazel build.

Which operating system are you running Bazel on?

Windows Server 2022

What is the output of bazel info release?

8.0.0

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

No response

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

[fmeum]

The JDK change requires opt-in, could you test running with --host_jvm_args=-XX:+UseAllWindowsProcessorGroups?

[peakschris]

I get:

startup --host_jvm_args=-XX:+UseAllWindowsProcessorGroups

Unrecognized VM option 'UseAllWindowsProcessorGroups'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

Is bazel using the version of JDK that has this backport?

[fmeum]

As it turns out it's not: openjdk/jdk21u-dev@c8b8f72
21.0.6 isn't even out yet from Oracle or Azul. We'll have to wait for that.