Add threading support for Apple Accelerate #169

imciner2 · 2025-10-18T23:37:50Z

These APIs are new in macOS 15, but only allow saying if you want single threaded or multi-threaded operation, not specifying the actual number of threads to use in a multi-threaded setup.

There is a symbol called APPLE_NTHREADS in the accelerate library, which might be able to say how many threads it will use in multi-threaded mode? However, I can't find any information about that anywhere online (just it listed in various symbol tables for the library). If someone with a macOS 15+ machine can examine that symbol to see what it does/looks like, we could update the TODO in this PR potentially.

This is the LBT implementation of JuliaLinearAlgebra/AppleAccelerate.jl#88. Fixes #115.

cc @ViralBShah @giordano

These APIs are new in macOS 15, but only allow saying if you want single threaded or multi-threaded operation, not specifying the actual number of threads to use in a multi-threaded setup.

ViralBShah · 2025-10-20T12:52:36Z

How would I examine the value of such a variable?

imciner2 · 2025-10-20T13:12:03Z

I think it might be a function. I was just talking with @giordano and he tried this on a machine he has with 16 cores and got this:

julia> @ccall AppleAccelerate.libacc.APPLE_NTHREADS()::Int
16

If you can give that a try and see what it says, that would be nice.

ViralBShah · 2025-10-20T13:13:54Z

I am on an M2 Max, and this is what I get:

julia> @ccall AppleAccelerate.libacc.APPLE_NTHREADS()::Int
12

julia> Sys.CPU_THREADS
8

Do we know when it was introduced?

giordano · 2025-10-20T13:17:43Z

For the record, I tested on M4 Max

imciner2 · 2025-10-20T13:20:03Z

Do we know when it was introduced?

Nope, I haven't been able to find anything about it other than a bunch of tbd symbol lists on GitHub, and one person who wrote a stub function for it. It was basically a shot in the dark that we could even call it properly.

In LBT we have to lookup the symbol in the symbol table anyway, so we should be safe to use it no matter the macOS version (basically just call it if we find it, otherwise report a default value).

giordano · 2025-10-20T13:21:47Z

Uhm, we should be able to find out by bisecting the SDKs hosted on GitHub.

giordano · 2025-10-20T13:29:28Z

I am on an M2 Max, and this is what I get:
julia> @ccall AppleAccelerate.libacc.APPLE_NTHREADS()::Int
12

julia> Sys.CPU_THREADS
8

I'd need to double check, but I seem to remember our Sys.CPU_THREADS shows only the performance cores. According to Wikipedia, the M2 Max has 8 performance cores + 4 efficiency ones, for a total of 12, which is what APPLE_NTHREADS seems to return. Similarly, the M4 Max has 12 performance cores + 4 efficiency ones, and I get

julia> Sys.CPU_THREADS
12

julia> @ccall "/System/Library/Frameworks/Accelerate.framework/Accelerate".APPLE_NTHREADS()::Int
16

imciner2 · 2025-10-20T13:30:52Z

I am on an M2 Max, and this is what I get:
julia> @ccall AppleAccelerate.libacc.APPLE_NTHREADS()::Int
12

julia> Sys.CPU_THREADS
8
Do we know when it was introduced?

According to Wikipedia, the M2 Max has 8 performance cores and 4 efficiency cores (although I think ARM calls them big/little). So the Sys.CPU_THREADS seems to say the number of big cores, but APPLE_NTHREADS seems to say the total number of cores (both big and little).

ViralBShah · 2025-10-20T13:35:01Z

htop shows 12 cores.

giordano · 2025-10-20T14:00:35Z

Uhm, we should be able to find out by bisecting the SDKs hosted on GitHub.

% git clone --depth=1 https://github.com/nickfnblum/MacOSX-SDKs
% grep -r '_APPLE_NTHREADS' --include='*.tbd' MacOSX-SDKs 
MacOSX-SDKs/MacOSX11.3.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/vecLib.tbd:                       _APPLE_NTHREADS, _ATLU_DestroyThreadMemory, _BLASStateRelease, 
MacOSX-SDKs/MacOSX11.3.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.tbd:                       _APPLE_NTHREADS, _ATLU_DestroyThreadMemory, _BLASStateRelease, 
MacOSX-SDKs/MacOSX11.3.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate.tbd:                       _APPLE_NTHREADS, _ATLU_DestroyThreadMemory, _BLASStateRelease, 
MacOSX-SDKs/MacOSX11.1.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/vecLib.tbd:                       _APPLE_NTHREADS, _ATLU_DestroyThreadMemory, _BLASStateRelease, 
MacOSX-SDKs/MacOSX11.1.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.tbd:                       _APPLE_NTHREADS, _ATLU_DestroyThreadMemory, _BLASStateRelease, 
MacOSX-SDKs/MacOSX11.1.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate.tbd:                       _APPLE_NTHREADS, _ATLU_DestroyThreadMemory, _BLASStateRelease, 
MacOSX-SDKs/MacOSX11.0.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/vecLib.tbd:                       _APPLE_NTHREADS, _ATLU_DestroyThreadMemory, _BLASStateRelease, 
MacOSX-SDKs/MacOSX11.0.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.tbd:                       _APPLE_NTHREADS, _ATLU_DestroyThreadMemory, _BLASStateRelease, 
MacOSX-SDKs/MacOSX11.0.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate.tbd:                       _APPLE_NTHREADS, _ATLU_DestroyThreadMemory, _BLASStateRelease, 
MacOSX-SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/vecLib.tbd:                       _APPLE_NTHREADS, _ATLU_DestroyThreadMemory, _CAXPY, _CAXPY_, 
MacOSX-SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.tbd:                       _APPLE_NTHREADS, _ATLU_DestroyThreadMemory, _CAXPY, _CAXPY_, 
MacOSX-SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate.tbd:                       _APPLE_NTHREADS, _ATLU_DestroyThreadMemory, _CAXPY, _CAXPY_, 
MacOSX-SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.tbd:                       _APL_sgemm_LU, _APL_sgemm_QR, _APL_strsm, _APPLE_NTHREADS, 
MacOSX-SDKs/MacOSX10.13.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.tbd:                       _APL_sgemm_LU, _APL_sgemm_QR, _APL_strsm, _APPLE_NTHREADS, 
MacOSX-SDKs/MacOSX10.12.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.tbd:                       _APL_sgemm_LU, _APL_sgemm_QR, _APL_strsm, _APPLE_NTHREADS, 
MacOSX-SDKs/MacOSX10.11.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.tbd:                       _APPLE_NTHREADS, _ATLU_DestroyThreadMemory, _CAXPY, 
MacOSX-SDKs/MacOSX10.10.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.tbd:                       _APPLE_NTHREADS, _ATLU_DestroyThreadMemory, _CAXPY,

I'd say this was introduced in macOS 10.10.

ViralBShah · 2025-10-20T15:25:32Z

Mirrored APPLE_NTHREADS in AppleAccelerate: JuliaLinearAlgebra/AppleAccelerate.jl#89

ViralBShah · 2025-10-20T19:26:18Z

src/threading.c

+            if(nthreads == ACCELERATE_BLAS_THREADING_MULTI_THREADED) {
+                // This number is arbitrary right now, but greater than 1 to mean multi-threaded.
+                // TODO: Can we guestimate the number of threads from the APPLE_NTHREADS symbol in accelerate?
+                max_threads = 2;


I went with APPLE_NTHREADS in AppleAccelerate.jl

imciner2 · 2025-10-21T02:11:40Z

Ok, added some tests for Apple Accelerate now (both loading the symbols and calling this threading API). This should be good for review and merge.

After merge, I want to hold off tagging until PRs #168 and #167 are merged, then we can do one release with all three items. (I think I am close to finishing those).

ViralBShah · 2025-10-21T02:20:16Z

What happens on macos intel, where both MKL (installed) and Accelerate are available?

imciner2 · 2025-10-22T00:26:45Z

What happens on macos intel, where both MKL (installed) and Accelerate are available?

This is dependent on if someone loads either the AppleAccelerate.jl or MKL.jl package. By default, the forwards won't be available for either of them unless they are loaded, so just loading MKL.jl would only give MKL.

Loading MKL.jl would clear all forwards, but if you load AppleAccelerate.jl later it would replace the MKL forwards it could, but leave the rest. The threading API would still call both the MKL threading functions and the Accelerate threading functions though, because we only go by what library is loaded for those (so if a library is loaded, it is searched for the threading symbol when we actually want to use it - we don't keep a symbol map of those).

imciner2 added the enhancement New feature or request label Oct 18, 2025

Add threading support for Apple Accelerate

f94f2ab

These APIs are new in macOS 15, but only allow saying if you want single threaded or multi-threaded operation, not specifying the actual number of threads to use in a multi-threaded setup.

imciner2 force-pushed the im/accelerate branch from 27f9d5b to f94f2ab Compare October 20, 2025 09:43

ViralBShah reviewed Oct 20, 2025

View reviewed changes

Get the number of threads from accelerate

ec3ec3a

imciner2 force-pushed the im/accelerate branch 2 times, most recently from 6e84867 to 6fee1ab Compare October 21, 2025 01:57

Test Apple Accelerate loading and threading

006da93

imciner2 force-pushed the im/accelerate branch from 6fee1ab to 006da93 Compare October 21, 2025 02:02

Add threading support for Apple Accelerate #169

Are you sure you want to change the base?

Add threading support for Apple Accelerate #169

Uh oh!

Conversation

imciner2 commented Oct 18, 2025

Uh oh!

ViralBShah commented Oct 20, 2025

Uh oh!

imciner2 commented Oct 20, 2025

Uh oh!

ViralBShah commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

giordano commented Oct 20, 2025

Uh oh!

imciner2 commented Oct 20, 2025

Uh oh!

giordano commented Oct 20, 2025

Uh oh!

giordano commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

imciner2 commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ViralBShah commented Oct 20, 2025

Uh oh!

giordano commented Oct 20, 2025

Uh oh!

ViralBShah commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ViralBShah Oct 20, 2025

Choose a reason for hiding this comment

Uh oh!

imciner2 commented Oct 21, 2025

Uh oh!

ViralBShah commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

imciner2 commented Oct 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ViralBShah commented Oct 20, 2025 •

edited

Loading

giordano commented Oct 20, 2025 •

edited

Loading

imciner2 commented Oct 20, 2025 •

edited

Loading

ViralBShah commented Oct 20, 2025 •

edited

Loading

ViralBShah commented Oct 21, 2025 •

edited

Loading