Support for cfunction + closures on Apple M1 chip

[JakobAsslaender]

Hi everyone,

it seems that the @batch macro in combination with closures does not on the arm64-apple-darwin21.2.0 platform (Apple silicon). Any chance this could be fixed?

Here is a MWE:

using Polyester

function test1()
    a = 5
    function test2()
        return a
    end
    @batch for _ = 1:100
        test2()
    end
    return nothing
end

test1()

which produces this error message:

ERROR: cfunction: closures are not supported on this platform
Stacktrace:

batch_closure at [~/.julia/packages/Polyester/4Q0NM/src/batch.jl]()

macro expansion at [~/.julia/packages/Polyester/4Q0NM/src/batch.jl]()

#_batch_no_reserve#5 at [~/.julia/packages/Polyester/4Q0NM/src/batch.jl]()

(::Polyester.var"#_batch_no_reserve##kw")(::NamedTuple{(:threadlocal,), Tuple{Val{false}}}, ::typeof(Polyester._batch_no_reserve), ::var"#9#11"{var"#test2#10"{Int64}}, ::Tuple{UInt8}, ::Tuple{UInt32}, ::Tuple{UInt8}, ::UInt64, ::UInt64, ::UInt64, ::Static.StaticInt{1}, ::Static.StaticInt{1}, ::Polyester.NoLoop, ::Polyester.CombineIndices) at [~/.julia/packages/Polyester/4Q0NM/src/batch.jl]()

#batch#7 at [~/.julia/packages/Polyester/4Q0NM/src/batch.jl]()

batch at [~/.julia/packages/Polyester/4Q0NM/src/batch.jl]()

macro expansion at [~/.julia/packages/Polyester/4Q0NM/src/closure.jl]()

test1() at [./Untitled-1]()

top-level scope at [Untitled-1]()

ERROR: cfunction: closures are not supported on this platform
Stacktrace:

batch_closure at [~/.julia/packages/Polyester/4Q0NM/src/batch.jl]()

macro expansion at [~/.julia/packages/Polyester/4Q0NM/src/batch.jl]()

#_batch_no_reserve#5 at [~/.julia/packages/Polyester/4Q0NM/src/batch.jl]()

(::Polyester.var"#_batch_no_reserve##kw")(::NamedTuple{(:threadlocal,), Tuple{Val{false}}}, ::typeof(Polyester._batch_no_reserve), ::var"#9#11"{var"#test2#10"{Int64}}, ::Tuple{UInt8}, ::Tuple{UInt32}, ::Tuple{UInt8}, ::UInt64, ::UInt64, ::UInt64, ::Static.StaticInt{1}, ::Static.StaticInt{1}, ::Polyester.NoLoop, ::Polyester.CombineIndices) at [~/.julia/packages/Polyester/4Q0NM/src/batch.jl]()

#batch#7 at [~/.julia/packages/Polyester/4Q0NM/src/batch.jl]()

batch at [~/.julia/packages/Polyester/4Q0NM/src/batch.jl]()

macro expansion at [~/.julia/packages/Polyester/4Q0NM/src/closure.jl]()

test1() at [./Untitled-1]()

top-level scope at [Untitled-1]()

running on this system/Julia version:

julia> versioninfo()
Julia Version 1.8.0-beta1
Commit 7b711ce699 (2022-02-23 15:09 UTC)
Platform Info:
  OS: macOS (arm64-apple-darwin21.2.0)
  CPU: 8 × Apple M1
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, apple-m1)
  Threads: 4 on 4 virtual cores
Environment:
  JULIA_EDITOR = code
  JULIA_NUM_THREADS = 4

Thanks a bunch for looking into this!

[kylebeggs]

I get this same error, but intermittent. If I re-run the function to compile it works sometimes. Like, literally do nothing else, just re-run 2 times in a row and it'll work the second time. Weird

[chriselrod]

Yeah, this isn't great. The M1 is popular and ARM is probably going to only get more popular in the future.

I think this should be less frequent in Polyester 0.6.8, as it should now be avoiding CFunctions when it succeeded in hoisting everything that would've been captured into the closure out and turning them into function arguments.
If you have examples of this error on 0.6.8+, a work around would perhaps be to strengthen this hoisting.

[lazarusA]

I'm getting also this error on the first run in Julia 1.11, and if I re-rerun the same line a second time, it seems to work :/. Does anyone has a workaround?

Edit: Oh, I see #131

[AbdAlazezAhmed]

Interesting. Running your MWE on a Snapdragon chip laptop doesn't error.

julia> versioninfo()
Julia Version 1.10.5
Commit 6f3fdf7b362 (2024-08-27 14:19 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (aarch64-linux-gnu)
  CPU: 12 × unknown
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, generic)
Threads: 12 default, 0 interactive, 6 GC (on 12 virtual cores)

So, here's the story how I arrived here: I was running this example from Thunderbolt.jl

and I get a similar error to yours

ERROR: cfunction: closures are not supported on this platform
Stacktrace:
  [1] macro expansion
    @ ~/.julia/packages/Polyester/eqrC9/src/batch.jl:36 [inlined]
  [2] batch_closure
    @ ~/.julia/packages/Polyester/eqrC9/src/batch.jl:29 [inlined]
  [3] macro expansion
    @ ~/.julia/packages/Polyester/eqrC9/src/batch.jl:186 [inlined]
  [4] _batch_no_reserve
    @ ~/.julia/packages/Polyester/eqrC9/src/batch.jl:168 [inlined]
  [5] batch
    @ ~/.julia/packages/Polyester/eqrC9/src/batch.jl:334 [inlined]
  [6] macro expansion
    @ ~/.julia/packages/Polyester/eqrC9/src/closure.jl:456 [inlined]
  [7] _update_pealinear_operator_on_subdomain!(beas::Thunderbolt.EAVector{…}, sdh::SubDofHandler{…}, element_cache::Thunderbolt.AnalyticalCoefficientElementCache{…}, time::Float64, chunksize::Int64)
    @ Thunderbolt ~/.julia/dev/Thunderbolt/src/discretization/operator.jl:542
  [8] macro expansion
    @ ~/.julia/dev/Thunderbolt/src/discretization/operator.jl:531 [inlined]
  [9] macro expansion
    @ ~/.julia/packages/TimerOutputs/NRdsv/src/TimerOutput.jl:253 [inlined]
 [10] _update_operator!(op::Thunderbolt.PEALinearOperator{…}, b::Vector{…}, time::Float64)
    @ Thunderbolt ~/.julia/dev/Thunderbolt/src/discretization/operator.jl:522
 [11] update_operator!
    @ ~/.julia/dev/Thunderbolt/src/discretization/operator.jl:510 [inlined]
 [12] macro expansion
    @ ~/.julia/dev/Thunderbolt/src/solver/time/euler.jl:160 [inlined]
 [13] macro expansion
    @ ~/.julia/packages/TimerOutputs/NRdsv/src/TimerOutput.jl:253 [inlined]
 [14] setup_solver_cache(f::TransientDiffusionFunction{…}, solver::BackwardEulerSolver{…}, t₀::Float64)
    @ Thunderbolt ~/.julia/dev/Thunderbolt/src/solver/time/euler.jl:157
 [15] #construct_inner_cache#283
    @ ~/.julia/dev/Thunderbolt/src/solver/time/time_integrator.jl:122 [inlined]
 [16] construct_inner_cache
    @ ~/.julia/dev/Thunderbolt/src/solver/time/time_integrator.jl:121 [inlined]
 [17] (::Thunderbolt.OS.var"#27#28"{Vector{…}, @Kwargs{…}, GenericSplitFunction{…}, LieTrotterGodunov{…}, Tuple{…}})(i::Int64)
    @ Thunderbolt.OS ~/.julia/dev/Thunderbolt/src/solver/operator_splitting/solver.jl:36
 [18] ntuple
    @ ./ntuple.jl:19 [inlined]
 [19] construct_inner_cache(f::GenericSplitFunction{…}, alg::LieTrotterGodunov{…}; uparent::Vector{…}, u0::Vector{…}, kwargs::@Kwargs{…})
    @ Thunderbolt.OS ~/.julia/dev/Thunderbolt/src/solver/operator_splitting/solver.jl:36
 [20] construct_inner_cache
    @ ~/.julia/dev/Thunderbolt/src/solver/operator_splitting/solver.jl:30 [inlined]
 [21] construct_inner_cache(f::GenericSplitFunction{…}, alg::Thunderbolt.ReactionTangentController{…}; u0::Vector{…}, kwargs::@Kwargs{…})
    @ Thunderbolt ~/.julia/dev/Thunderbolt/src/solver/time/rtc.jl:98
 [22] construct_inner_cache
    @ ~/.julia/dev/Thunderbolt/src/solver/time/rtc.jl:97 [inlined]
 [23] #init_cache#310
    @ ~/.julia/dev/Thunderbolt/src/solver/time/rtc.jl:93 [inlined]
 [24] init_cache
    @ ~/.julia/dev/Thunderbolt/src/solver/time/rtc.jl:88 [inlined]
 [25] __init(::OperatorSplittingProblem{…}, ::Thunderbolt.ReactionTangentController{…}; dt::Float64, tstops::Tuple{}, saveat::Nothing, save_everystep::Bool, callback::Nothing, advance_to_tstop::Bool, save_func::Thunderbolt.OS.var"#11#13", dtchangeable::Bool, kwargs::@Kwargs{…})
    @ Thunderbolt.OS ~/.julia/dev/Thunderbolt/src/solver/operator_splitting/integrator.jl:75
 [26] __init
    @ ~/.julia/dev/Thunderbolt/src/solver/operator_splitting/integrator.jl:46 [inlined]
 [27] init_call(_prob::OperatorSplittingProblem{…}, args::Thunderbolt.ReactionTangentController{…}; merge_callbacks::Bool, kwargshandle::Nothing, kwargs::@Kwargs{…})
    @ DiffEqBase ~/.julia/packages/DiffEqBase/HW4ge/src/solve.jl:543
 [28] init_call
    @ ~/.julia/packages/DiffEqBase/HW4ge/src/solve.jl:516 [inlined]
 [29] #init_up#43
    @ ~/.julia/packages/DiffEqBase/HW4ge/src/solve.jl:569 [inlined]
 [30] init_up
    @ ~/.julia/packages/DiffEqBase/HW4ge/src/solve.jl:564 [inlined]
 [31] #init#41
    @ ~/.julia/packages/DiffEqBase/HW4ge/src/solve.jl:557 [inlined]
 [32] top-level scope
    @ ~/.julia/dev/Thunderbolt/examples/conduction-velocity-benchmark.jl:83
Some type information was truncated. Use `show(err)` to see complete types.

However, if I run the example twice it does not error :D