Two CLR-based apps cannot coexist on Apple platforms

[MichalStrehovsky]

The duplicate is the Apple crypto native library ending up in the same process twice — once dynamically (the running .NET host) and once statically baked into our AOT dylib.

1. Host already has it (dynamically): the normal dotnet host loads the shared framework, which includes libSystem.Security.Cryptography.Native.Apple.dylib. That dylib contains pal_swiftbindings.o, whose Swift interop registers ObjC classes (HashBox, X25519KeyBox) with the process-wide ObjC runtime at load time.

2. NativeAOT bakes a second copy in (statically): in Microsoft.NETCore.Native.Unix.targets (ILCompiler 11.0.0-preview.5.26261.101), line ~150:

   <NetCoreAppNativeLibrary Include="System.Security.Cryptography.Native.Apple"
                            Condition="'$(_IsApplePlatform)' == 'true'" />

   The only condition is "is this an Apple platform" — there is no '$(NativeLib)' != 'Shared' guard. So our NativeLib=Shared output (libdotnet-aot.dylib) statically links the .a version of the same crypto library, including the same pal_swiftbindings.o and its HashBox/X25519KeyBox classes.

3. ObjC class names are process-global → collision: our dylib is loaded into the host process that already registered those classes from the framework copy. The dynamic loader then prints to stderr:

   objc[...]: Class HashBox is implemented in both .../libSystem.Security.Cryptography.Native.Apple.dylib and .../libdotnet-aot.dylib. One of the two will be used. Which one is undefined.

   That stderr line is what trips the StdErr.Should().BeEmpty() assertions in our tests.

Root flaw: the NativeAOT targets assume the output is a self-contained app that needs crypto statically baked in. For a NativeLib=Shared library loaded into an already-running .NET process, that static copy is redundant and collides with the framework's dynamic copy. The proper upstream fix looks like guarding line ~150 (and the matching DirectPInvoke) with '$(NativeLib)' != 'Shared' so shared-library AOT outputs don't re-embed framework-provided native libs.

Our RemoveAppleCryptoFromNativeAotLink target is essentially doing that guard manually from the csproj after the fact. Does guarding these NetCoreAppNativeLibrary/DirectPInvoke items on NativeLib != Shared sound like the right runtime-side fix, or is there a preferred way to handle native-lib linking for shared AOT outputs that load into an existing runtime?

Originally posted by @NikolaMilosavljevic in dotnet/sdk#54384 (comment)

[dotnet-policy-service]

Tagging subscribers to this area: @agocke, @dotnet/ilc-contrib
See info in area-owners.md if you want to be subscribed.

[MichalStrehovsky]

Looks like it is possible to work around: realm/SwiftLint#4901

For dotnet specifically, we could use libSystem.*.Native.dylib/.so/.dll that are already there because we also have a matching CoreCLR VM. We'd need to research whether they have any global state that assumes only one consumer per process.

[dotnet-policy-service]

Tagging subscribers to this area: @bartonjs, @vcsjones, @dotnet/area-system-security
See info in area-owners.md if you want to be subscribed.

[MichalStrehovsky]

Looks like it is possible to work around: realm/SwiftLint#4901

For dotnet specifically, we could use libSystem.*.Native.dylib/.so/.dll that are already there because we also have a matching CoreCLR VM. We'd need to research whether they have any global state that assumes only one consumer per process.

There is a bunch of global state in most of the libSystem.*.Native.dylib/.so/.dll, so I don't think we'd want to try sharing these.

For the immediate issue on Apple platforms (objc[...]: Class HashBox is implemented in both .../libSystem.Security.Cryptography.Native.Apple.dylib and .../libdotnet-aot.dylib. One of the two will be used. Which one is undefined.), I wonder if we could name these classes differently when building the static library libSystem.Security.Cryptography.Native.Apple.a (used for native AOT and single file deployments) vs when building the shared library libSystem.Security.Cryptography.Native.Apple.dylib (used otherwise).

That way we'd at least allow CoreCLR-JIT and native AOT coexist in the same app. It would still not allow two native AOT libraries to coexist in the same app, but Apple being a limited platform is par for the course.

[vcsjones]

Can you help me understand the urgency of this? Is this "we need it fixed for the next preview", "this needs to be fixed for .NET 11" or "it would be nice if this scenario worked" ?

[MichalStrehovsky]

Partially nativeAOT'd and partially JITted dotnet CLI is a .NET 11 deliverable (https://github.com/dotnet/sdk/blob/main/src/Cli/dotnet-aot/DESIGN.md).

Components that can be moved to AOT are moving to AOT. Looks like the first component that needs some crypto somewhere is moving to AOT and it ran into this: dotnet/sdk#54384 (comment).

Cc @NikolaMilosavljevic @baronfel @JeremyKuhne on urgency. I think we need at least some workaround. The workaround in the linked PR likely has problems.

[baronfel]

I think this likely blocks AOT on macos entirely for us, because our telemetry relies on crypto due to needing to submit to AppInsights via https (if nothing else).