[ci-scan] Test failure: Android X509 DynamicChainTests name-constraint status mismatch

[simonrozsival]

Note

This issue draft was prepared with GitHub Copilot assistance and should be reviewed before posting.

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1444966

Additional observed builds:

• https://dev.azure.com/dnceng-public/public/_build/results?buildId=1442805
• https://dev.azure.com/dnceng-public/public/_build/results?buildId=1442382

Build error leg or test failing: android-arm Release AllSubsets_CoreCLR - System.Security.Cryptography.Tests - System.Security.Cryptography.X509Certificates.Tests.DynamicChainTests.NameConstraintViolation_PermittedTree_Dns

Error Details

Several X509 name-constraint tests fail consistently on Android Helix hardware. The failures look like provider/status behavior mismatches: UPN cases fail chain.Build, while DNS cases report InvalidNameConstraints where the test expects PartialChain.

Observed repro details from a representative official runtime-extra-platforms job:

• Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1444966
• Queue/job: android-arm Release AllSubsets_CoreCLR
• Helix work item: https://helix.dot.net/api/jobs/b1f9d1bd-1d1b-438b-ab80-34b1f16ab24e/workitems/System.Security.Cryptography.Tests?api-version=2019-06-17
• Work item: System.Security.Cryptography.Tests
• Machine: DNCENGWIN-142
• XHarness: TESTS_FAILED, instrumentation exit 1
• Test summary: total 11739, passed 10748, failed 5, skipped 986

Observed failure summaries:

System.Security.Cryptography.X509Certificates.Tests.DynamicChainTests.NameConstraintViolation_PermittedTree_Upn
chain.Build

System.Security.Cryptography.X509Certificates.Tests.DynamicChainTests.NameConstraintViolation_PermittedTree_Dns
Assert.Equal() Failure: Values differ
Expected: PartialChain
Actual:   InvalidNameConstraints

System.Security.Cryptography.X509Certificates.Tests.DynamicChainTests.NameConstraintViolation_ExcludedTree_Upn
chain.Build

System.Security.Cryptography.X509Certificates.Tests.DynamicChainTests.NameConstraintViolation_ExcludedTree_Dns
Assert.Equal() Failure: Values differ
Expected: PartialChain
Actual:   InvalidNameConstraints

Other official runtime-extra-platforms hits found in Helix testResults.xml:

• Build 1444966 / 20260601.4, android-arm Release AllSubsets_Mono, Helix: https://helix.dot.net/api/jobs/59f4de60-e3e0-4d92-bba4-4ffa93ab9445/workitems/System.Security.Cryptography.Tests?api-version=2019-06-17
• Build 1444966 / 20260601.4, android-arm Release AllSubsets_CoreCLR, Helix: https://helix.dot.net/api/jobs/b1f9d1bd-1d1b-438b-ab80-34b1f16ab24e/workitems/System.Security.Cryptography.Tests?api-version=2019-06-17
• Build 1442805 / 20260531.2, android-arm Release AllSubsets_CoreCLR, Helix: https://helix.dot.net/api/jobs/6d2e01a6-4741-4cf6-857d-92a859bf8602/workitems/System.Security.Cryptography.Tests?api-version=2019-06-17
• Build 1442805 / 20260531.2, android-arm64 Release AllSubsets_CoreCLR, Helix: https://helix.dot.net/api/jobs/2957a5ca-18ce-4df1-9c23-2f332f8de6e7/workitems/System.Security.Cryptography.Tests?api-version=2019-06-17
• Build 1442382 / 20260530.1, android-arm Release AllSubsets_CoreCLR, Helix: https://helix.dot.net/api/jobs/a4492657-e917-4b7d-ae62-633f0eca9b9a/workitems/System.Security.Cryptography.Tests?api-version=2019-06-17
• Build 1442382 / 20260530.1, android-arm64 Release AllSubsets_CoreCLR, Helix: https://helix.dot.net/api/jobs/5cc7dc11-993e-4b51-a85a-24a5c7143ae6/workitems/System.Security.Cryptography.Tests?api-version=2019-06-17

Error Message

{
  "ErrorMessage": "",
  "ErrorPattern": [
    "\\[FAIL\\][^\\n]*System\\.Security\\.Cryptography\\.X509Certificates\\.Tests\\.DynamicChainTests\\.NameConstraintViolation_(PermittedTree|ExcludedTree)_(Dns|Upn)",
    "Exception messages: (chain\\.Build|Assert\\.Equal\\(\\) Failure: Values differ)"
  ],
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

The ordered ErrorPattern array intentionally matches the DOTNET/logcat summary for all four related failures:

• NameConstraintViolation_PermittedTree_Upn with chain.Build
• NameConstraintViolation_PermittedTree_Dns with InvalidNameConstraints
• NameConstraintViolation_ExcludedTree_Upn with chain.Build
• NameConstraintViolation_ExcludedTree_Dns with InvalidNameConstraints

Duplicate search

No open issue was found that directly tracks this Android-specific X509 name-constraint mismatch.

Related but not duplicates:

• [Linux] X509 chains with user principal name (UPN) name constraints fail to build #87413 - Linux UPN name constraints fail to build
  • Closed.
  • Related name-constraints area, but Linux/OpenSSL-specific and not this Android status mismatch.
• macOS + Linux: Name constraint violation crashes chain building #41748 - macOS + Linux: Name constraint violation crashes chain building
  • Closed.
  • Related historical name-constraint behavior, not Android-specific.
• System.Security.Cryptography.X509Certificates.Tests fail on iOS/tvOS #36897 - System.Security.Cryptography.X509Certificates.Tests fail on iOS/tvOS
  • Closed.
  • Search hit for the same test names, but iOS/tvOS-specific and not Android.
• [release/7.0] Networking and Security tests fail on spanish servers: There are no more endpoints available in the endpoint mapper #83226 - release/7.0 Networking and Security tests fail on spanish servers
  • Closed KBE.
  • Search hit for a test name, but unrelated environment/localization issue.

AzDO / Helix history search

Recent public runtime-extra-platforms builds (definition 154) on main were searched for this signature. The important detail is that the xUnit failure text is not present in the AzDO job console logs; it is present in the Helix testResults.xml artifacts linked from those jobs.

• Query window: recent builds from 1444966 (20260601.4) back through 1406427 (20260502.2).
• Scope: failed Android job console logs scanned, then System.Security.Cryptography.Tests Helix work items inspected via testResults.xml where available.
• Search strings: NameConstraintViolation_PermittedTree_Upn, NameConstraintViolation_PermittedTree_Dns, NameConstraintViolation_ExcludedTree_Upn, NameConstraintViolation_ExcludedTree_Dns, InvalidNameConstraints, and PartialChain.
• Result: 6 official runtime-extra-platforms Helix work items contained all four failures.

Architecture coverage:

• android-arm: 4 hits (AllSubsets_CoreCLR and one AllSubsets_Mono)
• android-arm64: 2 hits (AllSubsets_CoreCLR)
• android-x86: 0 hits in the inspected window
• android-x64: 0 hits in the inspected window

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=1444966
Error message validated: [\[FAIL\][^\n]*System\.Security\.Cryptography\.X509Certificates\.Tests\.DynamicChainTests\.NameConstraintViolation_(PermittedTree|ExcludedTree)_(Dns|Upn) Exception messages: (chain\.Build|Assert\.Equal\(\) Failure: Values differ)]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/2/2026 10:42:33 AM UTC

Report

Build	Repository	Test	Pull Request
1444966	dotnet/runtime	System.Security.Cryptography.Tests.WorkItemExecution

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
1	1	1

[dotnet-policy-service]

Tagging subscribers to this area: @bartonjs, @vcsjones, @dotnet/area-system-security
See info in area-owners.md if you want to be subscribed.

[bartonjs]

Is it a versioning problem? Either way, the fix is probably in

runtime/src/libraries/System.Security.Cryptography/tests/X509Certificates/DynamicChainTests.cs

Lines 1151 to 1176 in 291547b

	private static X509ChainStatusFlags PlatformNameConstraints(X509ChainStatusFlags flags)
	{
	if (PlatformDetection.UsesAppleCrypto)
	{
	const X509ChainStatusFlags AnyNameConstraintFlags =
	X509ChainStatusFlags.HasExcludedNameConstraint |
	X509ChainStatusFlags.HasNotDefinedNameConstraint |
	X509ChainStatusFlags.HasNotPermittedNameConstraint |
	X509ChainStatusFlags.HasNotSupportedNameConstraint |
	X509ChainStatusFlags.InvalidNameConstraints;
	if ((flags & AnyNameConstraintFlags) != 0)
	{
	flags &= ~AnyNameConstraintFlags;
	flags |= X509ChainStatusFlags.InvalidNameConstraints;
	}
	}
	else if (OperatingSystem.IsAndroid())
	{
	// Android always validates name constraints as part of building a path
	// so violations comes back as PartialChain with no elements.
	flags = X509ChainStatusFlags.PartialChain;
	}
	return flags;
	}

[dotnet-policy-service]

Tagging subscribers to 'arch-android': @vitek-karas, @simonrozsival, @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.