diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 2d468c368ad..4243333c417 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -206,13 +206,20 @@ When diagnosing runtime, build, or test failures, follow these practices. They e - **Reproduce CI failures locally — do not iterate through CI.** A clean local test cycle is minutes; a CI iteration is hours. Run device tests the same way CI does: ```bash make prepare && make all CONFIGURATION=Release - ./dotnet-local.sh build tests/Mono.Android-Tests/Mono.Android-Tests/Mono.Android.NET-Tests.csproj \ - -t:RunTestApp -c Release \ + ./dotnet-local.sh build -t:Install -c Release \ + tests/Mono.Android-Tests/Mono.Android-Tests/Mono.Android.NET-Tests.csproj \ -p:_AndroidTypeMapImplementation= \ -p:UseMonoRuntime= + ( + cd tests/Mono.Android-Tests/Mono.Android-Tests + ../../../dotnet-local.sh test Mono.Android.NET-Tests.csproj --no-build -c Release \ + --report-trx --results-directory ../../../bin/TestRelease/TestResults \ + -p:_AndroidTypeMapImplementation= \ + -p:UseMonoRuntime= + ) ``` On Windows, use `build.cmd` and `dotnet-local.cmd` instead of `make`/`dotnet-local.sh`. - Results land in `TestResult-Mono.Android.NET_Tests-*.xml` at the repo root. + Results land in `.trx` files under `bin/TestRelease/TestResults`. - **When the build gets into a weird state, delete `bin/` and `obj/` and rebuild from scratch.** Stale incremental output causes phantom errors. See **Troubleshooting → Build** below. diff --git a/.github/skills/tests/SKILL.md b/.github/skills/tests/SKILL.md index 957a5255655..597d66eae1c 100644 --- a/.github/skills/tests/SKILL.md +++ b/.github/skills/tests/SKILL.md @@ -2,9 +2,10 @@ name: tests description: >- Runs, discovers, or recommends dotnet/android tests. Handles standalone tests - (plain dotnet test) and full-build tests (local SDK required). Maps source - files to relevant test suites. Trigger on test execution requests, questions - about available tests, or "which tests should I run for this change?" + (plain dotnet test), full-build tests (local SDK required), and on-device + MTP/NUnit tests. Maps source files to relevant test suites. Trigger on test + execution requests, questions about available tests, or "which tests should I + run for this change?" --- # Tests @@ -21,7 +22,7 @@ Check for the local SDK: ls bin/Debug/dotnet/dotnet 2>/dev/null || ls bin/Release/dotnet/dotnet 2>/dev/null ``` -**If the local SDK is missing and full-build tests are requested, use `ask_user` before building.** Never silently skip tests. Present the choice: build with `make prepare && make` (slow) or skip full-build tests. +**If the local SDK is missing and full-build tests are requested, use `ask_user` before building.** Never silently skip tests. Present the choice: build with `make prepare && make all` (slow) or skip full-build tests. ## Workflow @@ -71,13 +72,17 @@ dotnet test .csproj -v minimal --filter "Name~TestName" ./dotnet-local.sh test bin/TestDebug/MSBuildDeviceIntegration/${TFM}/MSBuildDeviceIntegration.dll --filter "Name~InstallAndRunTests" ``` -### On-device runtime tests (NUnitLite, full-build + device) +### On-device runtime tests (stock NUnit/MTP, full-build + device) -These do NOT use `dotnet test`. Use the `RunTestApp` MSBuild target: +Build and install the APK first, then run `dotnet test` from the test project directory so the project-local `global.json` enables Microsoft Testing Platform: ```bash -./dotnet-local.sh build -t:RunTestApp tests/Mono.Android-Tests/Mono.Android-Tests/Mono.Android.NET-Tests.csproj +./dotnet-local.sh build -t:Install -c Debug tests/Mono.Android-Tests/Mono.Android-Tests/Mono.Android.NET-Tests.csproj +( + cd tests/Mono.Android-Tests/Mono.Android-Tests + ../../../dotnet-local.sh test Mono.Android.NET-Tests.csproj --no-build -c Debug --report-trx --results-directory ../../../bin/TestDebug/TestResults +) ``` -Results appear in `TestResult-*.xml` in the repo root. +Results are `.trx` files under `bin/TestDebug/TestResults/`. Repeat any `-c` and `-p:` arguments on both the build/install and `dotnet test --no-build` commands. ### Java.Interop tests Tooling tests are standalone (`dotnet test` on `.csproj`). JVM tests require the local SDK: diff --git a/.github/skills/tests/references/test-catalog.md b/.github/skills/tests/references/test-catalog.md index 87f6b599387..991dbee01fb 100644 --- a/.github/skills/tests/references/test-catalog.md +++ b/.github/skills/tests/references/test-catalog.md @@ -4,10 +4,10 @@ Mapping of test area keywords to assemblies, filters, and build prerequisites. **Legend:** - **Assembly**: The test DLL to pass to `dotnet test` (relative to repo root). `${TFM}` = current `DotNetStableTargetFramework` from `Directory.Build.props`. -- **Filter**: The `--filter` argument for `dotnet test`, or special instructions for non-`dotnet test` runs. +- **Filter**: The `--filter` argument for host-side `dotnet test`, or on-device MTP category/property notes. - **Build**: What must be built before running: - **Standalone** — Can run with plain `dotnet test .csproj`. No local SDK needed. - - **Full-build** — Requires the local SDK (`dotnet-local.sh`). Build with `./dotnet-local.sh build Xamarin.Android.sln -c Debug` or `make`. + - **Full-build** — Requires the local SDK (`dotnet-local.sh`). Build with `./dotnet-local.sh build Xamarin.Android.sln -c Debug` or `make prepare && make all`. - **Device**: Whether an Android device/emulator is required. --- @@ -41,7 +41,7 @@ These tests can be run immediately with `dotnet test` on the `.csproj`, even if ## Host-Side MSBuild Tests (full-build — requires local SDK) Assembly: `bin/TestDebug/${TFM}/Xamarin.Android.Build.Tests.dll` -Build: Full-build — `./dotnet-local.sh build Xamarin.Android.sln -c Debug` or `make` +Build: Full-build — `./dotnet-local.sh build Xamarin.Android.sln -c Debug` or `make prepare && make all` Device: No | Test Area | Filter | Test Classes / Notes | @@ -114,7 +114,9 @@ Device: **Yes** (most tests have `[Category("UsesDevice")]`) ## On-Device Runtime Tests (full-build — requires local SDK + device) -These use NUnitLite and run directly on the device via `-t:RunTestApp`. They do NOT use `dotnet test`. +Mono.Android device tests use stock NUnit through Microsoft Testing Platform (MTP), not NUnitLite. This changed in dotnet/android#11224. + +Build/install with `-t:Install`, then run `dotnet test --no-build --report-trx` from the test project's directory so the project-local `global.json` selects the MTP runner. Build: Full-build + the test project itself Device: **Yes** @@ -131,8 +133,7 @@ Device: **Yes** | **system.xml** | Same project — tests in `System.Xml/` | XML processing tests | | **threading** | Same project — tests in `System.Threading/` | Threading and async tests | | **drawing** | Same project — tests in `System.Drawing/` | System.Drawing tests | -| **locales (on-device)** | `tests/locales/Xamarin.Android.Locale-Tests/Xamarin.Android.Locale-Tests.csproj` | Locale/culture/globalization | -| **embedded DSOs** | `tests/EmbeddedDSOs/EmbeddedDSO/EmbeddedDSO.csproj` | Native library loading | +| **jcwgen (on-device)** | `tests/CodeGen-Binding/Xamarin.Android.JcwGen-Tests/Xamarin.Android.JcwGen-Tests.csproj` | Java callable wrapper generation tests; same MTP command shape | ### On-device test categories @@ -145,12 +146,23 @@ Other categories: `SSL`, `InetAccess`, `JavaList`, `RuntimeConfig`, `Intune`, `N Command: ```bash -./dotnet-local.sh build -t:RunTestApp tests/Mono.Android-Tests/Mono.Android-Tests/Mono.Android.NET-Tests.csproj +./dotnet-local.sh build -t:Install -c Debug tests/Mono.Android-Tests/Mono.Android-Tests/Mono.Android.NET-Tests.csproj +( + cd tests/Mono.Android-Tests/Mono.Android-Tests + ../../../dotnet-local.sh test Mono.Android.NET-Tests.csproj --no-build -c Debug --report-trx --results-directory ../../../bin/TestDebug/TestResults +) ``` -Results appear in `TestResult-*.xml` in the repo root. +Results are `.trx` files under `bin/TestDebug/TestResults/` and are published as VSTest results in CI. Always pass the same configuration and MSBuild properties to the install and `dotnet test --no-build` commands (for example, `-c Release -p:UseMonoRuntime=false`). ---- +For `Xamarin.Android.JcwGen-Tests`, use the same pattern from `tests/CodeGen-Binding/Xamarin.Android.JcwGen-Tests/`: +```bash +./dotnet-local.sh build -t:Install -c Debug tests/CodeGen-Binding/Xamarin.Android.JcwGen-Tests/Xamarin.Android.JcwGen-Tests.csproj +( + cd tests/CodeGen-Binding/Xamarin.Android.JcwGen-Tests + ../../../dotnet-local.sh test Xamarin.Android.JcwGen-Tests.csproj --no-build -c Debug --report-trx --results-directory ../../../bin/TestDebug/TestResults +) +``` ## Java.Interop Tests — Mixed Tiers diff --git a/Documentation/building/unix/instructions.md b/Documentation/building/unix/instructions.md index 7283fa25f3c..5c743aef6bb 100644 --- a/Documentation/building/unix/instructions.md +++ b/Documentation/building/unix/instructions.md @@ -107,8 +107,8 @@ The `dotnet/android` repo contains several unit tests: * NUnit-based unit tests, for stand-alone assemblies and utilities. - * `.apk`-based unit tests, which are NUnitLite-based tests that need to - execute on an Android device. + * `.apk`/instrumentation-based unit tests, which need to execute on an + Android device. All unit tests can be executed via the `make run-all-tests` target: @@ -338,4 +338,3 @@ For example, to rebuild Mono for armeabi-v7a: # This updates bin/$(Configuration)/lib/xamarin.android/xbuild/Xamarin/Android/lib/armeabi-v7a/libmonosgen-2.0.so $ msbuild /t:_InstallRuntimes src/mono-runtimes/mono-runtimes.csproj - diff --git a/Documentation/project-docs/ExploringSources.md b/Documentation/project-docs/ExploringSources.md index 3fe0b04bd2a..f35f40f7783 100644 --- a/Documentation/project-docs/ExploringSources.md +++ b/Documentation/project-docs/ExploringSources.md @@ -42,7 +42,6 @@ is used within many of the output directories which are created. * `Mono.Android.Export`: Builds `Mono.Android.Export.dll`. * `Xamarin.Android.Build.Tasks`: MSBuild tasks for Xamarin.Android projects. * `Xamarin.Android.Build.Utilities`: MSBuild tasks support sources. - * `Xamarin.Android.NUnitLite`: NUnitLite for Android sources. * `Xamarin.AndroidTools.Aidl`: AIDL processor, used in MSBuild tasks. * `Xamarin.Android.Tools.BootstrapTasks`: supplemental build tasks used by some build-tools. (This should be in `build-tools`; oops.) diff --git a/Documentation/workflow/UnitTests.md b/Documentation/workflow/UnitTests.md index b5c7518adca..51b8f62b5a6 100644 --- a/Documentation/workflow/UnitTests.md +++ b/Documentation/workflow/UnitTests.md @@ -391,30 +391,33 @@ public void MyAppShouldRunAndRespondToClick () ## On-Device Unit Tests There are a category of tests which run on the device itself, these test the -runtime behaviour. These run `NUnit` tests directly on the device. Some of -these are located in the runtime itself. We build them within the repo then run -the tests on the device. They use a custom mobile version of `NUnit` called -`NUnitLite`. For the most part they are the same. +runtime behaviour. The main runtime tests use stock `NUnit` through Microsoft +Testing Platform (MTP). We build and install them within the repo, then run +`dotnet test` from the test project directory. These tests are generally found in: * [`tests/Mono.Android-Tests`](../../tests/Mono.Android-Tests) + * [`tests/CodeGen-Binding/Xamarin.Android.JcwGen-Tests`](../../tests/CodeGen-Binding/Xamarin.Android.JcwGen-Tests) * [`tests/EmbeddedDSOs/EmbeddedDSO`](../../tests/EmbeddedDSOs/EmbeddedDSO) * [`tests/locales/Xamarin.Android.Locale-Tests`](../../tests/locales/Xamarin.Android.Locale-Tests) -These tests are run by using the `RunTestApp` target on the appropriate project -file, which includes: +The main runtime test project is: * `tests/Mono.Android-Tests/Mono.Android-Tests/Mono.Android.NET-Tests.csproj` For example: ```zsh -./dotnet-local.sh build -t:RunTestApp tests/Mono.Android-Tests/Mono.Android-Tests/Mono.Android.NET-Tests.csproj +./dotnet-local.sh build -t:Install -c Debug tests/Mono.Android-Tests/Mono.Android-Tests/Mono.Android.NET-Tests.csproj +( + cd tests/Mono.Android-Tests/Mono.Android-Tests + ../../../dotnet-local.sh test Mono.Android.NET-Tests.csproj --no-build -c Debug --report-trx --results-directory ../../../bin/TestDebug/TestResults +) ``` -After running the tests, a `TestResult*.xml` file will be created in the -top checkout directory containing the results of the tests. +Pass the same `-c` and `-p:` properties to both commands. After running the +tests, `.trx` result files will be created under `bin/TestDebug/TestResults`. The following is an example unit test. diff --git a/tests/Mono.Android-Tests/Mono.Android-Tests/Mono.Android.NET-Tests.csproj b/tests/Mono.Android-Tests/Mono.Android-Tests/Mono.Android.NET-Tests.csproj index a021ea0f2ee..bf233d38d9c 100644 --- a/tests/Mono.Android-Tests/Mono.Android-Tests/Mono.Android.NET-Tests.csproj +++ b/tests/Mono.Android-Tests/Mono.Android-Tests/Mono.Android.NET-Tests.csproj @@ -26,7 +26,7 @@