Skip to content

[native_toolchain_c] Use pair-wise testing#3187

Merged
dcharkes merged 1 commit intomainfrom
native-toolchain-c-pairwise-testing
Mar 25, 2026
Merged

[native_toolchain_c] Use pair-wise testing#3187
dcharkes merged 1 commit intomainfrom
native-toolchain-c-pairwise-testing

Conversation

@dcharkes
Copy link
Copy Markdown
Collaborator

@dcharkes dcharkes commented Mar 2, 2026

Gemini CLI vibe-coded test-case selection based on https://en.wikipedia.org/wiki/All-pairs_testing.

The interaction groups define the pairs we want to cover. The greedy algorithm covers all pairs (unless the configurations are invalid).

For the dimensions not mentioned in any interaction groups, the algorithm just does value coverage.

The test cases are written out to a markdown table such as

This file is generated. To regenerate, run:
`REGENERATE_TEST_CONFIGS=true dart test`

| #   | Architecture | Link Mode | Optimization Level | Android Api Level |
|-----|--------------|-----------|--------------------|-------------------|
| 1   | arm          | bundled   | unspecified        | 34                |
| 2   | arm          | static    | O2                 | 21                |
| 3   | arm64        | bundled   | O0                 | 34                |
| 4   | arm64        | static    | Os                 | 21                |
| 5   | ia32         | bundled   | O1                 | 21                |
| 6   | ia32         | static    | O2                 | 34                |
| 7   | riscv64      | bundled   | O0                 | 21                |
| 8   | riscv64      | static    | O1                 | 34                |
| 9   | x64          | bundled   | O3                 | 21                |
| 10  | x64          | static    | O3                 | 34                |

The test case generator is written in such a way that it is stable across runs and OSes (stable hashing).

Follow up of #3183.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 2, 2026

PR Health

Breaking changes ✔️
Package Change Current Version New Version Needed Version Looking good?
native_toolchain_c None 0.17.6 0.17.6 0.17.6 ✔️

This check can be disabled by tagging the PR with skip-breaking-check.

API leaks ✔️

The following packages contain symbols visible in the public API, but not exported by the library. Export these symbols or remove them from your publicly visible API.

Package Leaked API symbol Leaking sources

This check can be disabled by tagging the PR with skip-leaking-check.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 2, 2026

PR Health

Breaking changes ✔️
Package Change Current Version New Version Needed Version Looking good?
native_toolchain_c None 0.17.4 0.17.5-wip 0.17.5-wip ✔️

This check can be disabled by tagging the PR with skip-breaking-check.

License Headers ⚠️
""
Files
pkgs/native_toolchain_c/test/cbuilder/cbuilder_cross_android_test.dart
pkgs/native_toolchain_c/test/cbuilder/cbuilder_cross_ios_test.dart
pkgs/native_toolchain_c/test/cbuilder/cbuilder_cross_linux_host_test.dart
pkgs/native_toolchain_c/test/cbuilder/cbuilder_cross_macos_host_test.dart
pkgs/native_toolchain_c/test/clinker/objects_cross_android_test.dart
pkgs/native_toolchain_c/test/clinker/objects_cross_ios_test.dart
pkgs/native_toolchain_c/test/clinker/treeshake_cross_android_test.dart
pkgs/native_toolchain_c/test/clinker/treeshake_cross_ios_test.dart
pkgs/native_toolchain_c/test/utils/test_configuration_generator.dart
pkgs/native_toolchain_c/test/utils/test_configuration_generator_test.dart

All source files should start with a license header.

Unrelated files missing license headers
Files
pkgs/code_assets/example/api/code_asset_snippet.dart
pkgs/code_assets/example/api/code_assets_snippet.dart
pkgs/code_assets/example/api/code_config_snippet.dart
pkgs/code_assets/example/api/test_snippet.dart
pkgs/code_assets/example/host_name/hook/build.dart
pkgs/code_assets/example/host_name/lib/host_name.dart
pkgs/code_assets/example/host_name/lib/src/host_name.dart
pkgs/code_assets/example/host_name/test/host_name_test.dart
pkgs/code_assets/example/host_name/tool/ffigen.dart
pkgs/code_assets/example/mini_audio/bin/mini_audio.dart
pkgs/code_assets/example/mini_audio/hook/build.dart
pkgs/code_assets/example/mini_audio/lib/mini_audio.dart
pkgs/code_assets/example/mini_audio/lib/src/mini_audio.dart
pkgs/code_assets/example/mini_audio/test/mini_audio_test.dart
pkgs/code_assets/example/mini_audio/tool/ffigen.dart
pkgs/code_assets/example/sqlite/hook/build.dart
pkgs/code_assets/example/sqlite/lib/sqlite.dart
pkgs/code_assets/example/sqlite/lib/src/sqlite.dart
pkgs/code_assets/example/sqlite/test/sqlite_test.dart
pkgs/code_assets/example/sqlite/tool/ffigen.dart
pkgs/code_assets/example/sqlite_prebuilt/hook/build.dart
pkgs/code_assets/example/sqlite_prebuilt/lib/sqlite_prebuilt.dart
pkgs/code_assets/example/sqlite_prebuilt/lib/src/sqlite_prebuilt.dart
pkgs/code_assets/example/sqlite_prebuilt/test/sqlite_prebuilt_test.dart
pkgs/code_assets/example/sqlite_prebuilt/tool/ffigen.dart
pkgs/code_assets/example/stb_image/hook/build.dart
pkgs/code_assets/example/stb_image/lib/src/stb_image.dart
pkgs/code_assets/example/stb_image/lib/stb_image.dart
pkgs/code_assets/example/stb_image/test/stb_image_test.dart
pkgs/code_assets/example/stb_image/tool/ffigen.dart
pkgs/code_assets/lib/code_assets.dart
pkgs/code_assets/lib/src/code_assets/architecture.dart
pkgs/code_assets/lib/src/code_assets/c_compiler_config.dart
pkgs/code_assets/lib/src/code_assets/code_asset.dart
pkgs/code_assets/lib/src/code_assets/config.dart
pkgs/code_assets/lib/src/code_assets/extension.dart
pkgs/code_assets/lib/src/code_assets/ios_sdk.dart
pkgs/code_assets/lib/src/code_assets/link_mode.dart
pkgs/code_assets/lib/src/code_assets/link_mode_preference.dart
pkgs/code_assets/lib/src/code_assets/os.dart
pkgs/code_assets/lib/src/code_assets/testing.dart
pkgs/code_assets/lib/src/code_assets/validation.dart
pkgs/code_assets/test/code_assets/asset_test.dart
pkgs/code_assets/test/code_assets/code_asset_test.dart
pkgs/code_assets/test/code_assets/link_mode_test.dart
pkgs/code_assets/test/code_assets/validation_test.dart
pkgs/code_assets/test/example/external_references_test.dart
pkgs/code_assets/test/example/pub_publish_size_test.dart
pkgs/code_assets/test/json_schema/helpers.dart
pkgs/code_assets/test/json_schema/schema_test.dart
pkgs/data_assets/example/api/data_assets_snippet.dart
pkgs/data_assets/lib/data_assets.dart
pkgs/data_assets/lib/src/data_assets/config.dart
pkgs/data_assets/lib/src/data_assets/data_asset.dart
pkgs/data_assets/lib/src/data_assets/extension.dart
pkgs/data_assets/lib/src/data_assets/validation.dart
pkgs/data_assets/test/data_assets/asset_test.dart
pkgs/data_assets/test/data_assets/data_asset_test.dart
pkgs/data_assets/test/data_assets/validation_test.dart
pkgs/data_assets/test/json_schema/helpers.dart
pkgs/data_assets/test/json_schema/schema_test.dart
pkgs/ffi/example/main.dart
pkgs/ffi/lib/ffi.dart
pkgs/ffi/lib/src/allocation.dart
pkgs/ffi/lib/src/arena.dart
pkgs/ffi/lib/src/utf16.dart
pkgs/ffi/lib/src/utf8.dart
pkgs/ffi/test/allocation_test.dart
pkgs/ffi/test/arena_test.dart
pkgs/ffi/test/utf16_test.dart
pkgs/ffi/test/utf8_test.dart
pkgs/hooks/example/api/build_snippet_1.dart
pkgs/hooks/example/api/build_snippet_2.dart
pkgs/hooks/example/api/builder_snippet.dart
pkgs/hooks/example/api/config_snippet_1.dart
pkgs/hooks/example/api/config_snippet_2.dart
pkgs/hooks/example/api/config_snippet_3.dart
pkgs/hooks/example/api/config_snippet_4.dart
pkgs/hooks/example/api/config_snippet_5.dart
pkgs/hooks/example/api/link_snippet.dart
pkgs/hooks/example/api/test_snippet.dart
pkgs/hooks/example/build/download_asset/hook/build.dart
pkgs/hooks/example/build/download_asset/lib/src/hook_helpers/c_build.dart
pkgs/hooks/example/build/download_asset/lib/src/hook_helpers/download.dart
pkgs/hooks/example/build/download_asset/lib/src/hook_helpers/target_versions.dart
pkgs/hooks/example/build/download_asset/lib/src/hook_helpers/targets.dart
pkgs/hooks/example/build/download_asset/lib/src/hook_helpers/version.dart
pkgs/hooks/example/build/download_asset/test/native_add_test.dart
pkgs/hooks/example/build/download_asset/tool/build.dart
pkgs/hooks/example/build/download_asset/tool/generate_asset_hashes.dart
pkgs/hooks/example/build/local_asset/hook/build.dart
pkgs/hooks/example/build/local_asset/test/build_test.dart
pkgs/hooks/example/build/native_add_app/bin/native_add_app.dart
pkgs/hooks/example/build/native_add_app/test/native_add_library_test.dart
pkgs/hooks/example/build/native_add_library/hook/build.dart
pkgs/hooks/example/build/native_add_library/test/native_add_library_test.dart
pkgs/hooks/example/build/native_dynamic_linking/hook/build.dart
pkgs/hooks/example/build/native_dynamic_linking/test/add_test.dart
pkgs/hooks/example/build/system_library/hook/build.dart
pkgs/hooks/example/build/system_library/lib/memory.dart
pkgs/hooks/example/build/system_library/test/memory_test.dart
pkgs/hooks/example/build/use_dart_api/hook/build.dart
pkgs/hooks/example/build/use_dart_api/lib/use_dart_api.dart
pkgs/hooks/example/build/use_dart_api/test/use_dart_api_test.dart
pkgs/hooks/example/link/app_with_asset_treeshaking/bin/app_with_asset_treeshaking.dart
pkgs/hooks/example/link/app_with_asset_treeshaking/lib/app_with_asset_treeshaking.dart
pkgs/hooks/example/link/package_with_assets/hook/build.dart
pkgs/hooks/example/link/package_with_assets/hook/link.dart
pkgs/hooks/example/link/package_with_assets/lib/package_with_assets.dart
pkgs/hooks/lib/hooks.dart
pkgs/hooks/lib/src/api/build_and_link.dart
pkgs/hooks/lib/src/api/builder.dart
pkgs/hooks/lib/src/api/linker.dart
pkgs/hooks/lib/src/args_parser.dart
pkgs/hooks/lib/src/avoid_import_outside_src_rule_test.dart
pkgs/hooks/lib/src/config.dart
pkgs/hooks/lib/src/encoded_asset.dart
pkgs/hooks/lib/src/extension.dart
pkgs/hooks/lib/src/metadata.dart
pkgs/hooks/lib/src/test.dart
pkgs/hooks/lib/src/user_defines.dart
pkgs/hooks/lib/src/utils/datetime.dart
pkgs/hooks/lib/src/utils/file.dart
pkgs/hooks/lib/src/utils/json.dart
pkgs/hooks/lib/src/utils/uri.dart
pkgs/hooks/lib/src/validation.dart
pkgs/hooks/test/api/build_test.dart
pkgs/hooks/test/build_input_test.dart
pkgs/hooks/test/build_output_test.dart
pkgs/hooks/test/checksum_test.dart
pkgs/hooks/test/example/external_references_test.dart
pkgs/hooks/test/example/local_asset_test.dart
pkgs/hooks/test/example/native_add_library_test.dart
pkgs/hooks/test/example/native_dynamic_linking_test.dart
pkgs/hooks/test/example/pub_publish_size_test.dart
pkgs/hooks/test/helpers.dart
pkgs/hooks/test/json_schema/helpers.dart
pkgs/hooks/test/json_schema/schema_test.dart
pkgs/hooks/test/link_input_test.dart
pkgs/hooks/test/link_output_test.dart
pkgs/hooks/test/model/metadata_asset_test.dart
pkgs/hooks/test/test_test.dart
pkgs/hooks/test/validation_test.dart
pkgs/hooks/tool/generate_schemas.dart
pkgs/hooks/tool/generate_syntax.dart
pkgs/hooks/tool/normalize.dart
pkgs/hooks/tool/update_snippets.dart
pkgs/hooks_runner/lib/hooks_runner.dart
pkgs/hooks_runner/lib/src/build_runner/build_planner.dart
pkgs/hooks_runner/lib/src/build_runner/build_runner.dart
pkgs/hooks_runner/lib/src/build_runner/failure.dart
pkgs/hooks_runner/lib/src/build_runner/result.dart
pkgs/hooks_runner/lib/src/build_runner/tracing_file_system.dart
pkgs/hooks_runner/lib/src/dependencies_hash_file/dependencies_hash_file.dart
pkgs/hooks_runner/lib/src/locking/locking.dart
pkgs/hooks_runner/lib/src/model/build_result.dart
pkgs/hooks_runner/lib/src/model/hook_result.dart
pkgs/hooks_runner/lib/src/model/kernel_assets.dart
pkgs/hooks_runner/lib/src/model/link_result.dart
pkgs/hooks_runner/lib/src/model/target.dart
pkgs/hooks_runner/lib/src/package_layout/package_layout.dart
pkgs/hooks_runner/lib/src/utils/file.dart
pkgs/hooks_runner/lib/src/utils/run_process.dart
pkgs/hooks_runner/lib/src/utils/uri.dart
pkgs/hooks_runner/test/build_runner/absolute_path_test.dart
pkgs/hooks_runner/test/build_runner/build_dependencies_test.dart
pkgs/hooks_runner/test/build_runner/build_planner_test.dart
pkgs/hooks_runner/test/build_runner/build_process_helper.dart
pkgs/hooks_runner/test/build_runner/build_runner_asset_id_test.dart
pkgs/hooks_runner/test/build_runner/build_runner_build_output_format_test.dart
pkgs/hooks_runner/test/build_runner/build_runner_caching_test.dart
pkgs/hooks_runner/test/build_runner/build_runner_cycle_test.dart
pkgs/hooks_runner/test/build_runner/build_runner_failure_test.dart
pkgs/hooks_runner/test/build_runner/build_runner_non_root_package_test.dart
pkgs/hooks_runner/test/build_runner/build_runner_reusability_test.dart
pkgs/hooks_runner/test/build_runner/build_runner_test.dart
pkgs/hooks_runner/test/build_runner/concurrency_shared_test.dart
pkgs/hooks_runner/test/build_runner/concurrency_test.dart
pkgs/hooks_runner/test/build_runner/concurrency_test_helper.dart
pkgs/hooks_runner/test/build_runner/conflicting_dylib_test.dart
pkgs/hooks_runner/test/build_runner/environment_filter_test.dart
pkgs/hooks_runner/test/build_runner/fail_on_os_sdk_version_test.dart
pkgs/hooks_runner/test/build_runner/helpers.dart
pkgs/hooks_runner/test/build_runner/link_caching_test.dart
pkgs/hooks_runner/test/build_runner/link_test.dart
pkgs/hooks_runner/test/build_runner/metadata_test.dart
pkgs/hooks_runner/test/build_runner/no_build_output_test.dart
pkgs/hooks_runner/test/build_runner/package_layout_test.dart
pkgs/hooks_runner/test/build_runner/packaging_preference_test.dart
pkgs/hooks_runner/test/build_runner/parse_dep_file_test.dart
pkgs/hooks_runner/test/build_runner/pub_workspace_test.dart
pkgs/hooks_runner/test/build_runner/resources_test.dart
pkgs/hooks_runner/test/build_runner/system_library_test.dart
pkgs/hooks_runner/test/build_runner/version_skew_test.dart
pkgs/hooks_runner/test/build_runner/wrong_linker_test.dart
pkgs/hooks_runner/test/dependencies_hash_file/dependencies_hash_file_test.dart
pkgs/hooks_runner/test/helpers.dart
pkgs/hooks_runner/test/locking/locking_test.dart
pkgs/hooks_runner/test/locking/locking_test_helper.dart
pkgs/hooks_runner/test/model/kernel_assets_test.dart
pkgs/hooks_runner/test/model/target_test.dart
pkgs/hooks_runner/test/test_data/native_dynamic_linking_helper.dart
pkgs/hooks_runner/test/test_data/native_dynamic_linking_test.dart
pkgs/hooks_runner/test/test_data/reusable_dynamic_library_test.dart
pkgs/hooks_runner/test/test_data/transformer_test.dart
pkgs/hooks_runner/test/test_data/user_defines_test.dart
pkgs/hooks_runner/test_data/add_asset_link/bin/add_asset_link.dart
pkgs/hooks_runner/test_data/add_asset_link/hook/build.dart
pkgs/hooks_runner/test_data/add_asset_link/hook/link.dart
pkgs/hooks_runner/test_data/add_asset_link/lib/add_asset_link.dart
pkgs/hooks_runner/test_data/add_asset_link/lib/src/add_asset_link.dart
pkgs/hooks_runner/test_data/add_asset_link/lib/src/add_asset_link_bindings.dart
pkgs/hooks_runner/test_data/complex_link/bin/complex_link.dart
pkgs/hooks_runner/test_data/complex_link/hook/build.dart
pkgs/hooks_runner/test_data/complex_link/hook/link.dart
pkgs/hooks_runner/test_data/complex_link_helper/hook/build.dart
pkgs/hooks_runner/test_data/complex_link_helper/lib/complex_link_helper.dart
pkgs/hooks_runner/test_data/cyclic_link_package_1/hook/link.dart
pkgs/hooks_runner/test_data/cyclic_link_package_2/hook/link.dart
pkgs/hooks_runner/test_data/cyclic_package_1/hook/build.dart
pkgs/hooks_runner/test_data/cyclic_package_2/hook/build.dart
pkgs/hooks_runner/test_data/dart_app/bin/dart_app.dart
pkgs/hooks_runner/test_data/depend_on_fail_build/hook/build.dart
pkgs/hooks_runner/test_data/depend_on_fail_build/lib/depend_on_fail_build.dart
pkgs/hooks_runner/test_data/depend_on_fail_build_app/bin/depend_on_fail_build_app.dart
pkgs/hooks_runner/test_data/dev_dependency_with_hook/test/my_test.dart
pkgs/hooks_runner/test_data/download_assets/hook/build.dart
pkgs/hooks_runner/test_data/drop_dylib_link/bin/drop_dylib_link.dart
pkgs/hooks_runner/test_data/drop_dylib_link/hook/build.dart
pkgs/hooks_runner/test_data/drop_dylib_link/hook/link.dart
pkgs/hooks_runner/test_data/drop_dylib_link/lib/drop_dylib_link.dart
pkgs/hooks_runner/test_data/drop_dylib_link/lib/src/drop_dylib_link.dart
pkgs/hooks_runner/test_data/drop_dylib_link/lib/src/drop_dylib_link_bindings.dart
pkgs/hooks_runner/test_data/fail_build/hook/build.dart
pkgs/hooks_runner/test_data/fail_build/lib/fail_build.dart
pkgs/hooks_runner/test_data/fail_on_os_sdk_version/hook/build.dart
pkgs/hooks_runner/test_data/fail_on_os_sdk_version_link/hook/build.dart
pkgs/hooks_runner/test_data/fail_on_os_sdk_version_linker/hook/link.dart
pkgs/hooks_runner/test_data/flag_app/bin/flag_app.dart
pkgs/hooks_runner/test_data/flag_app/lib/flag_app.dart
pkgs/hooks_runner/test_data/flag_enthusiast_1/hook/link.dart
pkgs/hooks_runner/test_data/flag_enthusiast_1/lib/flag_enthusiast_1.dart
pkgs/hooks_runner/test_data/flag_enthusiast_1/lib/src/flag_enthusiast_1_base.dart
pkgs/hooks_runner/test_data/flag_enthusiast_2/hook/link.dart
pkgs/hooks_runner/test_data/flag_enthusiast_2/lib/flag_enthusiast_2.dart
pkgs/hooks_runner/test_data/flag_enthusiast_2/lib/src/flag_enthusiast_2_base.dart
pkgs/hooks_runner/test_data/fun_with_flags/hook/build.dart
pkgs/hooks_runner/test_data/fun_with_flags/hook/link.dart
pkgs/hooks_runner/test_data/fun_with_flags/lib/fun_with_flags.dart
pkgs/hooks_runner/test_data/fun_with_flags/lib/hook.dart
pkgs/hooks_runner/test_data/fun_with_flags/lib/src/fun_with_flags_base.dart
pkgs/hooks_runner/test_data/fun_with_flags/lib/src/hook.dart
pkgs/hooks_runner/test_data/infra_failure/bin/infra_failure.dart
pkgs/hooks_runner/test_data/infra_failure/hook/build.dart
pkgs/hooks_runner/test_data/link_inverse_app/hook/link.dart
pkgs/hooks_runner/test_data/link_inverse_package/hook/link.dart
pkgs/hooks_runner/test_data/manifest_generator.dart
pkgs/hooks_runner/test_data/native_add/hook/build.dart
pkgs/hooks_runner/test_data/native_add/lib/native_add.dart
pkgs/hooks_runner/test_data/native_add/lib/src/native_add.dart
pkgs/hooks_runner/test_data/native_add/test/native_add_test.dart
pkgs/hooks_runner/test_data/native_add_add_source/hook/build.dart
pkgs/hooks_runner/test_data/native_add_duplicate/bin/native_add_duplicate.dart
pkgs/hooks_runner/test_data/native_add_duplicate/hook/build.dart
pkgs/hooks_runner/test_data/native_add_duplicate/hook/link.dart
pkgs/hooks_runner/test_data/native_add_version_skew/hook/build.dart
pkgs/hooks_runner/test_data/native_add_version_skew/lib/native_add.dart
pkgs/hooks_runner/test_data/native_add_version_skew/lib/src/native_add.dart
pkgs/hooks_runner/test_data/native_add_version_skew/test/native_add_test.dart
pkgs/hooks_runner/test_data/native_add_version_skew_2/hook/build.dart
pkgs/hooks_runner/test_data/native_dynamic_linking/bin/native_dynamic_linking.dart
pkgs/hooks_runner/test_data/native_dynamic_linking/hook/build.dart
pkgs/hooks_runner/test_data/native_dynamic_linking/test/add_test.dart
pkgs/hooks_runner/test_data/native_subtract/hook/build.dart
pkgs/hooks_runner/test_data/native_subtract/lib/native_subtract.dart
pkgs/hooks_runner/test_data/native_subtract/lib/src/native_subtract.dart
pkgs/hooks_runner/test_data/no_asset_for_link/hook/link.dart
pkgs/hooks_runner/test_data/no_build_output/hook/build.dart
pkgs/hooks_runner/test_data/no_hook/lib/no_hook.dart
pkgs/hooks_runner/test_data/package_reading_metadata/hook/build.dart
pkgs/hooks_runner/test_data/package_with_metadata/hook/build.dart
pkgs/hooks_runner/test_data/pirate_adventure/bin/pirate_adventure.dart
pkgs/hooks_runner/test_data/pirate_speak/hook/build.dart
pkgs/hooks_runner/test_data/pirate_speak/hook/link.dart
pkgs/hooks_runner/test_data/pirate_speak/lib/pirate_speak.dart
pkgs/hooks_runner/test_data/pirate_speak/lib/src/definitions.dart
pkgs/hooks_runner/test_data/pirate_technology/hook/build.dart
pkgs/hooks_runner/test_data/pirate_technology/hook/link.dart
pkgs/hooks_runner/test_data/pirate_technology/lib/pirate_technology.dart
pkgs/hooks_runner/test_data/pirate_technology/lib/src/definitions.dart
pkgs/hooks_runner/test_data/recursive_invocation/bin/subprocess.dart
pkgs/hooks_runner/test_data/recursive_invocation/hook/build.dart
pkgs/hooks_runner/test_data/recursive_invocation/lib/recursive_invocation.dart
pkgs/hooks_runner/test_data/recursive_invocation/lib/src/recursive_invocation.dart
pkgs/hooks_runner/test_data/relative_path/hook/build.dart
pkgs/hooks_runner/test_data/reusable_dynamic_library/hook/build.dart
pkgs/hooks_runner/test_data/reusable_dynamic_library/lib/hook.dart
pkgs/hooks_runner/test_data/reusable_dynamic_library/test/add_test.dart
pkgs/hooks_runner/test_data/reuse_dynamic_library/hook/build.dart
pkgs/hooks_runner/test_data/reuse_dynamic_library/test/add_test.dart
pkgs/hooks_runner/test_data/simple_data_asset/hook/build.dart
pkgs/hooks_runner/test_data/simple_link/bin/simple_link.dart
pkgs/hooks_runner/test_data/simple_link/hook/build.dart
pkgs/hooks_runner/test_data/simple_link/hook/link.dart
pkgs/hooks_runner/test_data/some_dev_dep/bin/some_dev_dep.dart
pkgs/hooks_runner/test_data/system_library/hook/build.dart
pkgs/hooks_runner/test_data/system_library/lib/memory_executable.dart
pkgs/hooks_runner/test_data/system_library/lib/memory_process.dart
pkgs/hooks_runner/test_data/system_library/lib/memory_system.dart
pkgs/hooks_runner/test_data/system_library/test/memory_test.dart
pkgs/hooks_runner/test_data/transformer/hook/build.dart
pkgs/hooks_runner/test_data/transformer/lib/src/transform.dart
pkgs/hooks_runner/test_data/transformer/tool/generate_data.dart
pkgs/hooks_runner/test_data/treeshaking_native_libs/bin/treeshaking_native_libs.dart
pkgs/hooks_runner/test_data/treeshaking_native_libs/hook/build.dart
pkgs/hooks_runner/test_data/treeshaking_native_libs/hook/link.dart
pkgs/hooks_runner/test_data/treeshaking_native_libs/lib/src/treeshaking_native_libs.dart
pkgs/hooks_runner/test_data/treeshaking_native_libs/lib/treeshaking_native_libs.dart
pkgs/hooks_runner/test_data/treeshaking_native_libs/test/treeshaking_native_libs_test.dart
pkgs/hooks_runner/test_data/use_all_api/hook/build.dart
pkgs/hooks_runner/test_data/use_all_api/hook/link.dart
pkgs/hooks_runner/test_data/user_defines/bin/user_defines.dart
pkgs/hooks_runner/test_data/user_defines/hook/build.dart
pkgs/hooks_runner/test_data/user_defines/test/user_defines_test.dart
pkgs/hooks_runner/test_data/wrong_build_output/hook/build.dart
pkgs/hooks_runner/test_data/wrong_build_output_2/hook/build.dart
pkgs/hooks_runner/test_data/wrong_build_output_3/hook/build.dart
pkgs/hooks_runner/test_data/wrong_linker/hook/build.dart
pkgs/hooks_runner/test_data/wrong_namespace_asset/hook/build.dart
pkgs/jni/bin/setup.dart
pkgs/jni/example/integration_test/on_device_jni_test.dart
pkgs/jni/lib/src/accessors.dart
pkgs/jni/lib/src/build_util/build_util.dart
pkgs/jni/lib/src/errors.dart
pkgs/jni/lib/src/jarray.dart
pkgs/jni/lib/src/jclass.dart
pkgs/jni/lib/src/jimplementer.dart
pkgs/jni/lib/src/jni.dart
pkgs/jni/lib/src/jobject.dart
pkgs/jni/lib/src/jprimitives.dart
pkgs/jni/lib/src/jreference.dart
pkgs/jni/lib/src/jvalues.dart
pkgs/jni/lib/src/kotlin.dart
pkgs/jni/lib/src/lang/jboolean.dart
pkgs/jni/lib/src/lang/jbyte.dart
pkgs/jni/lib/src/lang/jcharacter.dart
pkgs/jni/lib/src/lang/jdouble.dart
pkgs/jni/lib/src/lang/jfloat.dart
pkgs/jni/lib/src/lang/jinteger.dart
pkgs/jni/lib/src/lang/jlong.dart
pkgs/jni/lib/src/lang/jnumber.dart
pkgs/jni/lib/src/lang/jshort.dart
pkgs/jni/lib/src/lang/jstring.dart
pkgs/jni/lib/src/lang/lang.dart
pkgs/jni/lib/src/method_invocation.dart
pkgs/jni/lib/src/nio/jbuffer.dart
pkgs/jni/lib/src/nio/jbyte_buffer.dart
pkgs/jni/lib/src/nio/nio.dart
pkgs/jni/lib/src/third_party/generated_bindings.dart
pkgs/jni/lib/src/types.dart
pkgs/jni/lib/src/util/jiterator.dart
pkgs/jni/lib/src/util/jlist.dart
pkgs/jni/lib/src/util/jmap.dart
pkgs/jni/lib/src/util/jset.dart
pkgs/jni/lib/src/util/util.dart
pkgs/jni/test/boxed_test.dart
pkgs/jni/test/debug_release_test.dart
pkgs/jni/test/exception_test.dart
pkgs/jni/test/global_env_test.dart
pkgs/jni/test/isolate_test.dart
pkgs/jni/test/jarray_test.dart
pkgs/jni/test/jbyte_buffer_test.dart
pkgs/jni/test/jlist_test.dart
pkgs/jni/test/jmap_test.dart
pkgs/jni/test/jobject_test.dart
pkgs/jni/test/jset_test.dart
pkgs/jni/test/jstring_test.dart
pkgs/jni/test/load_test.dart
pkgs/jni/test/test_util/test_util.dart
pkgs/jni/test/type_test.dart
pkgs/jni/tool/generate_ide_files.dart
pkgs/jni/tool/generate_jni_bindings.dart
pkgs/jni/tool/wrapper_generators/ffigen_util.dart
pkgs/jni/tool/wrapper_generators/generate_c_extensions.dart
pkgs/jni/tool/wrapper_generators/generate_dart_extensions.dart
pkgs/jni/tool/wrapper_generators/generate_helper_functions.dart
pkgs/jni/tool/wrapper_generators/logging.dart
pkgs/json_syntax_generator/lib/src/generator/code_generation_helpers.dart
pkgs/json_syntax_generator/lib/src/generator/enum_class_generator.dart
pkgs/json_syntax_generator/lib/src/generator/property_generator.dart
pkgs/json_syntax_generator/lib/src/generator/syntax_generator.dart
pkgs/json_syntax_generator/lib/src/model/class_info.dart
pkgs/json_syntax_generator/lib/src/model/dart_type.dart
pkgs/json_syntax_generator/lib/src/model/property_info.dart
pkgs/json_syntax_generator/lib/src/model/schema_info.dart
pkgs/json_syntax_generator/lib/src/model/utils.dart
pkgs/json_syntax_generator/lib/src/parser/schema_analyzer.dart
pkgs/native_doc_dartifier/example/dartify_example.dart
pkgs/native_doc_dartifier/example/dartify_rag_example.dart
pkgs/native_doc_dartifier/lib/src/ast.dart
pkgs/native_doc_dartifier/lib/src/code_processor.dart
pkgs/native_doc_dartifier/lib/src/context.dart
pkgs/native_doc_dartifier/lib/src/dartify_code.dart
pkgs/native_doc_dartifier/lib/src/populate_rag.dart
pkgs/native_doc_dartifier/lib/src/prompts.dart
pkgs/native_doc_dartifier/lib/src/public_abstractor.dart
pkgs/native_doc_dartifier/lib/src/rag.dart
pkgs/native_doc_dartifier/lib/src/rag_models.dart
pkgs/native_doc_dartifier/test/dartify_simple_cases/java_snippets.dart
pkgs/native_doc_dartifier/test/dartify_simple_cases/runtime_test.dart
pkgs/native_doc_dartifier/test/imported_packages_test.dart
pkgs/native_doc_dartifier/test/public_abstractor_test.dart
pkgs/native_doc_dartifier/test/rag_test.dart
pkgs/native_doc_dartifier/tool/compile_java.dart
pkgs/native_doc_dartifier/tool/prepare_dartify_test.dart
pkgs/native_test_helpers/lib/native_test_helpers.dart
pkgs/native_test_helpers/lib/src/find_package_root.dart
pkgs/native_test_helpers/lib/src/yaml_to_json.dart
pkgs/native_toolchain_c/lib/native_toolchain_c.dart
pkgs/native_toolchain_c/lib/src/cbuilder/build_mode.dart
pkgs/native_toolchain_c/lib/src/cbuilder/cbuilder.dart
pkgs/native_toolchain_c/lib/src/cbuilder/clinker.dart
pkgs/native_toolchain_c/lib/src/cbuilder/compiler_resolver.dart
pkgs/native_toolchain_c/lib/src/cbuilder/ctool.dart
pkgs/native_toolchain_c/lib/src/cbuilder/language.dart
pkgs/native_toolchain_c/lib/src/cbuilder/linker_options.dart
pkgs/native_toolchain_c/lib/src/cbuilder/linkmode.dart
pkgs/native_toolchain_c/lib/src/cbuilder/logger.dart
pkgs/native_toolchain_c/lib/src/cbuilder/optimization_level.dart
pkgs/native_toolchain_c/lib/src/cbuilder/output_type.dart
pkgs/native_toolchain_c/lib/src/cbuilder/run_cbuilder.dart
pkgs/native_toolchain_c/lib/src/native_toolchain/android_ndk.dart
pkgs/native_toolchain_c/lib/src/native_toolchain/apple_clang.dart
pkgs/native_toolchain_c/lib/src/native_toolchain/clang.dart
pkgs/native_toolchain_c/lib/src/native_toolchain/gcc.dart
pkgs/native_toolchain_c/lib/src/native_toolchain/msvc.dart
pkgs/native_toolchain_c/lib/src/native_toolchain/recognizer.dart
pkgs/native_toolchain_c/lib/src/native_toolchain/tool_likeness.dart
pkgs/native_toolchain_c/lib/src/native_toolchain/xcode.dart
pkgs/native_toolchain_c/lib/src/tool/tool.dart
pkgs/native_toolchain_c/lib/src/tool/tool_error.dart
pkgs/native_toolchain_c/lib/src/tool/tool_instance.dart
pkgs/native_toolchain_c/lib/src/tool/tool_requirement.dart
pkgs/native_toolchain_c/lib/src/tool/tool_resolver.dart
pkgs/native_toolchain_c/lib/src/utils/env_from_bat.dart
pkgs/native_toolchain_c/lib/src/utils/run_process.dart
pkgs/native_toolchain_c/lib/src/utils/sem_version.dart
pkgs/native_toolchain_c/test/cbuilder/cbuilder_build_failure_test.dart
pkgs/native_toolchain_c/test/cbuilder/cbuilder_cross_windows_host_test.dart
pkgs/native_toolchain_c/test/cbuilder/cbuilder_test.dart
pkgs/native_toolchain_c/test/cbuilder/compiler_resolver_test.dart
pkgs/native_toolchain_c/test/cbuilder/objective_c_test.dart
pkgs/native_toolchain_c/test/clinker/build_testfiles.dart
pkgs/native_toolchain_c/test/clinker/objects_cross_test.dart
pkgs/native_toolchain_c/test/clinker/objects_helper.dart
pkgs/native_toolchain_c/test/clinker/objects_test.dart
pkgs/native_toolchain_c/test/clinker/rust_test.dart
pkgs/native_toolchain_c/test/clinker/treeshake_cross_test.dart
pkgs/native_toolchain_c/test/clinker/treeshake_helper.dart
pkgs/native_toolchain_c/test/clinker/treeshake_test.dart
pkgs/native_toolchain_c/test/clinker/windows_module_definition_cross_test.dart
pkgs/native_toolchain_c/test/clinker/windows_module_definition_helper.dart
pkgs/native_toolchain_c/test/clinker/windows_module_definition_test.dart
pkgs/native_toolchain_c/test/helpers.dart
pkgs/native_toolchain_c/test/native_toolchain/apple_clang_test.dart
pkgs/native_toolchain_c/test/native_toolchain/clang_test.dart
pkgs/native_toolchain_c/test/native_toolchain/gcc_test.dart
pkgs/native_toolchain_c/test/native_toolchain/msvc_test.dart
pkgs/native_toolchain_c/test/native_toolchain/ndk_test.dart
pkgs/native_toolchain_c/test/native_toolchain/recognizer_test.dart
pkgs/native_toolchain_c/test/native_toolchain/xcode_test.dart
pkgs/native_toolchain_c/test/tool/tool_instance_test.dart
pkgs/native_toolchain_c/test/tool/tool_requirement_test.dart
pkgs/native_toolchain_c/test/tool/tool_resolver_test.dart
pkgs/native_toolchain_c/test/tool/tool_test.dart
pkgs/native_toolchain_c/test/utils/run_process_test.dart
pkgs/objective_c/example/command_line/lib/main.dart
pkgs/objective_c/example/flutter_app/lib/main.dart
pkgs/objective_c/hook/build.dart
pkgs/objective_c/lib/objective_c.dart
pkgs/objective_c/lib/src/autorelease.dart
pkgs/objective_c/lib/src/block.dart
pkgs/objective_c/lib/src/cf_string.dart
pkgs/objective_c/lib/src/converter.dart
pkgs/objective_c/lib/src/globals.dart
pkgs/objective_c/lib/src/internal.dart
pkgs/objective_c/lib/src/ns_array.dart
pkgs/objective_c/lib/src/ns_data.dart
pkgs/objective_c/lib/src/ns_date.dart
pkgs/objective_c/lib/src/ns_dictionary.dart
pkgs/objective_c/lib/src/ns_enumerator.dart
pkgs/objective_c/lib/src/ns_input_stream.dart
pkgs/objective_c/lib/src/ns_mutable_data.dart
pkgs/objective_c/lib/src/ns_number.dart
pkgs/objective_c/lib/src/ns_set.dart
pkgs/objective_c/lib/src/ns_string.dart
pkgs/objective_c/lib/src/observer.dart
pkgs/objective_c/lib/src/os_version.dart
pkgs/objective_c/lib/src/protocol_builder.dart
pkgs/objective_c/lib/src/selector.dart
pkgs/objective_c/test/autorelease_test.dart
pkgs/objective_c/test/cf_string_test.dart
pkgs/objective_c/test/converter_test.dart
pkgs/objective_c/test/generate_code_test.dart
pkgs/objective_c/test/interface_lists_test.dart
pkgs/objective_c/test/ns_input_stream_test.dart
pkgs/objective_c/test/ns_mutable_data_test.dart
pkgs/objective_c/test/nsarray_test.dart
pkgs/objective_c/test/nsdata_test.dart
pkgs/objective_c/test/nsdate_test.dart
pkgs/objective_c/test/nsdictionary_test.dart
pkgs/objective_c/test/nsmutablearray_test.dart
pkgs/objective_c/test/nsmutabledictionary_test.dart
pkgs/objective_c/test/nsmutableset_test.dart
pkgs/objective_c/test/nsnumber_test.dart
pkgs/objective_c/test/nsset_test.dart
pkgs/objective_c/test/nsstring_test.dart
pkgs/objective_c/test/observer_test.dart
pkgs/objective_c/test/os_version_test.dart
pkgs/objective_c/test/selector_test.dart
pkgs/objective_c/test/setup.dart
pkgs/objective_c/test/util.dart
pkgs/objective_c/tool/generate_code.dart
pkgs/pub_formats/lib/pub_formats.dart
pkgs/pub_formats/test/helpers.dart
pkgs/pub_formats/test/package_config_test.dart
pkgs/pub_formats/test/package_graph_test.dart
pkgs/pub_formats/test/pubspec_lock_test.dart
pkgs/pub_formats/test/pubspec_test.dart
pkgs/pub_formats/tool/generate.dart
pkgs/record_use/example/api/usage.dart
pkgs/record_use/example/api/usage_link.dart
pkgs/record_use/lib/record_use.dart
pkgs/record_use/lib/record_use_internal.dart
pkgs/record_use/lib/src/canonicalization_context.dart
pkgs/record_use/lib/src/constant.dart
pkgs/record_use/lib/src/definition.dart
pkgs/record_use/lib/src/helper.dart
pkgs/record_use/lib/src/loading_unit.dart
pkgs/record_use/lib/src/metadata.dart
pkgs/record_use/lib/src/recordings.dart
pkgs/record_use/lib/src/reference.dart
pkgs/record_use/lib/src/serialization_context.dart
pkgs/record_use/lib/src/version.dart
pkgs/record_use/test/canonicalization_test.dart
pkgs/record_use/test/complex_keys_test.dart
pkgs/record_use/test/extension_receiver_test.dart
pkgs/record_use/test/filter_test.dart
pkgs/record_use/test/instance_references_test.dart
pkgs/record_use/test/json_schema/schema_test.dart
pkgs/record_use/test/json_schema/uri_pattern_test.dart
pkgs/record_use/test/maybe_constant_test.dart
pkgs/record_use/test/non_constant_in_collection_test.dart
pkgs/record_use/test/semantic_equality_test.dart
pkgs/record_use/test/storage_2_test.dart
pkgs/record_use/test/storage_test.dart
pkgs/record_use/test/syntax/uri_pattern_test.dart
pkgs/record_use/test/syntax/validation_test.dart
pkgs/record_use/test/test_data.dart
pkgs/record_use/test/to_string_test.dart
pkgs/record_use/test_data/drop_data_asset/bin/drop_data_asset_calls.dart
pkgs/record_use/test_data/drop_data_asset/bin/drop_data_asset_instances.dart
pkgs/record_use/test_data/drop_data_asset/hook/build.dart
pkgs/record_use/test_data/drop_data_asset/hook/link.dart
pkgs/record_use/test_data/drop_data_asset/lib/drop_data_asset.dart
pkgs/record_use/test_data/drop_data_asset/lib/src/drop_data_asset.dart
pkgs/record_use/test_data/drop_dylib_recording/bin/drop_dylib_recording_calls.dart
pkgs/record_use/test_data/drop_dylib_recording/bin/drop_dylib_recording_instances.dart
pkgs/record_use/test_data/drop_dylib_recording/hook/build.dart
pkgs/record_use/test_data/drop_dylib_recording/hook/link.dart
pkgs/record_use/test_data/drop_dylib_recording/lib/drop_dylib_recording.dart
pkgs/record_use/test_data/drop_dylib_recording/lib/src/drop_dylib_recording.dart
pkgs/record_use/test_data/drop_dylib_recording/lib/src/drop_dylib_recording_bindings.dart
pkgs/record_use/test_data/library_uris/bin/my_bin.dart
pkgs/record_use/test_data/library_uris/hook/build.dart
pkgs/record_use/test_data/library_uris/hook/link.dart
pkgs/record_use/test_data/library_uris/lib/library_uris.dart
pkgs/record_use/test_data/library_uris/lib/src/call.dart
pkgs/record_use/test_data/library_uris/lib/src/definition.dart
pkgs/record_use/test_data/library_uris_helper/lib/library_uris_helper.dart
pkgs/record_use/test_data/library_uris_helper/lib/src/helper_call.dart
pkgs/record_use/test_data/library_uris_helper/lib/src/helper_definition.dart
pkgs/record_use/tool/generate_syntax.dart
tool/check_licenses.dart
tool/ci.dart

This check can be disabled by tagging the PR with skip-license-check.

API leaks ✔️

The following packages contain symbols visible in the public API, but not exported by the library. Export these symbols or remove them from your publicly visible API.

Package Leaked API symbol Leaking sources

This check can be disabled by tagging the PR with skip-leaking-check.

@dcharkes dcharkes force-pushed the native-toolchain-c-pairwise-testing branch from 8943a2f to 8ca4cd2 Compare March 3, 2026 12:32
Copy link
Copy Markdown
Contributor

@goderbauer goderbauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, I wonder if we should maybe explore a code-gen approach that writes the testcases to a generated Dart file. Then it's easier to verify when a .values field of an enum changes which test cases are run.

That's an unfortunate drawback of this approach that you can't actually see in the code which combinations are run...

optimizationLevel: OptimizationLevel.o2,
),
];
final configurations = TestConfigurationGenerator(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: let's mention somewhere what kind of testing we are doing with this config. Maybe either name it AllPairsTestConfigurationGenerator or instead of generate call it generateAllPairs?

@dcharkes dcharkes force-pushed the native-toolchain-c-pairwise-testing branch from 8ca4cd2 to da321f5 Compare March 5, 2026 11:25
@dcharkes
Copy link
Copy Markdown
Collaborator Author

dcharkes commented Mar 5, 2026

Okay, I've explored a code-generator approach. But going from runtime-values to valid Dart code requires a lot of fiddling.

As an alternative, I've now explored generating a markdown table as side effect next to the test that generates the list of test cases. I kinda like it. LMWYT @goderbauer @liamappelbe

@dcharkes dcharkes force-pushed the native-toolchain-c-pairwise-testing branch from da321f5 to 227d4a0 Compare March 5, 2026 11:28
@dcharkes dcharkes force-pushed the native-toolchain-c-pairwise-testing branch from 227d4a0 to 36baeba Compare March 5, 2026 11:43
@dcharkes dcharkes requested a review from liamappelbe March 5, 2026 11:46
@dcharkes dcharkes force-pushed the native-toolchain-c-pairwise-testing branch 5 times, most recently from 5c3f346 to 633d2c1 Compare March 6, 2026 09:32
@dcharkes dcharkes force-pushed the native-toolchain-c-pairwise-testing branch 5 times, most recently from fd50ea3 to c7b45cd Compare March 12, 2026 07:52
@dcharkes
Copy link
Copy Markdown
Collaborator Author

@liamappelbe PTAL. What do you think of the markdown tables in dartdoc comments?

(Ignore the test case generator for now, it's vibe coded, I didn't review it yet myself.)

@dcharkes dcharkes force-pushed the native-toolchain-c-pairwise-testing branch 2 times, most recently from 18ccfc8 to 00f68c6 Compare March 12, 2026 13:06
@dcharkes dcharkes marked this pull request as ready for review March 12, 2026 13:32
@dcharkes
Copy link
Copy Markdown
Collaborator Author

@liamappelbe This should be ready for review now.

@dcharkes dcharkes requested a review from goderbauer March 16, 2026 13:02
Comment thread pkgs/test_case_selector/lib/src/selector.dart Outdated
@dcharkes dcharkes force-pushed the native-toolchain-c-pairwise-testing branch 2 times, most recently from d14b200 to 4819b0b Compare March 23, 2026 08:26
…S interactions

[native_toolchain_c] Use pair-wise testing

feat: cover full cross-product for interaction groups in `TestCaseSelector`

- Update `TestCaseSelector` to support N-way interaction groups.
- Fix undefined name `flutterAndroidNdkVersionLowestSupported` in `native_toolchain_c`.
@dcharkes dcharkes force-pushed the native-toolchain-c-pairwise-testing branch from 4819b0b to 4e506e9 Compare March 23, 2026 10:24
Copy link
Copy Markdown
Contributor

@goderbauer goderbauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dcharkes dcharkes merged commit 0187bfd into main Mar 25, 2026
24 checks passed
@dcharkes dcharkes deleted the native-toolchain-c-pairwise-testing branch March 25, 2026 12:27
Hassnaa9 pushed a commit to Hassnaa9/native that referenced this pull request Apr 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants