math bruteforce is not testing edge case behavior

[bashbaug]

Both the OpenCL C spec and the OpenCL SPIR-V environment spec describe required edge case behavior for certain math functions. These sections also reference the C99 specifications, which describe required edge case behavior for other math functions:

• https://registry.khronos.org/OpenCL/specs/3.0-unified/html/OpenCL_C.html#edge-case-behavior
• https://registry.khronos.org/OpenCL/specs/3.0-unified/html/OpenCL_Env.html#_edge_case_behavior

Note that these sections both say: "Those values shall produce exactly the prescribed answers, and no other."

As far as I can tell, though, these edge case behaviors are not tested for in the math bruteforce tests or elsewhere. Instead, for these inputs, the usual tests for math accuracy apply. Do we need dedicated tests for these edge case values?

[shajder]

@bashbaug at the moment many of corner cases are tested through specialValues arrays used here and there in math_brute_force. Sadly, in most (most likely all) cases verification relies on ULP values for specific functions which is incorrect in the context of corner cases (the restult must be exact).

Honestly, I don't want to touch that code. It's a multi-threaded monster and adding an exact-match path in there would make it even harder to read. Much cleaner to just add a new separate file in the math_brute_force module with its own logic. Reviewers will also have an easier time since there's nothing to untangle from existing logic.

Plan is to build a list of templated structs for each corner case, holding the function name, a tuple of inputs, and the expected result, then iterate over them, dynamically building a single-workitem kernel per case and doing a bitwise comparison against the pre-set expected value. A wrong type in a test case is a compile error, not a runtime surprise. How does that sound for you?

[bashbaug]

Linking several other semi-related issues:

• The atomics suite doesn't meaningfully test min/max unsigned cases #10
• Conformance tests should include specific, representative inputs #34
• cl_khr_fp16 is poorly tested #142
• Missing cases in testing for shuffle and shuffle2 #1392

To be clear, these issues should be solved by separate PRs, I am only linking them because they are related.

[bashbaug]

Much cleaner to just add a new separate file in the math_brute_force module with its own logic. Reviewers will also have an easier time since there's nothing to untangle from existing logic.

Plan is to build a list of templated structs for each corner case, holding the function name, a tuple of inputs, and the expected result, then iterate over them, dynamically building a single-workitem kernel per case and doing a bitwise comparison against the pre-set expected value. A wrong type in a test case is a compile error, not a runtime surprise. How does that sound for you?

I think this makes sense. I agree it is not a good idea to check for these edge case values through the existing test logic - in addition to the engineering and review complexity, I also suspect it could slow down test execution, and these tests already take a long time to execute.

Could you maybe pick two functions to test using this logic so we can settle on the approach? Then, once that's approved, it will (hopefully) be easy to apply the same logic to the remaining functions.

[shajder]

Could you maybe pick two functions to test using this logic so we can settle on the approach? Then, once that's approved, it will (hopefully) be easy to apply the same logic to the remaining functions.

the test is already implemented in this PR - as far as I can see it is not terribly slow, at least in comparison to other bruteforce tests. That said, it would certainly be possible to refactor the logic to batch edge cases into a single kernel execution if needed. On my old i915 Intel with emulated FP64 the whole test runs in a few seconds.