Formalise the degrees of freedom required to uniquely specify a reproducible compliance assessment

[ramiromagno]

To support verifiable, reproducible, and auditable compliance claims in complyr, we propose defining a formal framework that enumerates the degrees of freedom (DoF) necessary to uniquely and deterministically specify a compliance scenario.

This conceptual framework will underpin provenance tracking, reproducibility guarantees, and long-term traceability of claims across packages, versions, and test infrastructures.

Objectives

1. Define a canonical specification of a compliance scenario as a composable, versioned tuple of inputs and conditions.
2. Support deterministic reproduction of any compliance evaluation, including historic and cross-version scenarios.
3. Provide a theoretical basis for compliance caching, reporting, comparison, and ledgering.

Conceptual Elements

A reproducible compliance scenario in complyr should minimally be defined by the following core components:

1. Subject under test: Package name and version, target function or object, declared compliance claim(s) (e.g., via @comply),

2. Auditor definition: auditor package name and version and specific test function(s) executed

3. Execution environment: R version, platform and architecture, system dependencies, locales, and test helper versions (if applicable)

4. Evaluation context: timestamp of execution, CI/CD metadata or local session UUID, Git commit SHA (for method and/or auditor)

Towards a Compliance Signature

We propose the introduction of a compliance signature, defined as a hashable data structure that captures all relevant DoF for a single test evaluation. This structure would support:

1. Reproducibility: Ensuring that the same inputs yield identical results
2. Comparability: Supporting diff-based regression and progress analysis
3. Provenance tracking: Enabling transparent audit trails over time

Example row (conceptual):

method_pkg	version	auditor_pkg	version	test_fn	R_version	platform	result	timestamp
mypkg	1.2.0	auditorABC	0.3.1	test_conformX	4.3.2	x86_64	pass	2025-05-24T14:03:00Z