Validator drops {"AND":[…]} conditions: a correctly-encoded conditional-nullability rule (CapacityReservationStatus-C-004-C) fails unconditionally

[Matt-Cowsert]

Summary

The validator silently discards any rule Condition encoded as a bare {"AND": [...]} object, then enforces the rule unconditionally. Conditional nullability rules become false-positive conformance failures on spec-compliant data.

This is a validator implementation issue, not a rule-definition issue. Proof: CapacityReservationStatus-C-004-C is encoded correctly in the FOCUS requirements model —

"Condition": {"AND": [
  {"CheckFunction": "CheckNotValue", "ColumnName": "CapacityReservationId", "Value": null},
  {"CheckFunction": "CheckValue",    "ColumnName": "ChargeCategory",        "Value": "Usage"}
]}

— i.e. "MUST NOT be null when CapacityReservationId is not null and ChargeCategory is "Usage"." Yet on a usage-only dataset where CapacityReservationId is null (condition not met, rule should not fire), the validator fails every row with the unconditional message CapacityReservationStatus MUST NOT be NULL. A correctly defined rule is being misapplied by the tool.

Affected version

focus_validator/rules/model-1.2.0.1.json (FOCUS 1.2); validator main @ 3e3c907.

Reproduction

1. FOCUS 1.2 CSV, all rows ChargeCategory="Usage", with CapacityReservationId and CapacityReservationStatus empty (a normal storage/usage export).
2. Run the validator against FOCUS 1.2.
3. Observed: CapacityReservationStatus-C-002-C (AND composite) -> failed ("AND failed - failed child rules: [CapacityReservationStatus-C-004-C]"); CapacityReservationStatus-C-004-C -> passed:false, errorMessage: "CapacityReservationStatus MUST NOT be NULL.". The message has no conditional clause — the tell that the condition was dropped.

Behaviour is identical whether empty cells or the literal token NULL are used for empty fields. Reproducible with a minimal synthetic CSV — no proprietary data required.

Root cause

focus_validator/config_objects/focus_to_duckdb_converter.py, _compile_condition_with_generators (~L5646-5648):

fn = spec.get("CheckFunction")
if not fn:
    return None

The composite branches read children from spec.get("Items"). But the model encodes conditions as {"AND": [ ... ]} — no CheckFunction key, no Items key. The function returns None before reaching the AND branch, so the condition is dropped, _apply_condition (~L386-394) becomes a no-op, and CheckNotValueGenerator.generateSql() (~L1206; value None + keyword "MUST NOT", ~L1215-1218) emits a bare <col> IS NULL violation predicate with message <Col> MUST NOT be NULL. (built from column + keyword only, hence no conditional text). The failure is silent: no log, no skip, no error.

Impact

False-positive conformance failures for any dataset without capacity reservations / commitment discounts — the common usage-only case, and typically a data generator's first submission, even though the data is fully compliant.

Suggested fix

• In _compile_condition_with_generators, handle the bare {"AND":[...]} / {"OR":[...]} (and single-child {"AND": {...}}) shapes in addition to {"CheckFunction":"AND","Items":[...]}; do not early-return on a missing CheckFunction when a composite key is present.
• When a non-empty Condition fails to compile, fail loudly or mark the rule errored/skipped — never silently enforce it unconditionally.
• Include the conditional clause in the violation message.

Related

A separate, independent rule-definition defect in the FOCUS requirements model (CommitmentDiscountStatus-C-004-C, self-contradictory with C-003-C) is tracked at FinOps-Open-Cost-and-Usage-Spec/FOCUS_Spec#2393. Working-group umbrella with the full validator-vs-rule triage matrix is at FinOps-Open-Cost-and-Usage-Spec/FOCUS_Spec#2394. Even after the rule fix, this validator fix is required for any {"AND":[…]} condition to be honored.