diff --git a/.claude/skills/measure-validation.md b/.claude/skills/measure-validation.md new file mode 100644 index 000000000..be8b4892d --- /dev/null +++ b/.claude/skills/measure-validation.md @@ -0,0 +1,186 @@ +# Measure Validation Architecture + +## Overview + +Measure validation ensures that prerequisites for CQL evaluation are met **before** evaluation begins, avoiding wasted compute and providing structured, actionable error feedback. Validation runs in two layers: + +1. **First-pass structural validation** (`R4MeasureDefBuilder.triggerFirstPassValidation`) — existing checks that throw `InvalidRequestException` for structural problems in the FHIR Measure resource itself. +2. **Pre-evaluation validation** (`CompositeMeasureDefValidator`) — a composable validator framework that probes the repository for library resolution, ValueSet availability, parameter configuration, and expression reference integrity. + +Both layers execute before CQL evaluation in `R4MeasureProcessor`. + +## Pipeline Position + +``` +R4MeasureProcessor.evaluateMultiMeasuresWithCqlEngine(): + + checkMeasureLibrary(measure) ← Quick library presence check + R4MeasureDefBuilder.triggerFirstPassValidation(measures) ← Structural checks (throws) + ──── NEW VALIDATION LAYER ──── + for each measure: + MeasureDef = R4MeasureDefBuilder.build(measure) + CompositeMeasureDefValidator.validate(context) ← Repository-probing checks + ──── END VALIDATION ──── + getMultiLibraryIdMeasureEngineDetails(measures) ← Full library resolution + resolveParameterMap(parameters) + MeasureEvaluationResultHandler.getEvaluationResults() ← CQL evaluation +``` + +## Validation Result Model (domain core) + +All classes in `org.opencds.cqf.fhir.cr.measure.common`: + +**`ValidationSeverity`** — Enum: `ERROR`, `WARNING`, `INFO` + +**`ValidationIssue`** — Record carrying a single issue: +- `severity` — blocking errors vs informational warnings +- `code` — machine-readable identifier (e.g. `"LIBRARY_NOT_FOUND"`) +- `description` — human-readable problem statement +- `remediation` — actionable fix guidance +- `location` — optional path within the Measure (e.g. `"Measure.library"`) + +**`ValidationResult`** — Mutable accumulator for issues: +- `addIssue(ValidationIssue)`, `merge(ValidationResult)` +- `hasErrors()`, `hasWarnings()`, `isEmpty()` +- `getBlockingErrors()` — filters for ERROR severity + +**`MeasureValidationException`** — `extends RuntimeException`, thrown when validation produces blocking errors. Carries the full `ValidationResult` for programmatic inspection via `getValidationResult()`. + +## Validator Interface + +**`MeasureDefValidator`** — Strategy interface: +```java +public interface MeasureDefValidator { + ValidationResult validate(MeasureDefValidationContext context); +} +``` + +**`MeasureDefValidationContext`** — Record providing: +- `measureDef()` — the built domain MeasureDef +- `measure()` — the raw FHIR Measure resource (`IBaseResource`) +- `repository()` — `IRepository` for probing resource availability +- `parameters()` — user-supplied parameters map (empty if none) + +**`CompositeMeasureDefValidator`** — Runs an ordered list of validators, merges all results into one `ValidationResult`. All validators execute regardless of earlier failures. + +## Individual Validators + +### R4CqlLibraryValidator (`r4/`) +- **Checks**: Library canonical URLs resolve in the repository +- **Follows**: Transitive `relatedArtifact` (type=depends-on) dependencies +- **Produces**: `LIBRARY_NOT_FOUND` (ERROR) +- **How**: `repository.search(Bundle.class, Library.class, Searches.byCanonical(url), null)` + +### R4ValueSetAvailabilityValidator (`r4/`) +- **Checks**: ValueSets referenced in Library `dataRequirement.codeFilter.valueSet` exist +- **Produces**: `VALUESET_UNAVAILABLE` (WARNING) — external terminology services may resolve at runtime +- **Does NOT** trigger expansion; existence check only + +### R4ParameterConfigurationValidator (`r4/`, extends `ParameterConfigurationValidator`) +- **Checks**: Required Library parameters (min > 0) are present; flags unknown parameters +- **Reads**: `Library.parameter` definitions (name, type, use, min) +- **Produces**: `MISSING_REQUIRED_PARAMETER` (ERROR), `UNKNOWN_PARAMETER` (WARNING) +- **Skips**: Well-known operation parameters like `"Measurement Period"` + +### R4ExpressionReferenceValidator (`r4/`) +- **Checks**: CQL expression names in populations, stratifiers, SDEs exist in the primary library +- **Parses**: Library ELM JSON content (`application/elm+json`) via Jackson, falls back to CQL text +- **Produces**: `EXPRESSION_NOT_FOUND` (WARNING) — expressions may exist in included libraries +- **ELM path**: `library.statements.def[].name` + +## First-Pass Structural Validation (existing) + +`R4MeasureDefBuilder.triggerFirstPassValidation(List)` checks: + +| Check | Method | Error | +|---|---|---| +| Measure has ID | `checkId(measure)` | `InvalidRequestException` | +| Population IDs unique per group | `validateUniquePopulationIds()` | `InvalidRequestException` | +| SDE usage codes present | `checkSDEUsage()` | `InvalidRequestException` | +| Improvement notation valid | `validateMeasureImprovementNotation()` | `InvalidRequestException` | +| Ratio CV structure (2 observations, criteria refs) | `validateRatioContinuousVariableIfApplicable()` | `InvalidRequestException` | + +Additional checks during `R4MeasureDefBuilder.build()`: +- All Elements have IDs (`checkId()`) +- Stratifiers have either criteria OR components, not both +- Criteria references for MEASUREOBSERVATION populations resolve to group population IDs +- Population basis and improvement notation coalescing (group-level overrides measure-level) + +## OperationOutcome Integration + +`R4MeasureReportBuilderContext` surfaces validation issues as contained `OperationOutcome` resources: + +**Existing**: `addOperationOutcomes()` — converts `MeasureDef.errors()` (runtime evaluation errors) to `OperationOutcome` with `IssueSeverity.ERROR` and `IssueType.EXCEPTION`. + +**New**: `addValidationOutcomes(ValidationResult)` — converts `ValidationIssue` objects with: + +| ValidationSeverity | OperationOutcome.IssueSeverity | +|---|---| +| ERROR | ERROR | +| WARNING | WARNING | +| INFO | INFORMATION | + +| Validation Code | IssueType | +|---|---| +| `LIBRARY_NOT_FOUND` | NOTFOUND | +| `VALUESET_UNAVAILABLE` | NOTFOUND | +| `EXPRESSION_NOT_FOUND` | NOTFOUND | +| `MISSING_REQUIRED_PARAMETER` | REQUIRED | +| `UNKNOWN_PARAMETER` | VALUE | + +Remediation text goes in `issue.diagnostics`. Error code goes in `issue.details.coding` with system `http://opencds.org/fhir/measure-validation`. + +## Key Files + +### Domain Core (`cqf-fhir-cr/.../measure/common/`) +| File | Type | +|---|---| +| `ValidationSeverity.java` | Enum | +| `ValidationIssue.java` | Record | +| `ValidationResult.java` | Class (mutable accumulator) | +| `MeasureDefValidator.java` | Interface | +| `MeasureDefValidationContext.java` | Record | +| `CompositeMeasureDefValidator.java` | Class | +| `ParameterConfigurationValidator.java` | Base class | +| `MeasureValidationException.java` | Exception | + +### R4 Validators (`cqf-fhir-cr/.../measure/r4/`) +| File | Codes Produced | +|---|---| +| `R4CqlLibraryValidator.java` | `LIBRARY_NOT_FOUND` | +| `R4ValueSetAvailabilityValidator.java` | `VALUESET_UNAVAILABLE` | +| `R4ParameterConfigurationValidator.java` | `MISSING_REQUIRED_PARAMETER`, `UNKNOWN_PARAMETER` | +| `R4ExpressionReferenceValidator.java` | `EXPRESSION_NOT_FOUND` | + +### Integration Points +| File | Role | +|---|---| +| `R4MeasureProcessor.java` | Constructs composite validator, invokes `runPreEvaluationValidation()` | +| `R4MeasureReportBuilderContext.java` | `addValidationOutcomes()` for OperationOutcome surfacing | +| `R4MeasureDefBuilder.java` | `triggerFirstPassValidation()` for structural checks | + +### Tests +| File | Coverage | +|---|---| +| `MeasureDefValidatorTest.java` | Unit tests for all validators, composite, ValidationResult | +| `InvalidMeasureTest.java` | Integration test: `evaluateThrowsErrorWhenLibraryUnavailable()` | + +## Adding a New Validator + +1. Create a class implementing `MeasureDefValidator` in `r4/` (or `common/` if version-agnostic) +2. Define a `public static final String` error code constant +3. Implement `validate(MeasureDefValidationContext)` — probe the repository, return `ValidationResult` +4. Add the validator to the `CompositeMeasureDefValidator` list in `R4MeasureProcessor` constructor +5. Add the error code to `R4MeasureReportBuilderContext.mapIssueType()` switch +6. Add unit tests in `MeasureDefValidatorTest` + +## Design Decisions + +| Decision | Choice | Rationale | +|---|---|---| +| Separate from `triggerFirstPassValidation` | New `MeasureDefValidator` interface | Clean separation; existing method is static, throws directly | +| Structured model vs raw strings | `ValidationIssue` record | Jira DQM-570 requires error code, description, remediation | +| Always-on | No opt-in flag | Validation checks are cheap repository lookups | +| ValueSet not found = WARNING | Not ERROR | External terminology services may resolve at runtime | +| Expression not found = WARNING | Not ERROR | Expressions may exist in included libraries | +| Custom exception | `MeasureValidationException` | Avoids coupling domain code to HAPI `InvalidRequestException` | diff --git a/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/CompositeMeasureDefValidator.java b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/CompositeMeasureDefValidator.java new file mode 100644 index 000000000..34284b675 --- /dev/null +++ b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/CompositeMeasureDefValidator.java @@ -0,0 +1,26 @@ +package org.opencds.cqf.fhir.cr.measure.common; + +import java.util.List; + +/** + * Runs an ordered list of {@link MeasureDefValidator} implementations and merges their results + * into a single {@link ValidationResult}. All validators are executed regardless of earlier failures, + * so the caller receives the complete set of issues in one pass. + */ +public class CompositeMeasureDefValidator implements MeasureDefValidator { + + private final List validators; + + public CompositeMeasureDefValidator(List validators) { + this.validators = List.copyOf(validators); + } + + @Override + public ValidationResult validate(MeasureDefValidationContext context) { + var result = new ValidationResult(); + for (var validator : validators) { + result.merge(validator.validate(context)); + } + return result; + } +} diff --git a/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/MeasureDefValidationContext.java b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/MeasureDefValidationContext.java new file mode 100644 index 000000000..778907647 --- /dev/null +++ b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/MeasureDefValidationContext.java @@ -0,0 +1,30 @@ +package org.opencds.cqf.fhir.cr.measure.common; + +import ca.uhn.fhir.repository.IRepository; +import jakarta.annotation.Nullable; +import java.util.Map; +import org.hl7.fhir.instance.model.api.IBaseResource; + +/** + * Immutable context provided to {@link MeasureDefValidator} implementations during pre-evaluation + * validation. Bundles together the domain-level {@link MeasureDef}, the raw FHIR Measure resource, + * the {@link IRepository} for probing resource availability, and any user-supplied parameters. + */ +public record MeasureDefValidationContext( + MeasureDef measureDef, IBaseResource measure, IRepository repository, Map parameters) { + + public MeasureDefValidationContext( + MeasureDef measureDef, + IBaseResource measure, + IRepository repository, + @Nullable Map parameters) { + this.measureDef = measureDef; + this.measure = measure; + this.repository = repository; + this.parameters = parameters != null ? parameters : Map.of(); + } + + public MeasureDefValidationContext(MeasureDef measureDef, IBaseResource measure, IRepository repository) { + this(measureDef, measure, repository, null); + } +} diff --git a/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/MeasureDefValidator.java b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/MeasureDefValidator.java new file mode 100644 index 000000000..047b01594 --- /dev/null +++ b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/MeasureDefValidator.java @@ -0,0 +1,10 @@ +package org.opencds.cqf.fhir.cr.measure.common; + +/** + * Strategy interface for validating a {@link MeasureDef} before CQL evaluation begins. + * Implementations check specific prerequisites (e.g. library resolution, ValueSet availability) + * and return a {@link ValidationResult} containing any issues found. + */ +public interface MeasureDefValidator { + ValidationResult validate(MeasureDefValidationContext context); +} diff --git a/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/MeasureValidationException.java b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/MeasureValidationException.java new file mode 100644 index 000000000..2e69a116a --- /dev/null +++ b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/MeasureValidationException.java @@ -0,0 +1,30 @@ +package org.opencds.cqf.fhir.cr.measure.common; + +import java.util.List; + +/** + * Thrown when pre-evaluation validation of a Measure fails with blocking errors. + * Contains the structured {@link ValidationResult} so callers can inspect individual issues. + */ +public class MeasureValidationException extends RuntimeException { + + private final ValidationResult validationResult; + + public MeasureValidationException(String measureUrl, ValidationResult validationResult) { + super(formatMessage(measureUrl, validationResult)); + this.validationResult = validationResult; + } + + public ValidationResult getValidationResult() { + return validationResult; + } + + private static String formatMessage(String measureUrl, ValidationResult validationResult) { + List errors = validationResult.getBlockingErrors(); + var errorMessages = errors.stream() + .map(issue -> "[%s] %s".formatted(issue.code(), issue.description())) + .toList(); + return "Measure validation failed for '%s' with %d error(s):\n%s" + .formatted(measureUrl, errors.size(), String.join("\n", errorMessages)); + } +} diff --git a/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/ParameterConfigurationValidator.java b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/ParameterConfigurationValidator.java new file mode 100644 index 000000000..09092de5f --- /dev/null +++ b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/ParameterConfigurationValidator.java @@ -0,0 +1,37 @@ +package org.opencds.cqf.fhir.cr.measure.common; + +import ca.uhn.fhir.repository.IRepository; +import java.util.Map; +import org.hl7.fhir.instance.model.api.IBaseResource; + +/** + * Version-agnostic base class for validating that user-supplied parameters match the CQL library's + * parameter definitions. Subclasses (e.g. {@code R4ParameterConfigurationValidator}) override + * {@link #validateLibraryParameters} to perform FHIR-version-specific checks. + */ +public class ParameterConfigurationValidator implements MeasureDefValidator { + + public static final String MISSING_REQUIRED_PARAMETER = "MISSING_REQUIRED_PARAMETER"; + public static final String UNKNOWN_PARAMETER = "UNKNOWN_PARAMETER"; + + @Override + public ValidationResult validate(MeasureDefValidationContext context) { + var result = new ValidationResult(); + + // Parameter validation requires version-specific Library access to read + // Library.parameter definitions. This base implementation validates that + // required operation-level parameters (like measurement period) are present. + // Version-specific subclasses can override to add Library parameter validation. + + return result; + } + + /** + * Subclasses should override this to extract parameter definitions from the Library resource + * and validate against the provided parameters map. + */ + protected void validateLibraryParameters( + IBaseResource library, Map parameters, IRepository repository, ValidationResult result) { + // Default no-op; version-specific implementations provide this + } +} diff --git a/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/ValidationIssue.java b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/ValidationIssue.java new file mode 100644 index 000000000..ebb565775 --- /dev/null +++ b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/ValidationIssue.java @@ -0,0 +1,34 @@ +package org.opencds.cqf.fhir.cr.measure.common; + +import jakarta.annotation.Nullable; + +/** + * A single validation issue discovered during pre-evaluation Measure validation. + * Each issue carries a machine-readable {@code code}, a human-readable {@code description}, + * actionable {@code remediation} guidance, and an optional {@code location} within the Measure resource. + * + * @param severity the severity level of this issue + * @param code machine-readable error code (e.g. {@code "LIBRARY_NOT_FOUND"}) + * @param description human-readable description of the problem + * @param remediation actionable guidance on how to resolve the issue + * @param location optional path within the Measure resource (e.g. {@code "Measure.library"}) + */ +public record ValidationIssue( + ValidationSeverity severity, + String code, + String description, + String remediation, + @Nullable String location) { + + public ValidationIssue(ValidationSeverity severity, String code, String description, String remediation) { + this(severity, code, description, remediation, null); + } + + public boolean isError() { + return severity == ValidationSeverity.ERROR; + } + + public boolean isWarning() { + return severity == ValidationSeverity.WARNING; + } +} diff --git a/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/ValidationResult.java b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/ValidationResult.java new file mode 100644 index 000000000..122db82ab --- /dev/null +++ b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/ValidationResult.java @@ -0,0 +1,42 @@ +package org.opencds.cqf.fhir.cr.measure.common; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; + +/** + * Accumulates {@link ValidationIssue} instances produced during pre-evaluation Measure validation. + * Provides convenience methods to query for blocking errors and warnings. + */ +public class ValidationResult { + + private final List issues = new ArrayList<>(); + + public void addIssue(ValidationIssue issue) { + issues.add(issue); + } + + public void merge(ValidationResult other) { + issues.addAll(other.issues); + } + + public List getIssues() { + return Collections.unmodifiableList(issues); + } + + public boolean hasErrors() { + return issues.stream().anyMatch(ValidationIssue::isError); + } + + public boolean hasWarnings() { + return issues.stream().anyMatch(ValidationIssue::isWarning); + } + + public List getBlockingErrors() { + return issues.stream().filter(ValidationIssue::isError).toList(); + } + + public boolean isEmpty() { + return issues.isEmpty(); + } +} diff --git a/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/ValidationSeverity.java b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/ValidationSeverity.java new file mode 100644 index 000000000..540153582 --- /dev/null +++ b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/common/ValidationSeverity.java @@ -0,0 +1,11 @@ +package org.opencds.cqf.fhir.cr.measure.common; + +/** + * Severity levels for pre-evaluation validation issues found during Measure validation. + * Maps to FHIR {@code OperationOutcome.IssueSeverity} when surfaced in a MeasureReport. + */ +public enum ValidationSeverity { + ERROR, + WARNING, + INFO +} diff --git a/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/r4/R4CqlLibraryValidator.java b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/r4/R4CqlLibraryValidator.java new file mode 100644 index 000000000..ff94c6929 --- /dev/null +++ b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/r4/R4CqlLibraryValidator.java @@ -0,0 +1,85 @@ +package org.opencds.cqf.fhir.cr.measure.r4; + +import java.util.HashSet; +import java.util.Set; +import org.hl7.fhir.r4.model.Bundle; +import org.hl7.fhir.r4.model.Library; +import org.hl7.fhir.r4.model.Measure; +import org.hl7.fhir.r4.model.RelatedArtifact; +import org.opencds.cqf.fhir.cr.measure.common.MeasureDefValidationContext; +import org.opencds.cqf.fhir.cr.measure.common.MeasureDefValidator; +import org.opencds.cqf.fhir.cr.measure.common.ValidationIssue; +import org.opencds.cqf.fhir.cr.measure.common.ValidationResult; +import org.opencds.cqf.fhir.cr.measure.common.ValidationSeverity; +import org.opencds.cqf.fhir.utility.search.Searches; + +/** + * Validates that the CQL libraries referenced by a Measure are resolvable in the repository. + * Checks the primary library and follows transitive {@code relatedArtifact} dependencies of type + * {@code depends-on}. Produces {@code LIBRARY_NOT_FOUND} errors for any missing library. + */ +public class R4CqlLibraryValidator implements MeasureDefValidator { + + public static final String LIBRARY_NOT_FOUND = "LIBRARY_NOT_FOUND"; + + @Override + public ValidationResult validate(MeasureDefValidationContext context) { + var result = new ValidationResult(); + var measure = (Measure) context.measure(); + + if (!measure.hasLibrary() || measure.getLibrary().isEmpty()) { + result.addIssue(new ValidationIssue( + ValidationSeverity.ERROR, + LIBRARY_NOT_FOUND, + "Measure '%s' does not have a primary library specified".formatted(measure.getUrl()), + "Add a library reference to the Measure resource using the 'library' element.", + "Measure.library")); + return result; + } + + var checked = new HashSet(); + for (var libraryCanonical : measure.getLibrary()) { + var url = libraryCanonical.asStringValue(); + validateLibraryExists(url, context, result, checked); + } + + return result; + } + + private void validateLibraryExists( + String canonicalUrl, MeasureDefValidationContext context, ValidationResult result, Set checked) { + + if (!checked.add(canonicalUrl)) { + return; + } + + var bundle = context.repository().search(Bundle.class, Library.class, Searches.byCanonical(canonicalUrl), null); + + if (bundle.getEntry().isEmpty()) { + result.addIssue(new ValidationIssue( + ValidationSeverity.ERROR, + LIBRARY_NOT_FOUND, + "Library with canonical URL '%s' was not found in the repository".formatted(canonicalUrl), + "Ensure the Library resource with URL '%s' is loaded in the repository.".formatted(canonicalUrl), + "Measure.library")); + return; + } + + // Check transitive dependencies via relatedArtifact + var library = (Library) bundle.getEntryFirstRep().getResource(); + for (var relatedArtifact : library.getRelatedArtifact()) { + if (relatedArtifact.getType() == RelatedArtifact.RelatedArtifactType.DEPENDSON + && relatedArtifact.hasResource() + && relatedArtifact.getResource().startsWith("Library/") + || isLibraryCanonical(relatedArtifact)) { + validateLibraryExists(relatedArtifact.getResource(), context, result, checked); + } + } + } + + private static boolean isLibraryCanonical(RelatedArtifact relatedArtifact) { + return relatedArtifact.getType() == RelatedArtifact.RelatedArtifactType.DEPENDSON + && relatedArtifact.hasResource() + && !relatedArtifact.getResource().startsWith("http://hl7.org/fhir/Library/FHIR-ModelInfo"); + } +} diff --git a/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/r4/R4ExpressionReferenceValidator.java b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/r4/R4ExpressionReferenceValidator.java new file mode 100644 index 000000000..018b5bf64 --- /dev/null +++ b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/r4/R4ExpressionReferenceValidator.java @@ -0,0 +1,215 @@ +package org.opencds.cqf.fhir.cr.measure.r4; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import java.util.HashSet; +import java.util.Set; +import org.hl7.fhir.r4.model.Attachment; +import org.hl7.fhir.r4.model.Bundle; +import org.hl7.fhir.r4.model.Library; +import org.hl7.fhir.r4.model.Measure; +import org.opencds.cqf.fhir.cr.measure.common.GroupDef; +import org.opencds.cqf.fhir.cr.measure.common.MeasureDefValidationContext; +import org.opencds.cqf.fhir.cr.measure.common.MeasureDefValidator; +import org.opencds.cqf.fhir.cr.measure.common.PopulationDef; +import org.opencds.cqf.fhir.cr.measure.common.SdeDef; +import org.opencds.cqf.fhir.cr.measure.common.StratifierDef; +import org.opencds.cqf.fhir.cr.measure.common.ValidationIssue; +import org.opencds.cqf.fhir.cr.measure.common.ValidationResult; +import org.opencds.cqf.fhir.cr.measure.common.ValidationSeverity; +import org.opencds.cqf.fhir.utility.search.Searches; + +/** + * Validates that CQL expression names referenced in the Measure (population criteria, stratifiers, + * supplemental data elements) exist in the primary CQL library. Parses the library's ELM JSON + * content to extract top-level statement names. Produces {@code EXPRESSION_NOT_FOUND} warnings + * since expressions may exist in included libraries not checked here. + */ +public class R4ExpressionReferenceValidator implements MeasureDefValidator { + + public static final String EXPRESSION_NOT_FOUND = "EXPRESSION_NOT_FOUND"; + private static final String ELM_JSON_CONTENT_TYPE = "application/elm+json"; + private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper(); + + @Override + public ValidationResult validate(MeasureDefValidationContext context) { + var result = new ValidationResult(); + var measure = (Measure) context.measure(); + + if (!measure.hasLibrary() || measure.getLibrary().isEmpty()) { + return result; + } + + var url = measure.getLibrary().get(0).asStringValue(); + var expressionNames = extractExpressionNames(url, context); + + if (expressionNames == null) { + // Could not parse library content; skip expression validation + return result; + } + + var measureDef = context.measureDef(); + + // Validate population expressions + for (int gi = 0; gi < measureDef.groups().size(); gi++) { + GroupDef group = measureDef.groups().get(gi); + + for (PopulationDef pop : group.populations()) { + validateExpression( + pop.expression(), + expressionNames, + "Measure.group[%d].population '%s'".formatted(gi, pop.id()), + result); + } + + // Validate stratifier expressions + for (StratifierDef strat : group.stratifiers()) { + if (strat.expression() != null) { + validateExpression( + strat.expression(), + expressionNames, + "Measure.group[%d].stratifier '%s'".formatted(gi, strat.id()), + result); + } + for (var comp : strat.components()) { + if (comp.expression() != null) { + validateExpression( + comp.expression(), + expressionNames, + "Measure.group[%d].stratifier '%s' component '%s'".formatted(gi, strat.id(), comp.id()), + result); + } + } + } + } + + // Validate SDE expressions + for (SdeDef sde : measureDef.sdes()) { + validateExpression( + sde.expression(), expressionNames, "Measure.supplementalData '%s'".formatted(sde.id()), result); + } + + return result; + } + + private void validateExpression( + String expression, Set expressionNames, String location, ValidationResult result) { + if (expression == null || expression.isBlank()) { + return; + } + if (!expressionNames.contains(expression)) { + result.addIssue(new ValidationIssue( + ValidationSeverity.WARNING, + EXPRESSION_NOT_FOUND, + "Expression '%s' referenced in %s was not found in the primary CQL library" + .formatted(expression, location), + "Check that the expression name '%s' is defined in the CQL library and matches exactly (case-sensitive). " + .formatted(expression) + + "The expression may exist in an included library.", + location)); + } + } + + private Set extractExpressionNames(String libraryUrl, MeasureDefValidationContext context) { + var bundle = context.repository().search(Bundle.class, Library.class, Searches.byCanonical(libraryUrl), null); + + if (bundle.getEntry().isEmpty()) { + return null; + } + + var library = (Library) bundle.getEntryFirstRep().getResource(); + + // Try to extract expression names from ELM JSON content + for (Attachment content : library.getContent()) { + if (ELM_JSON_CONTENT_TYPE.equals(content.getContentType()) && content.hasData()) { + return parseElmExpressionNames(content.getData()); + } + } + + // If no ELM content, try extracting from CQL content + for (Attachment content : library.getContent()) { + if ("text/cql".equals(content.getContentType()) && content.hasData()) { + return parseCqlExpressionNames(content.getData()); + } + } + + return null; + } + + private Set parseElmExpressionNames(byte[] elmData) { + try { + var root = OBJECT_MAPPER.readTree(elmData); + var names = new HashSet(); + + // Navigate: library.statements.def[] -> extract "name" from each top-level definition + var library = root.get("library"); + if (library == null) { + return null; + } + + var statements = library.get("statements"); + if (statements == null) { + return null; + } + + var def = statements.get("def"); + if (def == null || !def.isArray()) { + return null; + } + + for (JsonNode statement : def) { + var nameNode = statement.get("name"); + if (nameNode != null && nameNode.isTextual()) { + names.add(nameNode.asText()); + } + } + + return names.isEmpty() ? null : names; + } catch (Exception e) { + return null; + } + } + + private Set parseCqlExpressionNames(byte[] cqlData) { + try { + var cql = new String(cqlData); + var names = new HashSet(); + + var lines = cql.split("\n"); + for (var line : lines) { + var trimmed = line.trim(); + if (trimmed.startsWith("define ") || trimmed.startsWith("define\t")) { + var afterDefine = trimmed.substring(7).trim(); + // Remove optional "function" keyword + if (afterDefine.startsWith("function ")) { + afterDefine = afterDefine.substring(9).trim(); + } + // Handle quoted name + if (afterDefine.startsWith("\"")) { + int endQuote = afterDefine.indexOf('"', 1); + if (endQuote > 0) { + names.add(afterDefine.substring(1, endQuote)); + } + } else { + // Unquoted name ends at colon, paren, or whitespace + var nameEnd = afterDefine.length(); + for (int i = 0; i < afterDefine.length(); i++) { + char c = afterDefine.charAt(i); + if (c == ':' || c == '(' || Character.isWhitespace(c)) { + nameEnd = i; + break; + } + } + if (nameEnd > 0) { + names.add(afterDefine.substring(0, nameEnd)); + } + } + } + } + + return names.isEmpty() ? null : names; + } catch (Exception e) { + return null; + } + } +} diff --git a/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/r4/R4MeasureProcessor.java b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/r4/R4MeasureProcessor.java index 095d3dfaa..62f0de6b1 100644 --- a/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/r4/R4MeasureProcessor.java +++ b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/r4/R4MeasureProcessor.java @@ -33,11 +33,15 @@ import org.opencds.cqf.fhir.cql.VersionedIdentifiers; import org.opencds.cqf.fhir.cr.measure.MeasureEvaluationOptions; import org.opencds.cqf.fhir.cr.measure.common.CompositeEvaluationResultsPerMeasure; +import org.opencds.cqf.fhir.cr.measure.common.CompositeMeasureDefValidator; +import org.opencds.cqf.fhir.cr.measure.common.MeasureDefValidationContext; +import org.opencds.cqf.fhir.cr.measure.common.MeasureDefValidator; import org.opencds.cqf.fhir.cr.measure.common.MeasureEvalType; import org.opencds.cqf.fhir.cr.measure.common.MeasureEvaluationResultHandler; import org.opencds.cqf.fhir.cr.measure.common.MeasureProcessorTimeUtils; import org.opencds.cqf.fhir.cr.measure.common.MeasureReference; import org.opencds.cqf.fhir.cr.measure.common.MeasureReportType; +import org.opencds.cqf.fhir.cr.measure.common.MeasureValidationException; import org.opencds.cqf.fhir.cr.measure.common.MultiLibraryIdMeasureEngineDetails; import org.opencds.cqf.fhir.cr.measure.r4.utils.R4DateHelper; import org.opencds.cqf.fhir.cr.measure.r4.utils.R4MeasureServiceUtils; @@ -52,6 +56,7 @@ public class R4MeasureProcessor { private final MeasureEvaluationOptions measureEvaluationOptions; private final FhirContext fhirContext = FhirContext.forR4Cached(); private final MeasureEvaluationResultHandler measureEvaluationResultHandler; + private final MeasureDefValidator preEvaluationValidator; public R4MeasureProcessor(IRepository repository, MeasureEvaluationOptions measureEvaluationOptions) { @@ -60,6 +65,11 @@ public R4MeasureProcessor(IRepository repository, MeasureEvaluationOptions measu measureEvaluationOptions != null ? measureEvaluationOptions : MeasureEvaluationOptions.defaultOptions(); this.measureEvaluationResultHandler = new MeasureEvaluationResultHandler(this.measureEvaluationOptions, new R4PopulationBasisValidator()); + this.preEvaluationValidator = new CompositeMeasureDefValidator(List.of( + new R4CqlLibraryValidator(), + new R4ValueSetAvailabilityValidator(), + new R4ParameterConfigurationValidator(), + new R4ExpressionReferenceValidator())); } // Expose this so CQL measure evaluation can use the same Repository as the one passed to the @@ -179,6 +189,9 @@ MeasureDefAndR4MeasureReport evaluateMeasureCaptureDef( // setup MeasureDef var measureDef = new R4MeasureDefBuilder().build(measure); + // Run pre-evaluation validation + runPreEvaluationValidation(measureDef, measure, null); + // Process Criteria Expression Results measureEvaluationResultHandler.processResults(fhirContext, results, measureDef, evaluationType); @@ -393,6 +406,13 @@ public CompositeEvaluationResultsPerMeasure evaluateMultiMeasuresWithCqlEngine( // Trigger first-pass validation on measure scoring as well as other aspects of the Measures R4MeasureDefBuilder.triggerFirstPassValidation(measures); + // Run pre-evaluation validation checks (library resolution, valueset availability, etc.) + final Map parametersMap = new HashMap<>(resolveParameterMap(parameters)); + for (Measure measure : measures) { + var measureDef = new R4MeasureDefBuilder().build(measure); + runPreEvaluationValidation(measureDef, measure, parametersMap); + } + var multiLibraryIdMeasureEngineDetails = getMultiLibraryIdMeasureEngineDetails(measures); var measureUrls = measures.stream() @@ -400,8 +420,6 @@ public CompositeEvaluationResultsPerMeasure evaluateMultiMeasuresWithCqlEngine( .map(url -> Optional.ofNullable(url).orElse("Unknown Measure URL")) .toList(); - final Map parametersMap = new HashMap<>(resolveParameterMap(parameters)); - MeasureProcessorTimeUtils.resolveMeasurementPeriodIntoParameters( measurementPeriodParams, context, @@ -537,4 +555,28 @@ public Interval buildMeasurementPeriod(ZonedDateTime periodStart, ZonedDateTime } return measurementPeriod; } + + private void runPreEvaluationValidation( + org.opencds.cqf.fhir.cr.measure.common.MeasureDef measureDef, + Measure measure, + @Nullable Map parameters) { + var validationContext = new MeasureDefValidationContext(measureDef, measure, repository, parameters); + var validationResult = preEvaluationValidator.validate(validationContext); + + // Log warnings + for (var issue : validationResult.getIssues()) { + if (issue.isWarning()) { + log.warn( + "Measure validation warning [{}]: {} - {}", + issue.code(), + issue.description(), + issue.remediation()); + } + } + + // Block on errors + if (validationResult.hasErrors()) { + throw new MeasureValidationException(measure.getUrl(), validationResult); + } + } } diff --git a/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/r4/R4MeasureReportBuilderContext.java b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/r4/R4MeasureReportBuilderContext.java index 6e6a7edfa..4d52baa3f 100644 --- a/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/r4/R4MeasureReportBuilderContext.java +++ b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/r4/R4MeasureReportBuilderContext.java @@ -16,6 +16,9 @@ import org.hl7.fhir.r4.model.Resource; import org.hl7.fhir.r4.model.StringType; import org.opencds.cqf.fhir.cr.measure.common.MeasureDef; +import org.opencds.cqf.fhir.cr.measure.common.ValidationIssue; +import org.opencds.cqf.fhir.cr.measure.common.ValidationResult; +import org.opencds.cqf.fhir.cr.measure.common.ValidationSeverity; /** * Package-private context class for building R4 MeasureReports. @@ -149,6 +152,15 @@ public void addOperationOutcomes() { } } + public void addValidationOutcomes(ValidationResult validationResult) { + if (validationResult == null || validationResult.isEmpty()) { + return; + } + for (var issue : validationResult.getIssues()) { + addContained(createOperationOutcome(issue)); + } + } + private OperationOutcome createOperationOutcome(String errorMsg) { OperationOutcome op = new OperationOutcome(); op.addIssue() @@ -157,4 +169,48 @@ private OperationOutcome createOperationOutcome(String errorMsg) { .setDiagnostics(errorMsg); return op; } + + private OperationOutcome createOperationOutcome(ValidationIssue issue) { + OperationOutcome op = new OperationOutcome(); + var outcomeIssue = op.addIssue() + .setSeverity(mapSeverity(issue.severity())) + .setCode(mapIssueType(issue.code())) + .setDiagnostics(issue.description() + + (issue.remediation() != null ? " Remediation: " + issue.remediation() : "")); + + if (issue.code() != null) { + outcomeIssue + .getDetails() + .addCoding() + .setSystem("http://opencds.org/fhir/measure-validation") + .setCode(issue.code()); + } + + if (issue.location() != null) { + outcomeIssue.addLocation(issue.location()); + } + + return op; + } + + private static OperationOutcome.IssueSeverity mapSeverity(ValidationSeverity severity) { + return switch (severity) { + case ERROR -> OperationOutcome.IssueSeverity.ERROR; + case WARNING -> OperationOutcome.IssueSeverity.WARNING; + case INFO -> OperationOutcome.IssueSeverity.INFORMATION; + }; + } + + private static IssueType mapIssueType(String validationCode) { + if (validationCode == null) { + return IssueType.PROCESSING; + } + return switch (validationCode) { + case "LIBRARY_NOT_FOUND", "VALUESET_UNAVAILABLE" -> IssueType.NOTFOUND; + case "EXPRESSION_NOT_FOUND" -> IssueType.NOTFOUND; + case "MISSING_REQUIRED_PARAMETER" -> IssueType.REQUIRED; + case "UNKNOWN_PARAMETER", "PARAMETER_TYPE_MISMATCH" -> IssueType.VALUE; + default -> IssueType.PROCESSING; + }; + } } diff --git a/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/r4/R4ParameterConfigurationValidator.java b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/r4/R4ParameterConfigurationValidator.java new file mode 100644 index 000000000..49814ecc7 --- /dev/null +++ b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/r4/R4ParameterConfigurationValidator.java @@ -0,0 +1,91 @@ +package org.opencds.cqf.fhir.cr.measure.r4; + +import ca.uhn.fhir.repository.IRepository; +import java.util.HashSet; +import java.util.Map; +import org.hl7.fhir.instance.model.api.IBaseResource; +import org.hl7.fhir.r4.model.Bundle; +import org.hl7.fhir.r4.model.Library; +import org.hl7.fhir.r4.model.Measure; +import org.hl7.fhir.r4.model.ParameterDefinition; +import org.opencds.cqf.fhir.cr.measure.common.MeasureDefValidationContext; +import org.opencds.cqf.fhir.cr.measure.common.ParameterConfigurationValidator; +import org.opencds.cqf.fhir.cr.measure.common.ValidationIssue; +import org.opencds.cqf.fhir.cr.measure.common.ValidationResult; +import org.opencds.cqf.fhir.cr.measure.common.ValidationSeverity; +import org.opencds.cqf.fhir.utility.search.Searches; + +/** + * R4-specific parameter configuration validator. Reads parameter definitions from the Library + * resource and validates that required parameters are provided and that no unknown parameters + * are passed. Produces {@code MISSING_REQUIRED_PARAMETER} errors and {@code UNKNOWN_PARAMETER} warnings. + */ +public class R4ParameterConfigurationValidator extends ParameterConfigurationValidator { + + @Override + public ValidationResult validate(MeasureDefValidationContext context) { + var result = new ValidationResult(); + var measure = (Measure) context.measure(); + + if (!measure.hasLibrary() || measure.getLibrary().isEmpty()) { + return result; + } + + var url = measure.getLibrary().get(0).asStringValue(); + var bundle = context.repository().search(Bundle.class, Library.class, Searches.byCanonical(url), null); + + if (bundle.getEntry().isEmpty()) { + return result; + } + + var library = (Library) bundle.getEntryFirstRep().getResource(); + validateLibraryParameters(library, context.parameters(), context.repository(), result); + + return result; + } + + @Override + protected void validateLibraryParameters( + IBaseResource libraryResource, + Map parameters, + IRepository repository, + ValidationResult result) { + + var library = (Library) libraryResource; + if (!library.hasParameter()) { + return; + } + + // Collect defined parameter names (input parameters only) + var definedInputParams = new HashSet(); + for (ParameterDefinition paramDef : library.getParameter()) { + if (paramDef.getUse() == ParameterDefinition.ParameterUse.IN) { + definedInputParams.add(paramDef.getName()); + + // Check required parameters (min > 0) are present + if (paramDef.getMin() > 0 && !parameters.containsKey(paramDef.getName())) { + result.addIssue(new ValidationIssue( + ValidationSeverity.ERROR, + MISSING_REQUIRED_PARAMETER, + "Required parameter '%s' (type: %s) is not provided" + .formatted(paramDef.getName(), paramDef.getType()), + "Provide the required parameter '%s' of type '%s' in the operation request." + .formatted(paramDef.getName(), paramDef.getType()))); + } + } + } + + // Check for unknown parameters (excluding well-known operation parameters) + var wellKnownParams = java.util.Set.of("Measurement Period"); + for (var paramName : parameters.keySet()) { + if (!definedInputParams.contains(paramName) && !wellKnownParams.contains(paramName)) { + result.addIssue(new ValidationIssue( + ValidationSeverity.WARNING, + UNKNOWN_PARAMETER, + "Parameter '%s' is not defined in the Library's parameter definitions".formatted(paramName), + "Check that the parameter name '%s' matches a parameter defined in the CQL library." + .formatted(paramName))); + } + } + } +} diff --git a/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/r4/R4ValueSetAvailabilityValidator.java b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/r4/R4ValueSetAvailabilityValidator.java new file mode 100644 index 000000000..c6f32ea06 --- /dev/null +++ b/cqf-fhir-cr/src/main/java/org/opencds/cqf/fhir/cr/measure/r4/R4ValueSetAvailabilityValidator.java @@ -0,0 +1,78 @@ +package org.opencds.cqf.fhir.cr.measure.r4; + +import java.util.HashSet; +import java.util.Set; +import org.hl7.fhir.r4.model.Bundle; +import org.hl7.fhir.r4.model.Library; +import org.hl7.fhir.r4.model.Measure; +import org.hl7.fhir.r4.model.ValueSet; +import org.opencds.cqf.fhir.cr.measure.common.MeasureDefValidationContext; +import org.opencds.cqf.fhir.cr.measure.common.MeasureDefValidator; +import org.opencds.cqf.fhir.cr.measure.common.ValidationIssue; +import org.opencds.cqf.fhir.cr.measure.common.ValidationResult; +import org.opencds.cqf.fhir.cr.measure.common.ValidationSeverity; +import org.opencds.cqf.fhir.utility.search.Searches; + +/** + * Validates that ValueSets referenced in the Measure's CQL library are available in the repository. + * Extracts ValueSet canonical URLs from the library's {@code dataRequirement.codeFilter.valueSet} + * entries and performs an existence check (no expansion). Produces {@code VALUESET_UNAVAILABLE} + * warnings since external terminology services may still resolve them at evaluation time. + */ +public class R4ValueSetAvailabilityValidator implements MeasureDefValidator { + + public static final String VALUESET_UNAVAILABLE = "VALUESET_UNAVAILABLE"; + + @Override + public ValidationResult validate(MeasureDefValidationContext context) { + var result = new ValidationResult(); + var measure = (Measure) context.measure(); + + if (!measure.hasLibrary() || measure.getLibrary().isEmpty()) { + return result; + } + + // Collect ValueSet references from the primary library's dataRequirement + var valueSetUrls = collectValueSetUrls(measure, context); + + for (var vsUrl : valueSetUrls) { + var bundle = context.repository().search(Bundle.class, ValueSet.class, Searches.byUrl(vsUrl), null); + + if (bundle.getEntry().isEmpty()) { + result.addIssue(new ValidationIssue( + ValidationSeverity.WARNING, + VALUESET_UNAVAILABLE, + "ValueSet '%s' referenced by the Measure library is not available in the repository" + .formatted(vsUrl), + "Ensure the ValueSet with URL '%s' is loaded in the repository or available via a configured terminology service." + .formatted(vsUrl))); + } + } + + return result; + } + + private Set collectValueSetUrls(Measure measure, MeasureDefValidationContext context) { + var valueSetUrls = new HashSet(); + + for (var libraryCanonical : measure.getLibrary()) { + var url = libraryCanonical.asStringValue(); + var bundle = context.repository().search(Bundle.class, Library.class, Searches.byCanonical(url), null); + + if (bundle.getEntry().isEmpty()) { + continue; + } + + var library = (Library) bundle.getEntryFirstRep().getResource(); + for (var dataReq : library.getDataRequirement()) { + for (var codeFilter : dataReq.getCodeFilter()) { + if (codeFilter.hasValueSet()) { + valueSetUrls.add(codeFilter.getValueSet()); + } + } + } + } + + return valueSetUrls; + } +} diff --git a/cqf-fhir-cr/src/test/java/org/opencds/cqf/fhir/cr/measure/r4/InvalidMeasureTest.java b/cqf-fhir-cr/src/test/java/org/opencds/cqf/fhir/cr/measure/r4/InvalidMeasureTest.java index 872da15d8..25832979d 100644 --- a/cqf-fhir-cr/src/test/java/org/opencds/cqf/fhir/cr/measure/r4/InvalidMeasureTest.java +++ b/cqf-fhir-cr/src/test/java/org/opencds/cqf/fhir/cr/measure/r4/InvalidMeasureTest.java @@ -4,9 +4,9 @@ import static org.junit.jupiter.api.Assertions.assertTrue; import ca.uhn.fhir.rest.server.exceptions.InvalidRequestException; -import ca.uhn.fhir.rest.server.exceptions.ResourceNotFoundException; import org.cqframework.cql.cql2elm.CqlIncludeException; import org.junit.jupiter.api.Test; +import org.opencds.cqf.fhir.cr.measure.common.MeasureValidationException; import org.opencds.cqf.fhir.cr.measure.r4.Measure.Given; // This class has tests that verify failure behavior for various types of invalid measures. @@ -28,7 +28,9 @@ void evaluateThrowsErrorWhenLibraryUnavailable() { .when() .measureId("LibraryUnavailable") .evaluate(); - assertThrows(ResourceNotFoundException.class, when::then); + var e = assertThrows(MeasureValidationException.class, when::then); + assertTrue(e.getMessage().contains("LIBRARY_NOT_FOUND")); + assertTrue(e.getMessage().contains("was not found in the repository")); } @Test diff --git a/cqf-fhir-cr/src/test/java/org/opencds/cqf/fhir/cr/measure/r4/MeasureDefValidatorTest.java b/cqf-fhir-cr/src/test/java/org/opencds/cqf/fhir/cr/measure/r4/MeasureDefValidatorTest.java new file mode 100644 index 000000000..cd4aa6449 --- /dev/null +++ b/cqf-fhir-cr/src/test/java/org/opencds/cqf/fhir/cr/measure/r4/MeasureDefValidatorTest.java @@ -0,0 +1,352 @@ +package org.opencds.cqf.fhir.cr.measure.r4; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.ArgumentMatchers.isNull; +import static org.mockito.Mockito.lenient; + +import ca.uhn.fhir.repository.IRepository; +import com.google.common.collect.Multimap; +import java.util.List; +import java.util.Map; +import org.hl7.fhir.r4.model.Attachment; +import org.hl7.fhir.r4.model.Bundle; +import org.hl7.fhir.r4.model.CanonicalType; +import org.hl7.fhir.r4.model.DataRequirement; +import org.hl7.fhir.r4.model.Library; +import org.hl7.fhir.r4.model.Measure; +import org.hl7.fhir.r4.model.ParameterDefinition; +import org.hl7.fhir.r4.model.ValueSet; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; +import org.mockito.junit.jupiter.MockitoSettings; +import org.mockito.quality.Strictness; +import org.opencds.cqf.fhir.cr.measure.common.CompositeMeasureDefValidator; +import org.opencds.cqf.fhir.cr.measure.common.MeasureDef; +import org.opencds.cqf.fhir.cr.measure.common.MeasureDefValidationContext; +import org.opencds.cqf.fhir.cr.measure.common.ValidationIssue; +import org.opencds.cqf.fhir.cr.measure.common.ValidationResult; +import org.opencds.cqf.fhir.cr.measure.common.ValidationSeverity; + +@ExtendWith(MockitoExtension.class) +@MockitoSettings(strictness = Strictness.LENIENT) +class MeasureDefValidatorTest { + + private static final String LIBRARY_URL = "http://example.com/Library/TestLibrary"; + private static final String VALUESET_URL = "http://example.com/ValueSet/TestValueSet"; + + @Mock + private IRepository repository; + + private Measure measure; + + @BeforeEach + void setUp() { + measure = new Measure(); + measure.setId("TestMeasure"); + measure.setUrl("http://example.com/Measure/TestMeasure"); + measure.setLibrary(List.of(new CanonicalType(LIBRARY_URL))); + } + + @SuppressWarnings("unchecked") + private void mockLibrarySearch(Bundle result) { + lenient() + .doReturn(result) + .when(repository) + .search(eq(Bundle.class), eq(Library.class), any(Multimap.class), isNull()); + } + + @SuppressWarnings("unchecked") + private void mockValueSetSearch(Bundle result) { + lenient() + .doReturn(result) + .when(repository) + .search(eq(Bundle.class), eq(ValueSet.class), any(Multimap.class), isNull()); + } + + private Bundle bundleWith(org.hl7.fhir.r4.model.Resource resource) { + var bundle = new Bundle(); + bundle.addEntry().setResource(resource); + return bundle; + } + + private Bundle emptyBundle() { + return new Bundle(); + } + + // --- R4CqlLibraryValidator tests --- + + @Test + void libraryValidator_noLibrary_producesError() { + measure.setLibrary(List.of()); + var measureDef = MeasureDef.fromIdAndUrl(measure.getIdElement(), measure.getUrl()); + var context = new MeasureDefValidationContext(measureDef, measure, repository); + + var result = new R4CqlLibraryValidator().validate(context); + + assertTrue(result.hasErrors()); + assertEquals( + R4CqlLibraryValidator.LIBRARY_NOT_FOUND, + result.getBlockingErrors().get(0).code()); + } + + @Test + void libraryValidator_libraryExists_noErrors() { + var library = new Library(); + library.setUrl(LIBRARY_URL); + mockLibrarySearch(bundleWith(library)); + + var measureDef = MeasureDef.fromIdAndUrl(measure.getIdElement(), measure.getUrl()); + var context = new MeasureDefValidationContext(measureDef, measure, repository); + + var result = new R4CqlLibraryValidator().validate(context); + + assertFalse(result.hasErrors()); + } + + @Test + void libraryValidator_libraryNotFound_producesError() { + mockLibrarySearch(emptyBundle()); + + var measureDef = MeasureDef.fromIdAndUrl(measure.getIdElement(), measure.getUrl()); + var context = new MeasureDefValidationContext(measureDef, measure, repository); + + var result = new R4CqlLibraryValidator().validate(context); + + assertTrue(result.hasErrors()); + var error = result.getBlockingErrors().get(0); + assertEquals(R4CqlLibraryValidator.LIBRARY_NOT_FOUND, error.code()); + assertTrue(error.description().contains(LIBRARY_URL)); + assertTrue(error.remediation().contains("Ensure the Library resource")); + } + + // --- R4ValueSetAvailabilityValidator tests --- + + @Test + void valueSetValidator_valueSetMissing_producesWarning() { + var library = new Library(); + library.setUrl(LIBRARY_URL); + var dataReq = new DataRequirement(); + dataReq.addCodeFilter().setValueSet(VALUESET_URL); + library.addDataRequirement(dataReq); + + mockLibrarySearch(bundleWith(library)); + mockValueSetSearch(emptyBundle()); + + var measureDef = MeasureDef.fromIdAndUrl(measure.getIdElement(), measure.getUrl()); + var context = new MeasureDefValidationContext(measureDef, measure, repository); + + var result = new R4ValueSetAvailabilityValidator().validate(context); + + assertFalse(result.hasErrors()); + assertTrue(result.hasWarnings()); + var warning = result.getIssues().get(0); + assertEquals(R4ValueSetAvailabilityValidator.VALUESET_UNAVAILABLE, warning.code()); + assertEquals(ValidationSeverity.WARNING, warning.severity()); + } + + @Test + void valueSetValidator_valueSetExists_noWarnings() { + var library = new Library(); + library.setUrl(LIBRARY_URL); + var dataReq = new DataRequirement(); + dataReq.addCodeFilter().setValueSet(VALUESET_URL); + library.addDataRequirement(dataReq); + + mockLibrarySearch(bundleWith(library)); + mockValueSetSearch(bundleWith(new ValueSet().setUrl(VALUESET_URL))); + + var measureDef = MeasureDef.fromIdAndUrl(measure.getIdElement(), measure.getUrl()); + var context = new MeasureDefValidationContext(measureDef, measure, repository); + + var result = new R4ValueSetAvailabilityValidator().validate(context); + + assertTrue(result.isEmpty()); + } + + // --- R4ParameterConfigurationValidator tests --- + + @Test + void parameterValidator_missingRequiredParam_producesError() { + var library = new Library(); + library.setUrl(LIBRARY_URL); + var paramDef = new ParameterDefinition(); + paramDef.setName("requiredParam"); + paramDef.setUse(ParameterDefinition.ParameterUse.IN); + paramDef.setMin(1); + paramDef.setType("String"); + library.addParameter(paramDef); + + mockLibrarySearch(bundleWith(library)); + + var measureDef = MeasureDef.fromIdAndUrl(measure.getIdElement(), measure.getUrl()); + var context = new MeasureDefValidationContext(measureDef, measure, repository, Map.of()); + + var result = new R4ParameterConfigurationValidator().validate(context); + + assertTrue(result.hasErrors()); + assertEquals( + "MISSING_REQUIRED_PARAMETER", result.getBlockingErrors().get(0).code()); + } + + @Test + void parameterValidator_unknownParam_producesWarning() { + var library = new Library(); + library.setUrl(LIBRARY_URL); + var paramDef = new ParameterDefinition(); + paramDef.setName("knownParam"); + paramDef.setUse(ParameterDefinition.ParameterUse.IN); + paramDef.setMin(0); + library.addParameter(paramDef); + + mockLibrarySearch(bundleWith(library)); + + var measureDef = MeasureDef.fromIdAndUrl(measure.getIdElement(), measure.getUrl()); + var context = new MeasureDefValidationContext(measureDef, measure, repository, Map.of("unknownParam", "value")); + + var result = new R4ParameterConfigurationValidator().validate(context); + + assertFalse(result.hasErrors()); + assertTrue(result.hasWarnings()); + assertEquals("UNKNOWN_PARAMETER", result.getIssues().get(0).code()); + } + + // --- R4ExpressionReferenceValidator tests --- + + @Test + void expressionValidator_expressionNotFound_producesWarning() { + var elmJson = """ + { + "library": { + "statements": { + "def": [ + { "name": "Patient", "context": "Patient" }, + { "name": "Initial Population" } + ] + } + } + } + """; + + var library = new Library(); + library.setUrl(LIBRARY_URL); + library.addContent( + new Attachment().setContentType("application/elm+json").setData(elmJson.getBytes())); + + mockLibrarySearch(bundleWith(library)); + + var measureDef = new R4MeasureDefBuilder().build(createMeasureWithPopulation("Missing Expression")); + var context = new MeasureDefValidationContext(measureDef, measure, repository); + + var result = new R4ExpressionReferenceValidator().validate(context); + + assertTrue(result.hasWarnings()); + assertEquals( + R4ExpressionReferenceValidator.EXPRESSION_NOT_FOUND, + result.getIssues().get(0).code()); + } + + @Test + void expressionValidator_expressionExists_noWarnings() { + var elmJson = """ + { + "library": { + "statements": { + "def": [ + { "name": "Patient", "context": "Patient" }, + { "name": "Initial Population" } + ] + } + } + } + """; + + var library = new Library(); + library.setUrl(LIBRARY_URL); + library.addContent( + new Attachment().setContentType("application/elm+json").setData(elmJson.getBytes())); + + mockLibrarySearch(bundleWith(library)); + + var measureDef = new R4MeasureDefBuilder().build(createMeasureWithPopulation("Initial Population")); + var context = new MeasureDefValidationContext(measureDef, measure, repository); + + var result = new R4ExpressionReferenceValidator().validate(context); + + assertTrue(result.isEmpty()); + } + + // --- CompositeMeasureDefValidator tests --- + + @Test + void compositeValidator_mergesResults() { + mockLibrarySearch(emptyBundle()); + + var measureDef = MeasureDef.fromIdAndUrl(measure.getIdElement(), measure.getUrl()); + var context = new MeasureDefValidationContext(measureDef, measure, repository); + + var composite = new CompositeMeasureDefValidator( + List.of(new R4CqlLibraryValidator(), new R4ValueSetAvailabilityValidator())); + + var result = composite.validate(context); + + assertTrue(result.hasErrors()); + } + + // --- ValidationResult tests --- + + @Test + void validationResult_merge() { + var r1 = new ValidationResult(); + var r2 = new ValidationResult(); + r1.addIssue(new ValidationIssue(ValidationSeverity.ERROR, "CODE1", "desc1", "fix1")); + r2.addIssue(new ValidationIssue(ValidationSeverity.WARNING, "CODE2", "desc2", "fix2")); + + r1.merge(r2); + + assertEquals(2, r1.getIssues().size()); + assertTrue(r1.hasErrors()); + assertTrue(r1.hasWarnings()); + assertEquals(1, r1.getBlockingErrors().size()); + } + + @Test + void validationResult_emptyHasNoIssues() { + var result = new ValidationResult(); + + assertTrue(result.isEmpty()); + assertFalse(result.hasErrors()); + assertFalse(result.hasWarnings()); + } + + // Helper + + private Measure createMeasureWithPopulation(String expressionName) { + var m = new Measure(); + m.setId("TestMeasure"); + m.setUrl("http://example.com/Measure/TestMeasure"); + m.setLibrary(List.of(new CanonicalType(LIBRARY_URL))); + m.getScoring() + .addCoding() + .setSystem("http://terminology.hl7.org/CodeSystem/measure-scoring") + .setCode("cohort"); + + var group = m.addGroup(); + group.setId("group-1"); + var pop = group.addPopulation(); + pop.setId("initial-population"); + pop.getCode() + .addCoding() + .setSystem("http://terminology.hl7.org/CodeSystem/measure-population") + .setCode("initial-population"); + pop.getCriteria().setLanguage("text/cql-identifier").setExpression(expressionName); + + return m; + } +}