Skip to content

Add ability to set parameters for test case evaluation #120

@brynrhodes

Description

@brynrhodes

Add a way to set input parameters for test case evaluation.

Need a generic structure for test cases, that generalizes the existing test cases in use for knowledge artifacts, which include:

  • Library test cases, expressed as an Parameters-expected-results resource, defaulting to "true"
  • PlanDefinition test cases, expressed as an expected GuidanceResponse resource
  • Measure test cases, expressed as an expected MeasureReport resource

Need the ability to specify, for each test case:

  1. The artifact to be tested
  2. The operation to be performed
  3. The input parameters
  4. The input test data
  5. The expected output resource(s)

Need the ability to specify groups of test cases associated with a particular artifact

Need the ability to specify suites of test case groups across artifacts

Need the ability to specify common/shared data among test cases at any level

Proposed structure:

Test case:

Library/$evaluate test case:

  • Parameters-input.json
  • Parameters-expected.json

Measure/$evaluate test case:

  • Parameters-input.json
  • MeasureReport-expected.json

Simplest approach is just to expect the presence of a Parameters-input resource that, if present, is used to provide parameters.

This resource can be in the input/tests, input/tests/[artifact] or input/tests/[artifact]/[case] folders

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions