798: Action to check engine against published rules#1700
798: Action to check engine against published rules#1700alexfurmenkov wants to merge 16 commits into
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| - name: Checkout cdisc-open-rules | ||
| uses: actions/checkout@v6 | ||
| with: | ||
| repository: cdisc-org/cdisc-open-rules | ||
| ref: ${{ inputs.rules_ref || 'main' }} | ||
| path: open-rules | ||
| # If cdisc-open-rules is private, add a PAT secret: | ||
| # token: ${{ secrets.CDISC_OPEN_RULES_TOKEN }} | ||
|
|
||
| # ----------------------------------------------------------------------- | ||
| # 3. Set up Python | ||
| # ----------------------------------------------------------------------- | ||
| - name: Set up Python 3.12 |
gerrycampion
left a comment
gerrycampion
left a comment
There was a problem hiding this comment.
This is looking really nice, but I have some comments
| uses: actions/checkout@v6 | ||
| with: | ||
| repository: cdisc-org/cdisc-rules-engine | ||
| ref: ${{ inputs.engine_ref || github.sha }} |
There was a problem hiding this comment.
remove ref for engine checkout. It should use whatever the default is.
gerrycampion
left a comment
gerrycampion
left a comment
There was a problem hiding this comment.
The summary and detailed outputs look awesome. I have one comment, sorry I didn't pickup on this earlier. Hopefully it is an easy conversion for ai.
alexfurmenkov
changed the title
| repository: cdisc-org/cdisc-open-rules | ||
| ref: ${{ inputs.rules_ref || 'rules_2' }} | ||
| path: open-rules | ||
| # If cdisc-open-rules is private, add a PAT secret: |
There was a problem hiding this comment.
I think instead of leaving a comment like this we should either use it or totally remove to basedon the requirement.
| uses: actions/checkout@v6 | ||
| with: | ||
| repository: cdisc-org/cdisc-open-rules | ||
| ref: ${{ inputs.rules_ref || 'rules_2' }} |
There was a problem hiding this comment.
here the system will always fallback to using branch rules_2 when it is auto triggered because of push to main. Input.rules_ref is set on manual override. Please let me know if I am misunderstanding the logic.
|
|
||
| if not os.path.isdir(published_dir): | ||
| print(f"WARNING: Published/ not found under {rules_root}", file=sys.stderr) | ||
| return False |
There was a problem hiding this comment.
Do you think it would be better to fail the test when published rule folder is missing, instead of printing. a warning and silently passing the test? Silently passing might create a false positive here.
|
Thanks for the updates. The PR seems ready to me for merge now, this s this PR depends on changes in PR we will wait for it to be merged before approving this one. |
alexfurmenkov
changed the title
| "Expected": "\u2014", | ||
| "Got": "\u2014", |
There was a problem hiding this comment.
We are a bit inconsistent with naming and it can be a little confusing.
This applies to both the summary table and detailed report. I think we should name these "Expected" and "Actual". This also applies to the diff results which has "committed" and "generated". We should use "Expected" and "Actual" here as well.
|
made this PR to remove the .csv file extension in the results.csv as requested cdisc-org/cdisc-open-rules#38 |
5 participants
workflow to run validation for all published rules, comparing with benchmark validation results
cdisc-org/cdisc-open-rules#22 with changes to allow validation from this repo