Rules blocked: FB6602, FB6510, FB6508, FB6507, FB6506, FB6505, FB6504, FB6503, FB6601

[JozefAerts]

Similar to "define_variable_has_no_data", we need an operator "define_dataset_has_no_data" in order to be able to tackle FDA business rules FB6602, FB6510, FB6508, FB6507, FB6506, FB6505, FB6504, FB6503, FB6601, ...

[SFJohnson24]

This is likely a metadata variable and not an operator

[alexfurmenkov]

Hi @JozefAerts , I couldn't find any of these rules in the rules editor UI. Can you please share it? And a dataset if possible

[JozefAerts]

Hi @alexfurmenkov : you are right, we haven't started on these rules yet.
I will start on at least one tomorrow (it's already late afternoon here) and also set up the test files, and then let you know.
Many thanks, Jozef

[JozefAerts]

Hi @alexfurmenkov, I have started working on rule FB5610: "When VS domain is present in the define.xml, but no VS dataset is present in the submission , then the define.xml must contain ItemGroupDef for VS with attribute def:HasNoData marked as "Yes".
A prototype YAML is now in the rule editor.
I attach test Excel files, one with VS present and one with VS absent.
Similarly, I add two define.xml files, one with for VS dataset definition, having HasNoData="Yes" and one without that attribute.
If you need something more, or something is not OK, please let me know.

define_negative_VS_HasNoData.xml
define_positive_VS_HasNoData.xml
unit-test-coreid-FB5610_VS_Absent.xlsx
unit-test-coreid-FB5610_VS_Present.xlsx

[alexfurmenkov]

@JozefAerts as part of this PR I've extended the define XML reader to add the define_dataset_has_no_data column to the dataset being validated. Will it be enough for the needs of these rules?

[JozefAerts]

Hi @alexfurmenkov My test define.xml was not very well matching the Excel datasets, so I generated a new one (attached).
You can try it in combination with the two attached Excel files also attached, and the current YAML code in the rule-editor. Please feel free to change anything there ...
I now got errors like:
"Failed to build dataset for rule validation. Builder: DatasetMetadataDefineDatasetBuilder, Dataset: POOLDEF, Error: Missing required keyword argument _content in TranslatedText" and/or
"Failed to build dataset for rule validation. Builder: DatasetMetadataDefineDatasetBuilder, Dataset: POOLDEF, Error: Missing required keyword argument _content in TranslatedText"
but I could not find anything wrong with "TranslatedText" elements in my define.xml - but maybe I am missing something.
I also looked into the source code, but could not figure out whether this could come from.

For "Rule Type", I am currently using "Dataset Metadata Check against Define XML" or "Dataset Contents again Define XML" or "Define Item Group Metadata check", but also there, I am not sure which I need to use in this case.

SDTM-ETL_Define_cleaned_POOLID_VS_only.xml
unit-test-coreid-FB6510_VS_Absent.xlsx
unit-test-coreid-FB6510_VS_Present.xlsx

[RamilCDISC]

@JozefAerts were you able to confirm if there is an issue in define? If define is fine then we will have to decide if we fix the issue in this ticket or in a separate ticket.

[JozefAerts]

Hi @RamilCDISC: the define.xml is fine, it validates well against the XML-Schema.
I also tried with "pooldef.xpt" written as lowercase in "def:leaf" for the POOLDEF definition in the define.xml (attached).

It looks to me as the system has a problem with reading the "Description/TranslatedText" element within "ItemGroupDef" for "POOLDEF". But I am not sure ...

SDTM-ETL_Define_cleaned_POOLID_VS_only_pooldef_lowercase.xml

[JozefAerts]

P.S. There also seem to be a lot wrong with the numbering of the rule. But I propose we currently keep "as is" not to make the confusion even higher.