CT cache submission_lookup issue - submission values are not unique

[ASL-rmarshall]

Describe the bug
At the moment, a submission_lookup dictionary is created within each cached CT file. This dictionary uses the CT submission value a key, with values containing codelist code and term code (which is "N/A" for codelists), and is (intended to be) populated with all codelists and all terms.

However, submission values are not unique:

• A codelist may have the same submission value as a term (e.g., "TTYPE" is the submission value for both codelist "C66739" and term "C49660").
• Two (or more) terms may have the same submission value (e.g., "0" is the submission value for 53 terms in the SDTM CT 2024-09-27).

[Note that submission values for codelists are unique: no two codelists within the same version of CT for the same standard will have the same submission value]

It appears that the submission_lookup ends up being populated with the last occurrence of a submission value, regardless of whether it's for a codelist or for a term. This means that:

• There may be no entry in submission_lookup for any codelist whose submission value matches the submission value of a (later) term.
• There will be only one entry in submission_lookup for any term submission value that is used for multiple terms.

Expected behavior

I think the submission_lookup dictionary needs to be updated so that the value is a list of the codelist code and term code combinations for all occurrences of the submission value key. Any functionality that references submission_lookup should then be updated parse the value list - either to find codelist entries or to process each of the term entries.

[SFJohnson24]

We can add codelist and term submission value to the lookup key (see lower example. This could be stringified or use a delimiter to separate codelist/term) or we could create a list for terms with the codelists and term CCodes, then iterate through that list (see above). Either is viable.

[ASL-rmarshall]

@SFJohnson24 I finally got round to creating the specification for updates to the codelist_terms operation (#1177). When retrieving codelist information, we'll need to be able to specify the terminology package type(s) and version(s) from which to retrieve information, and also to be able to group the retrieved information by version, codelist submission value or code, and/or term submission value or code. I don't think it should be necessary to include the version within the cached terminology package - it should just be possible to retrieve this from the package name.

[ASL-rmarshall]

@gerrycampion As the issue with non-unique submission values has not been resolved, I don't think this issue should have been closed. Reopening.

[gerrycampion]

closed with #1462