Fix generate_delft_data round-trip for citation model training TEI#683
Merged
Conversation
…ookup biblScope elements generated for the citation model can carry from/to page-range attributes (added for sciencebeam-judge's fpage extraction) alongside unit. generate_delft_data failed to parse such elements back because the element-path matcher only supported a single attribute.
…fier type get_post_processed_xml_root tags <idno> elements with any detected external identifier type (DOI, PMCID, PMID, ARXIV, PII), but only idno[@type="DOI"] had a training path back to a label (<idno>); bare idno mapped to <pubnum>. Once tagged with e.g. type="PMCID", generate_delft_data failed to parse the element back since no path matched. Alias the other detected types back to <pubnum>, matching the label they were generated from.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
part of https://github.com/eLifePathways/ScienceBeam2.0/issues/113
biblScope and idno elements in citation training TEI can carry extra
attributes added purely for downstream consumers (sciencebeam-judge's
fpage extraction, and external-identifier-type detection), which broke
generate_delft_data's ability to parse the TEI back into labels:
biblScope[@Unit="page"] elements post-processed with @from/@to
page-range attributes raised "only supporting up to one attribute"
when matching the element path. from/to are metadata only, never used
for label lookup, so they're now excluded from the path expression.
idno elements are tagged with any detected external identifier type
(DOI, PMCID, PMID, ARXIV, PII), but only idno[@type="DOI"] had a path
back to a label (); bare idno mapped to . Once tagged
type="PMCID" etc., there was no matching path. The other types now
alias back to , matching the label they were generated from.
Adds regression tests for both cases, including a full round-trip test
(generate training TEI, including post-processing, then parse it back).