Refactor fragment parsing algorithm to match implementations

[noamr]

This makes a few key changes to the fragment parsing workflow, to better match existing implementations and tests.

• Instead of inserting elements to a fake root element of a connected inert document and then appending them to a DocumentFragment, we let the parser insert the root's children straight into that DocumentFragment. This introduces an "adjusted root insertion target", which is a DocumentFragment to insert to if the parser wants to insert nodes to the root.
• Instead of using allow declarative shadow roots directly from the document, it becomes a parser flag. This removes some confusion that makes some subtle cases use the wrong setting. It is taken from the document only in the "document opening steps" (document.open() of a document created elsewhere), as it's the only place where this setting is sticky.
• The custom element registry is taken from the target of the fragment parse rather than from the context document. Together with the other changes, this essential for picking the correct registry e.g. when calling shadowRoot.innerHTML
• The fragment parsing algorithms take in the target of the parse and infer the context element from that (i.e. it's either the target itself, or its fragment host if it's a DocumentFragment).
• The "adjusted insertion target" is separate from the "appropriate place from inserting a node", as the parser internals make some decisions based on the "intended parent" which should remain unchanged by this PR.
• The "adjusted insertion target" is also an enabler that can be used later on for HTML streaming and sanitize-while-parsing, as it can be the context element instead of the intermediate fragment.
• The intermediate inert document is still around. It is needed to avoid creation time side effects like image loading and custom element constructors to occur before sanitization.

The above changes are a lot closer to how blink implements fragment parsing (and given the passing tests, likely WebKit and Gecko as well).

Closes #11023

• At least two implementers are interested (and none opposed):
  • This PR is built to standardize what implementers already do.
• Tests are written and can be reviewed and commented upon at:
  • See Refactor fragment parsing algorithm to match implementations #12624 (comment). These tests already pass everywhere, but they would fail based on the existing spec wording.
  • Add test-case for DSD when calling document.open() on an XHR response web-platform-tests/wpt#61042 for testing document.open() on an XHR response
• Implementation bugs are filed:
  • Chromium: already works
  • Gecko: already works
  • WebKit: already works
  • Node.js (only for timers, structured clone, base64 utils, channel messaging, and module resolution): …
• Corresponding HTML AAM & ARIA in HTML issues & PRs: N/A
• MDN issue is filed: N/A
• The top of this comment includes a clear commit message to use.

(See WHATWG Working Mode: Changes for more details.)

---

/document-lifecycle.html ( diff )
/dynamic-markup-insertion.html ( diff )
/parsing.html ( diff )
/xhtml.html ( diff )

[noamr]

@shannonbooth do you have a list of WPTs that this fixes? I want to add them to the OP.

[shannonbooth]

Sure!

Original cases which motivated these changes W.R.T connectedness on insertion for fragment parsing:

• https://github.com/web-platform-tests/wpt/blob/0d0249113dc429fbbb4fce7778caeceae0466739/domparsing/createContextualFragment.html#L85
• https://github.com/web-platform-tests/wpt/blob/0d0249113dc429fbbb4fce7778caeceae0466739/html/semantics/forms/the-input-element/radio-disconnected-group-owner.html#L186

More custom element registry tests passing due to a combination of the "allow declarative shadow roots" move to the parser, and also the fix for shadow roots.

• https://github.com/web-platform-tests/wpt/blob/2d258c4a3d47b7eaa9c65252abd834b321ed1679/custom-elements/registries/scoped-registry-initialize-upgrades.html#L154 This subtest
• https://github.com/web-platform-tests/wpt/blob/master/custom-elements/registries/scoped-registry-define-upgrade-criteria.html 7 subtests in here (to all passing)
• https://github.com/web-platform-tests/wpt/blob/master/custom-elements/registries/scoped-registry-append-does-not-upgrade.html 4 subtests in here (to all passing)
• https://github.com/web-platform-tests/wpt/blob/master/custom-elements/registries/ShadowRoot-innerHTML.html All tests
• https://github.com/web-platform-tests/wpt/blob/2d258c4a3d47b7eaa9c65252abd834b321ed1679/custom-elements/form-associated/form-disabled-callback.html#L72 This subtest

[noamr]

Fixed editorial changes, also made the root adjustment for text/comment/PI

[annevk]

@noamr AI identified to more potential subtle issues here (bit wordy unfortunately):

1. Custom element registry for template.innerHTML. The PR changed the fragment root's registry from context's registry (main:149796) to "look up a custom element registry given target" (source:149791). For template.innerHTML, target is the template contents (a plain DocumentFragment), and look up a custom element
   registry returns null for a non-shadow DocumentFragment (source:78283) — whereas main used the template element's registry. For the common global-registry case this is identical, but for scoped registries it's an observable difference.
2. Comments/PIs inserted directly into root in the fragment case. In "after body"/"in frameset" etc., comments are inserted "as the last child of the html element" via an explicit location, which bypasses the new adjusted insertion location root-insertion-target redirection (source:144573 only redirects when the
   location is the topmost node via the default path). Such a comment would land on root, but the algorithm now returns fragment (not root's children), so it would be dropped where main included it. This requires the "after body" mode to be reachable in a fragment parse — which for an html context element it plausibly is.

[noamr]

@noamr AI identified to more potential subtle issues here (bit wordy unfortunately):

1. Custom element registry for template.innerHTML. The PR changed the fragment root's registry from context's registry (main:149796) to "look up a custom element registry given target" (source:149791). For template.innerHTML, target is the template contents (a plain DocumentFragment), and look up a custom element
   registry returns null for a non-shadow DocumentFragment (source:78283) — whereas main used the template element's registry. For the common global-registry case this is identical, but for scoped registries it's an observable difference.

Fixed. Special-casing the template contents case to take the CER from the template element.

2. Comments/PIs inserted directly into root in the fragment case. In "after body"/"in frameset" etc., comments are inserted "as the last child of the html element" via an explicit location, which bypasses the new adjusted insertion location root-insertion-target redirection (source:144573 only redirects when the
   location is the topmost node via the default path). Such a comment would land on root, but the algorithm now returns fragment (not root's children), so it would be dropped where main included it. This requires the "after body" mode to be reachable in a fragment parse — which for an html context element it plausibly is.

Fixed. "adjusted insertion location" now takes in the override target.