test (docs) Guide for writing e2e tests that take summaries#27488
test (docs) Guide for writing e2e tests that take summaries#27488agarwal-navin wants to merge 3 commits into
Conversation
Add WritingTestsThatTakeSummaries.md documenting the pattern for end-to-end tests that drive summaries on demand: the two container configurations (interactive vs. summarizer), how to take and inspect a summary, how to load a container or summarizer from a specific summary, and the rules that keep these tests deterministic. Also add a table of contents and Introduction section to the package README and WritingCompatCorrectTests.md, and link the new doc from the README. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Hi! Thank you for opening this PR. Want me to review it? Based on the diff (332 lines, 4 files), I've queued these reviewers:
How this works
|
…ting guide Point readers who browse src/test/summarization/ to WritingTestsThatTakeSummaries.md, with a few worked examples. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
🔗 No broken links found! ✅ Your attention to detail is admirable. linkcheck output |
| @@ -0,0 +1,18 @@ | |||
| # @fluidframework/test-end-to-end-tests/test/src/test/summarization | |||
There was a problem hiding this comment.
Nit: our best practices docs for Markdown recommend line-breaking along sentence boundaries. This helps make future changes easier to review.
https://github.com/microsoft/FluidFramework/wiki/Markdown-Best-Practices#line-breaks-along-sentence-boundaries
|
|
||
| The minimal round looks like this: | ||
|
|
||
| ```ts |
There was a problem hiding this comment.
Since this is the first code example demonstrating the test flow, it might be worth including additional boilerplate beforehand, including where provider comes from (or at least a quick note about it)
There was a problem hiding this comment.
Actually, having read the rest of the document, including more code here probably doesn't make sense. But a quick note that provider will be explained more later in the document might be useful.
|
|
||
| Putting it together — create, summarize, load-and-validate, then summarize from a new summarizer: | ||
|
|
||
| ```ts |
There was a problem hiding this comment.
A thought: since we don't want this to get out-of-date, it might be worth checking this in as an actual test in code (if it isn't already). Then we could use our markdown-magic pragmas to embed the code here, rather than copying it. That way, if we end up having to update this example for one reason or another, our docs won't become out-of-date.
There was a problem hiding this comment.
An example of this pattern can be found here: https://github.com/microsoft/FluidFramework/blob/main/tools/api-markdown-documenter/README.md?plain=1#L225
Josmithr
left a comment
There was a problem hiding this comment.
I left a couple nitpicks and mechanical suggestions, but the docs look great! Thanks for doing this!
Description
Adds a new doc,
WritingTestsThatTakeSummaries.md, to thetest-end-to-end-testspackage documenting the pattern for writing end-to-end tests that drive summaries on demand. It covers:summarizeNowand inspect the result.syncSummarizer: true,ensureSynchronized()before summarizing, chaining viasummaryVersion, closing one summarizer before starting another, etc.).It also adds a Table of Contents and an Introduction section to the package
README.mdandWritingCompatCorrectTests.mdfor consistency and navigation, links the new doc from the README, and fixes a heading typo ("Enpdoint" → "Endpoint") in the README.Documentation-only change — no code or public API is affected.