Implements NestingType.Flat#2409
Draft
GarrettBeatty wants to merge 1 commit into
Draft
Conversation
|
|
||
| COPY bin/publish/ ${LAMBDA_TASK_ROOT} | ||
|
|
||
| ENTRYPOINT ["/var/task/bootstrap"] |
|
|
||
| COPY bin/publish/ ${LAMBDA_TASK_ROOT} | ||
|
|
||
| ENTRYPOINT ["/var/task/bootstrap"] |
There was a problem hiding this comment.
Pull request overview
This PR implements NestingType.Flat support for ParallelAsync and MapAsync in Amazon.Lambda.DurableExecution, switching per-unit execution to “virtual” child contexts that do not emit per-branch/per-item CONTEXT checkpoints, while still keeping deterministic operation IDs and re-parenting inner operations to the nearest non-virtual ancestor. It also adds unit + integration tests to validate replay behavior and the new checkpoint/payload contract.
Changes:
- Enable Flat nesting by suppressing per-unit child
CONTEXTSTART/SUCCEED/FAIL checkpoints and persisting per-unit results/errors inline on the parent operation payload. - Add support for “virtual” child ID generation so inner ops keep branch/item-scoped IDs but report the non-virtual parent as
ParentId. - Add/extend unit and integration tests validating: no per-unit contexts, correct re-parenting, and correct replay reconstruction from inline payloads.
Reviewed changes
Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| Libraries/test/Amazon.Lambda.DurableExecution.Tests/ParallelOperationTests.cs | Updates tests to validate Flat parallel semantics (no branch contexts, inline replay, re-parenting). |
| Libraries/test/Amazon.Lambda.DurableExecution.Tests/MapOperationTests.cs | Updates tests to validate Flat map semantics (no item contexts, inline replay, re-parenting). |
| Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/ParallelFlatNestingFunction/ParallelFlatNestingFunction.csproj | New integration test function project for Flat parallel. |
| Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/ParallelFlatNestingFunction/Function.cs | New Flat parallel test workflow exercising replay via WaitAsync. |
| Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/ParallelFlatNestingFunction/Dockerfile | Container packaging for the new Flat parallel test function. |
| Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/MapFlatNestingFunction/MapFlatNestingFunction.csproj | New integration test function project for Flat map. |
| Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/MapFlatNestingFunction/Function.cs | New Flat map test workflow exercising replay via WaitAsync. |
| Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/TestFunctions/MapFlatNestingFunction/Dockerfile | Container packaging for the new Flat map test function. |
| Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/ParallelFlatNestingTest.cs | New end-to-end AWS integration test asserting Flat parallel history/IDs/parenting. |
| Libraries/test/Amazon.Lambda.DurableExecution.IntegrationTests/MapFlatNestingTest.cs | New end-to-end AWS integration test asserting Flat map history/IDs/parenting. |
| Libraries/src/Amazon.Lambda.DurableExecution/ParallelConfig.cs | Updates docs to describe Flat’s inline result/error behavior. |
| Libraries/src/Amazon.Lambda.DurableExecution/NestingType.cs | Updates enum docs to reflect Flat now being implemented and its semantics. |
| Libraries/src/Amazon.Lambda.DurableExecution/MapConfig.cs | Updates docs to describe Flat’s inline result/error behavior. |
| Libraries/src/Amazon.Lambda.DurableExecution/Internal/ParallelOperation.cs | Plumbs Flat configuration into concurrent orchestration via isVirtual. |
| Libraries/src/Amazon.Lambda.DurableExecution/Internal/MapOperation.cs | Plumbs Flat configuration into concurrent orchestration via isVirtual. |
| Libraries/src/Amazon.Lambda.DurableExecution/Internal/OperationIdGenerator.cs | Adds “virtual child” ID generator that decouples ID prefix from reported ParentId. |
| Libraries/src/Amazon.Lambda.DurableExecution/Internal/ConcurrentOperation.cs | Implements inline per-unit payload persistence (Flat) and replay reconstruction paths. |
| Libraries/src/Amazon.Lambda.DurableExecution/Internal/ChildContextOperation.cs | Suppresses per-unit CONTEXT checkpoints when the child context is virtual (Flat). |
| Libraries/src/Amazon.Lambda.DurableExecution/Internal/BatchSummary.cs | Extends summary payload to optionally include inline Result / Error for Flat units. |
| Libraries/src/Amazon.Lambda.DurableExecution/Internal/BatchJsonContext.cs | Adds source-gen metadata for ErrorObject in the batch summary payload. |
| Libraries/src/Amazon.Lambda.DurableExecution/DurableContext.cs | Removes Flat “not supported” guard and updates child-context factory to support virtual parenting. |
| Libraries/src/Amazon.Lambda.DurableExecution/CLAUDE.md | Adds repository-specific architecture/testing guidance (documentation only). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+612
to
+634
| // Flat (virtual) units have no child checkpoint — their result/error | ||
| // was recorded inline on this summary. Nested units read from the | ||
| // child's own CONTEXT checkpoint. A unit is "inline" when the summary | ||
| // entry carries a Result/Error, which only Flat writes. | ||
| if (_isVirtual && summaryEntry != null) | ||
| { | ||
| if (status == BatchItemStatus.Succeeded && summaryEntry.Result != null) | ||
| { | ||
| unitResult = DeserializeResult(summaryEntry.Result); | ||
| } | ||
| else if (status == BatchItemStatus.Failed && summaryEntry.Error != null) | ||
| { | ||
| var err = summaryEntry.Error; | ||
| unitError = new ChildContextException(err.ErrorMessage ?? "Unit failed") | ||
| { | ||
| SubType = ChildSubType, | ||
| ErrorType = err.ErrorType, | ||
| ErrorData = err.ErrorData, | ||
| OriginalStackTrace = err.StackTrace | ||
| }; | ||
| } | ||
| } | ||
| else if (status == BatchItemStatus.Succeeded && childOp?.ContextDetails?.Result != null) |
97d51e0 to
e0b6f5f
Compare
54a24e4 to
e1db9a7
Compare
fb4572a to
02c23f5
Compare
e1db9a7 to
f01d9f7
Compare
f01d9f7 to
bb1b112
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue #, if available: #2216
Description of changes:
Summary
Wires up
NestingType.FlatforParallelAsyncandMapAsync. Both previously threwNotSupportedExceptionforFlat; onlyNested(the default) worked. The implementation uses an inline-payload strategy matching the Python, JS, and Java durable-execution SDKs.What
FlatdoesEach parallel/map branch normally runs in its own child
CONTEXToperation that emitsSTART+SUCCEED/FAILcheckpoints — roughly one extra op per branch.NestingTypecontrols this:Nested(default) — each branch emits a fullCONTEXTop. Maximum trace observability, more checkpoints.Flat— each branch runs in a virtual context that emits noCONTEXTcheckpoint; per-branch results/errors are recorded inline on the parent operation's payload. Fewer checkpoints, less granular traces.Nestedstays the default, so opting in is non-breaking.How it works
A flat branch is a logical scope that owns an ID namespace and logger scope but is invisible in the checkpoint tree:
OperationIdGenerator.CreateVirtualChild) — inner-op IDs are still prefixed by the branch's own op-id (so siblings never collide and the ID space matchesNested, which is load-bearing for replay), but inner ops report the nearest non-virtual ancestor (the parallel/map op) as their wireParentId.ChildContextOperation) — anisVirtualflag suppresses the branch'sCONTEXTcheckpoints. Exceptions still propagate; inner ops still checkpoint, re-parented per (1).BatchSummary/BatchUnitSummary) — with no child checkpoints to read back on replay, each unit's serialized result/ErrorObjectis stored inline on the parent'sBatchSummary.ConcurrentOperationpersists this even onFAIL.ConcurrentOperation.ReconstructFromCheckpoints) — when virtual, per-unit results/errors are read from the inline summary.Tests
ParallelOperationTests,MapOperationTests) — replaced the*_NestingTypeFlat_ThrowsNotSupportedtests with Flat coverage: suppressed per-branch contexts, inner-op re-parenting, inline partial-failure surfacing, and replay-from-inline-payload (success + failure).ParallelFlatNestingTest,MapFlatNestingTest) — end-to-end against the real service with 3 branches each (step + durable wait to force suspend/resume). Assert exactly oneCONTEXTop, inner ops re-parented to the parallel/map op, inner-op IDs from the per-branch space, and per-branch results surviving replay.All 331 unit tests pass on net8.0 and net10.0.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.