Use a stageJoiner to join stages together#59
Open
mhagger wants to merge 4 commits into
Open
Conversation
mhagger
force-pushed
the
moar-streams
branch
2 times, most recently
from
56b4c3f to
07cbbcf
Compare
The two aspects of a stage's stream requirements are coupled. For example, of the four possible combinations of `(StreamRequirement, NeedsFile)`, one of them, `(StreamForbidden, true)`, makes no sense. So instead of encoding these two aspects separately, encode the three meaningful combinations into a single `StreamRequirement` type with possible values `StreamAcceptAny`, `StreamPreferFile`, and `StreamForbidden`.
The old code used a `stageStarter` type to help with starting up stages. There are two awkward things about that approach: * The pipe that is needed to join stages depends on the two adjacent stages, not on the stdin and stdout of a single stage. So `stageStarter` wasn't able to figure out what pipe was needed; that logic kindof needed to live in the `Pipeline` type. * Since the read and write ends of a pipe are created together, the `stageStarter` could also not be as helpful in closing those streams if there was an error. So instead, use a new helper type, `stageJoiner`, to figure out how to join two adjacent stages together. The `stageJoiner` can now do part of the work for us: * `stageJoiner.validate()` checks that the adjacent stages' requirements are met WRT whether they need stdin/stdout to be supplied at all. * `stageJoiner.createPipe()` figures out what kind of pipe to create for the two adjacent stages. * `stageJoiner.closePipe()` closes the pipe (if it has already been created and needs closing) on errors. This isn't a panacea; for example, some loops have to iterate over stages, and some over stage joiners. But I think that it's less confusing this way.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The old code used a
stageStartertype to help with starting up stages. There are two awkward things about that approach:The pipe that is needed to join stages depends on the two adjacent stages, not on the stdin and stdout of a single stage. So
stageStarterwasn't able to figure out what pipe was needed; that logic kindof needed to live in thePipelinetype.Since the read and write ends of a pipe are created together, the
stageStartercould also not be as helpful in closing those streams if there was an error.So instead, use a new helper type,
stageJoiner, to figure out how to join two adjacent stages together. ThestageJoinercan now do part of the work for us:stageJoiner.validate()checks that the adjacent stages' requirements are met WRT whether they need stdin/stdout to be supplied at all.stageJoiner.createPipe()figures out what kind of pipe to create for the two adjacent stages.stageJoiner.closePipe()closes the pipe (if it has already been created and needs closing) on errors.This also removes most of the need for special-casing the last stage.
/cc @znull