improve passing context

[thaJeztah]

Relates to OTEL changes and graceful exit changes

See below PRs

• Add OTEL support (carry 4556) #4698
• feat: global signal handling to cancel ctx for graceful exits #4993

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 0% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 61.08%. Comparing base (8651906) to head (709df33).

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #4774   +/-   ##
=======================================
  Coverage   61.08%   61.08%           
=======================================
  Files         295      295           
  Lines       20660    20660           
=======================================
  Hits        12621    12621           
  Misses       7142     7142           
  Partials      897      897           

[thaJeztah]

Let me move this one out of draft, but please have a close look to see if the approach makes sense. I recall I was fiddling a bit with passing the context with the combination of closures and call-sites of different parts of the code.

[Benehiko]

I think there are still a few places this can be improved, especially on setting context in the dockerCLI.
dockerCli, err := command.NewDockerCli(command.WithBaseContext(ctx))

I can also move this PR forward if you'd like me to since I am working on adding more context to the docker client in moby and adding support in the CLI for it.

[laurazard]

LGTM

[laurazard]

@thaJeztah can you TA(nother)L?

[vvoland]

Leaving request changes for now; I don't think we should merge this for v26 since it's a breaking change to the plugin.Run function.

[Benehiko]

Do plugins add their own ctx to the dockerCli with the makeCmd?

[krissetto]

compose seems to use whatever cobra's context is (https://github.com/docker/compose/blob/85567ae09246ea79191e35f060d230aa5b7c0e5b/cmd/compose/compose.go#L384)

buildx uses this func defined in buildkit though, but watch out since it already handles signals (https://github.com/docker/buildx/blob/a9575a872ab2018418e668649ed488090402a04a/commands/root.go#L35

https://github.com/moby/buildkit/blob/master/util/appcontext/appcontext.go)

[Benehiko]

I see, yeah this will be breaking for plugins. I'll see what I can do to work around that.

but watch out since it already handles signals

that's fine, this PR doesn't do any signal handling here, but this is a good point and will work around this

[Benehiko]

take note, instead of passing ctx to the root command through convoluted function calls I've cleaned it up a bit by instead passing context to cobra when we call ExecuteContext. This should only affect the CLI and not any plugins that are run through the CLI.

[thaJeztah]

Ah, yes, this was one of the things to look into!

(Still catching up on latest changes in this PR, but yes, this was one where my approach was probably not right)

[Benehiko]

removed this here since we can use the cobra ExecuteContext instead. This prevents us from passing in ctx to the plugins through RunPlugin.

v26.1.0 has been released

[krissetto]

LGTM

[laurazard]

Did this get borked during a rebase? It's returning early, should be returning after hooks get called I think

[Benehiko]

ah true! will fix

[thaJeztah]

Oh! Should it still handle context errors early though? (cancelled context? or timeout?) I guess in that case the hooks should not be called?

[laurazard]

I think it's fine to do it regardless in any case here? The plugin can decide if it wants to print something or not. Even if it's cancelled, a hook might want to add some hint/explain something.

[thaJeztah]

Yeah, not sure if a cancelled context should still be used; isn't context cancelation basically; "cancel my request" (on the first possible occasion) ? In some cases that may mean some work is needed to complete the transaction (flush to disk, e.g.) but calling hooks (and execute other binaries) feels like that should not be part of that.

[laurazard]

that's valid, sure. @Benehiko wdyt

[krissetto]

mmh i'm not sure if hooks should be called after a cancellation, it'd feel wrong to me

[Benehiko]

I don't have too much opinion on this either way. We currently execute the hooks even when the cobra Execute function returns an error (which is a similar thing to context cancellation).

Cancelling the context early will still execute the cobra commands, but the commands checking for context cancellation will just return (with maybe an error?).

See below an example of a cobra command not acting on the context cancellation. Even if it was acting and returning an error it will still execute.

package main

import (
	"context"
	"fmt"

	"github.com/spf13/cobra"
)

func main() {
	ctx, cancel := context.WithCancel(context.Background())

	rootCmd := &cobra.Command{
		Use: "",
		RunE: func(cmd *cobra.Command, args []string) error {
			cmd.Help()
			fmt.Printf("Cobra command ctx: %v\n", cmd.Context().Err())
			return nil
		},
	}

	cancel()

	fmt.Printf("Before Cobra ctx: %v\n", ctx.Err())

	err := rootCmd.ExecuteContext(ctx)
	fmt.Printf("%v\n", err)
}

⋊> ~/G/test-cobra go run .                        09:14:06
Before Cobra ctx: context canceled
Usage:
   [flags]

Flags:
  -h, --help   help for this command
Cobra command ctx: context canceled
<nil>

[Benehiko]

See here what the code currently looks like on the master branch
https://github.com/docker/cli/blob/master/cmd/docker/docker.go#L355-L367

[laurazard]

LGTM

[vvoland]

Ah, one more thing, can you please add the changelog description with highlighting the API change?

EDIT: Sorry, I confused PRs 🙈 There was no API change in the end here (we didn't alter the RunPlugin signature), so no need for changelog!

[Benehiko]

Ah, one more thing, can you please add the changelog description with highlighting the API change?

EDIT: Sorry, I confused PRs 🙈 There was no API change in the end here (we didn't alter the RunPlugin signature), so no need for changelog!

@vvoland You had the PR right, I just reverted those changes in the PR and squashed the commits