design: add design doc for secret driver plugins

[Veector40]

Addresses #27856

As per @ashley-cui 's request in the original issue, this PR introduces the design document for Secret Driver Plugins.

Summary of the Design:

• Proposes a plugin API for external secret drivers (like systemd-creds or cloud secret managers).
• Plugins are registered in containers.conf under [secrets.driver_plugins].
• Important Pivot: To address Ashley's concern regarding passing sensitive data via environment variables, this design defines the API contract to use structured JSON payloads passed entirely over stdin and stdout.

This API design is extensible, avoids global process-tree visibility issues, and also isolates the initial implementation from the core secrets injection logic.

cc @matt-allan

[matt-allan]

Thanks for writing this @Veector40.

I hope the podman maintainers don't mind -- I left some comments to clarify a few details I was thinking about when writing the original issue.

[Veector40]

Thanks for the detailed feedback, @matt-allan! I’ve updated the design doc to incorporate your suggestions. Specifically:

• Mapping semantics: Clarified containers.conf mapping and confirmed that built-in drivers take precedence (shadowing behavior).
• Simplified API: Removed the environment variable justification and shifted focus to the standard input implementation.
• Payload separation: Updated the protocol to use a JSON header followed by a newline and raw secret bytes to avoid Base64 overhead and risk of exposure.
• Metadata ownership: Removed Name and CreatedAt from the plugin's responsibility; the driver now strictly tracks ID.
• Execution: Noted that Podman execs the binary directly without a shell.
• Tooling: Added a note about eventually providing a Go library to simplify driver development.

[packit-as-a-service]

[NON-BLOCKING] Packit jobs failed. @containers/packit-build please check. Everyone else, feel free to ignore.

[Luap99]

Thanks for the PR but I would be strongly against any new "drivers" that have to be implemented as I find that concept not good.

See my comments in #26488
It would be much better if just we focus how to get the secret data exposed to the container with having to use commands like podman secret create or remove.

[Luap99]

that is not really true as there is the shell driver that could be used so that needs to mention why and how that not works

[matt-allan]

Thanks for the PR but I would be strongly against any new "drivers" that have to be implemented as I find that concept not good.
See my comments in #26488
It would be much better if just we focus how to get the secret data exposed to the container with having to use commands like podman secret create or remove.

The concept that secrets use is the same concept used by the rest of Podman: secrets are resources managed by Podman. You have to use create a secret the same way you have to create a network, a volume, etc.

The approach outlined in #26488 is a major departure from that model. It's not clear how the existing API endpoints and CLI endpoints should work with those changes. My intent with #27856 was to fix the main limitations of the existing secrets without departing from the existing resource model.

I can already use a bind mount to expose secrets to Podman containers directly. The value of having secrets managed by podman (for me) is having the resources I can list from the API and CLI.

The problem the author of #26488 is trying to solve can't be solved with the shell driver. That doesn't mean the driver interface is bad; it's specifically the shell driver that has limitations.

With the shell driver you can't pass through options. Without options you have no way to reference an existing secret. If we pass through options, you can do something like this:

printf "" | podman secrets create \
    --driver systemd-creds
    --driver-opts 'cred=db_password.cred'
    db_password -

The same idea works for existing secrets in vault, SSM, etc. You still have to "create" them in Podman, but that's really just adding the metadata to the secrets DB. This is similar to volume plugins, where you can 'create' a volume that really just mounts an existing resource.

(The printf is necessary because Podman requires some input, even if empty. That could be changed separately if this is a common use case).

[Veector40]

Thanks for the detailed feedback, @Luap99 and @matt-allan

I had just finished completely reworking the draft design doc to accommodate @Luap99's request to move away from stateful drivers, right as @matt-allan commented defending the original driver concept.
Since it looks like we have two fundamentally different architectural directions for how external secrets should be handled, I've left drafts for both approaches in this PR's commit history to make it easier to compare and decide:
1. The stateless "Provider" model (current HEAD: cedfd97)
This is the rework I just pushed based on @Luap99's feedback. It drops the driver concept completely. It proposes a just-in-time lookup using podman run --secret provider=..., bypassing the Podman database and podman secret create entirely.
2. The stateful "Driver plugin" model (previous commit: 716d9c7)
Per @matt-allan's original issue #27856 and recent comment, this keeps secrets as Podman-managed resources. It uses podman secret create --driver ... to store metadata in the Podman DB, so secrets can still be tracked and listed via the CLI/API, even if the payload lives externally.
I am fully on board to build whichever direction aligns best with Podman's roadmap. Could you both take a look at the two commits and let me know which architectural model we should officially move forward with? I will finalize the design doc and start working on the code once we have a consensus.

[github-actions]

A friendly reminder that this PR had no activity for 30 days.