Define structure of signatures in OCI

[stevvooe]

For signatures to work and be compatible across implementations, we need to define two aspects:

1. What is the scope of the statement being signed? Is it the manifest/list/config directly or do we include something with metadata, such as an annotated descriptor?
2. Where are signing subsystems resolved and how are they structured? How do we balance "resolution" versus "abstraction" without sacrificing functionality?

Number 1 must come before number 2 or we risk a vertically integrated, incompatible mess.

To be clear, this will not be successful if this becomes a file-format discussion, as that won't solve the problem. We need to define the framework within which these formats can operate.

TL; DR We need to define an interface to the signing world.

Context: #22 (comment)

Actions:

• Define potential signing targets for integrated signature systems
• Decide on the scope of the statement provided by signing a target
• Ensure that 1.0 version of specification doesn't limit possibilities

[runcom]

/cc @mtrmac @aweiteka

[wking]

On Wed, Oct 19, 2016 at 02:38:42PM -0700, Stephen Day wrote:

1. What is the scope of the statement being signed? Is it the
   manifest/list/config directly or do we include something with
   metadata, such as an annotated descriptor?
   …
   Number 1 must come before number 2 or we risk a vertically
   integrated, incompatible mess.

Can we just focus on (1) here then? These discussions are complicated
enough without knowingly biting off multiple pieces at once ;).

My personal position on the signing scope is that folks should be
signing an assertion (like the name-assertion object I float in #176).
Without an assertion like that, the signature says “I think this blob
is good for something” but it's not clear what you think it's good
for. With a name-assertion, it's clear that you mean “I think this
blob represents debian:7” (or whatever the name is that you're
asserting).

Where the name being asserted lives doesn't really matter 1. You
could put it:

a. In a generic named-assertion media type with its own blob (like
#176 suggests).
b. As a ‘name’ field in the the signed object (e.g. in
application/vnd.oci.image.manifest.list.v1+json,
application/vnd.oci.image.manifest.v1+json, and
application/vnd.oci.image.config.v1+json. This sounds like your
“the manifest/list/config directly”).
c. As a ‘name’ field in descriptor schema, naming the referenced
object (this sounds like your “annotated descriptor”.

The (small) differences between the approaches are:

• Latency: (a) requires an extra blob retrieval which (b) and (c) do
  not. You can work around that with HTTP/2-style “you'll probably
  want this too” pushes, but still, it will be more work to make (a)
  as performant.
• Composablility: (b) and (c) require existing handlers to be updated
  when a new assertion is added (e.g. you need to teach the
  application/vnd.oci.image.manifest.v1+json validator and unpacker
  about the new ‘name’ field). (a) requires a new handler per
  assertion type, but you don't need to adjust the existing handlers.
• Flexibility: (b) limits nameable blobs to those who have a known way
  to extract the name from the blob itself. Both (a) and (c) allow
  you to name arbitrary blobs.

[stevvooe]

@wking As was expected, my point is completely lost to you. Again, this is not a naming or format discussion. We need to choose the signing targets.

Please refrain from commenting on this issue further before consulting with the maintainers. We must keep this focused and in scope or no progress will be made.

[aecolley]

Intuitively, users who use signing are likely to refer to an image using a name, and are going to assume that the image retrieved is actually signed and named by one of the configured trusted signers. For this reason, I suggest that the signature has to cover the image name (including the ref tag) and one of the content-hashes. I think it's reasonable to make an exception for the ref tag latest.

In particular, I want to be sure that it isn't possible for an attacker to substitute the signed but old-and-vulnerable fooserver:1.0 image for the signed but new-and-secure fooserver:1.1 image that the user requested.

I'm saying nothing about how the image is discovered or how the name is resolved or structured, or where signers come from.

[wking]

On Sat, Oct 22, 2016 at 10:07:59PM -0700, Adrian Colley wrote:

Intuitively, users who use signing are likely to refer to an image using a name, and are going to assume that the image retrieved is actually signed and named by one of the configured trusted signers. For this reason, I suggest that the signature has to cover the image name (including the ref tag) and one of the content-hashes.

I'm pretty sure we want to have the signatures and signed objects in CAS, since that makes them easy to mirror and distribute. #176 proposes structures for doing that, although you can keep the name information in other places too if you don't want a separate name-assertion blob. You don't want to sign the ref (because the ref is mutable data that lives outside of CAS), but a name-assertion is a lot like ref that has moved into CAS (it's just a name string and a blob descriptor, #176). An example procedure for using signed name-assertions to validate names on unpacking is here.

[wking]

On Sat, Oct 22, 2016 at 10:07:59PM -0700, Adrian Colley wrote:

In particular, I want to be sure that it isn't possible for an attacker to substitute the signed but old-and-vulnerable fooserver:1.0 image for the signed but new-and-secure fooserver:1.1 image that the user requested.

IPFS addresses this by providing a number of validity schemes. I expect we'll want something similar here, but “valid forever” assertions (with no additional validation data like validity-range, time-to-live, or parent) seem like a good place to start. Once we agree on that, we can come back in a subsequent round and add more granular validity schemes.

[aweiteka]

The signature format that @mtrmac and others have implemented against is proposed as a specification here: https://github.com/containers/image/pull/59/files. Example:

{
    "critical": {
        "identity": {
            "docker-reference": "docker.io/library/busybox"
        },
        "image": {
            "docker-manifest-digest": "sha256:a59906e33509d14c036c8678d687bd4eec81ed7c4b8ce907b888c607f6a1e0e6"
        },
        "type": "atomic container signature"
    },
    "optional": {
        "creator": "atomic 0.1.0-dev",
        "timestamp": 1471035347
    }
}

Before quibbling about specific details (data structure, values, etc), does this make sense at a high level?

[aecolley]

@aweiteka Just two quibbles. The JSON you quoted describes itself as a signature; but it's actually a document which is signed, and the signature is stored elsewhere. The PR you linked to proposes encrypting the document as well as signing it, which makes no sense. Other than those, and ignoring the details of how the JSON might be integrated into OCI and how keys might be discovered, it makes sense to me.

[stevvooe]

@aweiteka As I outlined in this issue, this cannot be a format discussion. If I'm understanding you're proposal, it seems like you want agreement on the scope of a signing statement.

Before we do this, the first agreement needs to be made on a potential signing target. This needs to be one of the existing formats, perhaps a manifest or manifest list. Then we can talk about the scope of the statement made by the signature.

I've separated out an action on this issue that can help to address the confusion.

[wking]

On Mon, Oct 24, 2016 at 12:10:39PM -0700, Stephen Day wrote:

Before we do this, the first agreement needs to be made on a potential signing target. This needs to be one of the existing formats, perhaps a manifest or manifest list.

I think there are useful workflows around signing configs manifests, and manifest-lists (although there are pros and cons to each choice). I'm happy to go over my arguments again, but it might be more productive if folks who think there is a single reasonable choice pitch their case for that choice.

[aecolley]

Maybe it's a matter of terminology, but I don't see any difference between the target of a signature and the scope of the signature. Either way, it is the thing whose integrity is protected against undetected malicious change.

The manifest and manifest list are both fine targets, as is the descriptor: all of them contain a content-hash which can secure the rest of the download by a client. I don't think the image config will do because it doesn't have hashes for the compressed layers. Clients need the length of each layer file before downloading so they can't be attacked with large files to provoke memory exhaustion, something the descriptor spec protects against quite well.

There's obvious value in allowing different signers for different manifests referenced by the same manifest list. So that's my suggestion: manifests should be signed.

[wking]

On Mon, Oct 24, 2016 at 01:42:37PM -0700, Adrian Colley wrote:

I don't think the image config will do because it doesn't have hashes for the compressed layers.

But it does have diffIDs, so the trade-off is potential gzip-bomb exposure vs. not having to re-sign after layer re-compression.

[wking]

On Mon, Oct 24, 2016 at 01:42:37PM -0700, Adrian Colley wrote:

There's obvious value in allowing different signers for different manifests referenced by the same manifest list. So that's my suggestion: manifests should be signed.

And an unsigned manifest-list opens manifest-list consumers up to denial-of-service type attacks where a malicious intermediate puts bogus entries into the manifest-list for a given platform.

An attacker could also spoof some platform settings (platform.os.features, platform.variant, …) which are currently only exposed in the manifest list. Although I'd like to close this particular vulnerability by using a single platform structure (probably defined in runtime-spec) for all the places where we talk about platforms.

[stevvooe]

@aecolley While the target infers scope, we already have hash-stable data structures that make a very nice target for signing. When deciding on the target, we are saying that a signature should include the hash of that target.

When we talk about scope, we are referring to the semantics of the statement. This includes a superset of what is asserted in the signing target. At minimum, we can make the strong statement that manifest h(X) is signed by key Y. We can increase scope by saying that S + h(X) is signed by key Y, where S are some statements about what is asserted by the holder of the key.

So, this discussion must first decide on what X refers too, the signing target (probably manifest and manifest-list), and the extent of S. We also need to decide how to store and resolve sig(S + h(X)), in addition to S + h(X).