Restrictions on Overlay input OADs?

[handrews]

In the TDC call this past week, I tried to use Overlays as an example for talking about the scope of each specification, and said that Overlays require a "valid" OpenAPI document, where "valid" means "any compliant parser will parse it successfully."

While everyone agreed that un-parsable OAD documents are outside of the scope of Overlays, the word "valid" and/or its definition proved unexpectedly contentious.

@hellobudha were you using a different definition of "valid"? That is what it sounded like to me but I was on 3 hours of sleep and may have just gotten my wires crossed.

@lornajane If I understand you correctly, you perhaps want a higher bar than "it parses"? Perhaps "usable"? What would that look like?

In practical terms, here's the use case I want to understand:

1. anytime anyone asks us to support setting global headers or error responses, we tell them to use an Overlay
2. An OAD without those headers would definitely parse, but it would arguably not be usable in the sense that you would fail to provide mandatory headers or fail to understand common error responses.

Is this a valid use case for Overlays? If so, how does it square with setting a higher bar for an Overaly input than "it parses"?

Also, how would an Overlay tool enforce a higher bar than "it parses"?

Other than the above use case, I don't really have a horse in this race. But I do want to know whether the advice we're giving people in the OAS repo is actually within the scope of Overlays.

[lornajane]

I'm not suggesting a higher bar, I was trying to clarify your comment about having an openapi description that needed an overlay applying to it before use. We don't currently have a concept of "unready" API descriptions, and I'm not sure I'm comfortable with introducing one. I don't think we're disagreeing very much though, the suggestion of having some indication of overlays available for an openapi document might be useful.

[handrews]

@lornajane thanks for the clarification, this helps.

How would you characterize the example I gave of an OAD that needs global headers and error responses applied with an Overlay, if not as "unready"?

[lornajane]

I just don't think that's anything a standards body should be legislating, or maybe I don't understand what you're suggesting. People have their own API description pipelines that they build up to meet the needs of their specific contexts, a description probably isn't in a binary ready-or-not status - there are probably different transforms needed for different use cases.

[handrews]

@lornajane I think there's definitely some confusion here as I don't understand what you mean by "legislating" here, or why you think I'm going for a binary ready-or-not definition of anything.

Before trying to solve that, can we come to an agreement (or determine where we disagree) on whether my example is a valid use for Overlays? It's still not clear to me whether you think it is or not, yet it is what we have told many people to do.

So, without worrying about what we call the state of the input or output OADs (we can pick the best terminology later) or worrying about whether this answer sets any sort of precedent, legislative or otherwise (it's a thought experiment, we aren't bound by the outcome):

Given a very large API that has a mandatory header that MUST be sent with every request, and for which every operation could respond with a specific form of error that requires documentation (for client code generation, for example), is it a valid use case of Overlays to write an OAD without the header and error response, and then use an Overlay to:

• add the header's Parameter Object to every Operation Object's parameter list, and
• add the error response's Response Object to every Responses Object under the appropriate error code?

This is not a theoretical question, it's something that comes up with consulting clients and I'd like to know if I'm giving them advice that complies with our specifications.

[handrews]

Perhaps another possible source of confusion is that I titled this "Restrictions..?" I'm not trying to add restrictions here, beyond "the OAD has to parse to apply an Overlay" which I think we all agreee on. My sense is that you (@lornajane and @hellobudha ) don't want to set the bar that low. I can't figure out why, or where you want to set the bar. This is why the "legislating" thing confuses me- I'm trying to not mandate any requirements beyond "it parses", and I feel like there is resistance to that.

(but please also answer about the use case- to me it is the primary Overlay use case I tell people about and it's mildly freaking me out that no one will validate it)

[lornajane]

What is confusing to me about this conversation is not what is written here, but what was discussed in the call. The Overlay needs a valid OpenAPI description, yes it does, I don't think anyone disagrees.

However we also discussed about indicating that an OpenAPI is not itself "usable" (insert other words here, I think the terminology is still a bit under discussion) unless and until an Overlay has been applied. That's the part I didn't understand or align with, and it's probably more of an OpenAPI discussion than an Overlay one.

[handrews]

@lornajane the confusion for me is that the use case I've given here is an example of an OAD that is not usable without an Overlay. I'm not worrying about "indicating" anything at this moment, because that doesn't matter. The use case involves an OAD that is:

1. Syntactically valid
2. Actually unusable, due to the missing "global" mandatory header and missing error response descriptions

So is this a valid use case? It doesn't matter whether the non-usableness is indicated. It's not usable whether we say so or not. Is this OK?

[ThomasRooney]

While everyone agreed that un-parsable OAD documents are outside of the scope of Overlays, the word "valid" and/or its definition proved unexpectedly contentious.

From the perspective of a tool vendor, "valid" means "whatever the customer brings" 😓 . Overlays have proven to be a very useful tool for when multiple tools (e.g. server frameworks) are producing OADs across an organization, and the tool vendor are effectively operating in the space of merging everything together and producing an artifact (client SDKs, gateway configurations, MCP servers, documentation). To do that without insanity, an OAD is a great intermediary artifact but it is commonplace that the "input" before it reaches us is only "almost" an OAD when strictly defined.

As such we intentionally made a choice to apply overlays to any JSON or YAML file, before we ran it through our parser so that we could "fix" any invalidity in the OAD. Some of the most common things that come to mind in this respect are:

1. required: true applied to a request/response body json schema field instead of a parameter
2. Multiple operations re-using the same operationId
3. Invalid oneOf discriminators (e.g. pointing at optional fields)
4. Operations which lack operationId or operationRef
5. Defining a well-defined security scheme (like oAuth) but skipping scopes
6. Defining a custom security scheme

[miqui]

terminology to reach consensus:

• valid
• usable
• syntactically invalid
• parsable