diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..4a68050 --- /dev/null +++ b/.gitignore @@ -0,0 +1,3 @@ +openapi/dali +openapi/vosi +openapi/uws diff --git a/Makefile b/Makefile index 022f830..718ca0b 100644 --- a/Makefile +++ b/Makefile @@ -7,7 +7,7 @@ DOCNAME = TAP DOCVERSION = 1.2 # Publication date, ISO format; update manually for "releases" -DOCDATE = 2024-11-11 +DOCDATE = 2025-05-20 # What is it you're writing: NOTE, WD, PR, or REC DOCTYPE = WD diff --git a/README.md b/README.md index cd14455..fe6ebaf 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,16 @@ # Table Access Protocol +## State of Development +The current content is WD-TAP-1.2 with an OpenAPI 3.1 specification and is a work in progress. + +The DALI and VOSI OpenAPI components (yaml files) can be copied in with `rsync-openapi-components.sh`, +but source locations are hard-coded so check it and use with care. + +If you have docker available, the OpenAPI can be validated with `openapi-linter`. + - Creative Commons License + Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. diff --git a/TAP.tex b/TAP.tex index 01d4f3e..c0b6590 100644 --- a/TAP.tex +++ b/TAP.tex @@ -8,6 +8,7 @@ \lstset{flexiblecolumns=true,basicstyle=\small,tagstyle=\ttfamily, showstringspaces=False} \usepackage[utf8]{inputenc} +\usepackage{todonotes} \hyphenation{asyn-chro-n-ous-ly} @@ -151,8 +152,8 @@ \subsection{Role within the VO Architecture} Language. \item[VOSI \citep{2017ivoa.spec.0524G}] The VO Support Interfaces standard -defines endpoints for metadata discovery; for TAP, this is the tables, -capabilities, and availability endpoints. Note that while TAP 1.1 does not +defines endpoints for metadata discovery; for TAP, this is the tables and +capabilities endpoints. Note that while TAP 1.1 does not require the use of any particular minor version of the VOSI standard, the VOSI-tables resource in VOSI 1.1 provides important usability features, and implementors of TAP 1.1 are encouraged to support VOSI 1.1 or later. @@ -278,7 +279,15 @@ \section{Resources} \label{sec:resources} An implementation of a TAP service provides the following RESTful resources -under the base URL. +under the base URL. As of TAP-1.2, these endpoints are defined using OpenAPI +(in yaml syntax) and depend on OpenAPI components imported from other standards: +VOSI, UWS, and DALI. The OpenAPI specification is considered the definitive +specification of the TAP API; the text below is intended to clarify and provide +examples, but it may provide details that go beyond the OpenAPI description when +describing behaviour or content. +\todo{decide if the aggregate OpenAPI specification of TAP is static or dynamic aka +do we refer to a specific version of a component? or can services implement the latest +in some range?} \medskip \begin{inlinetable} @@ -288,7 +297,6 @@ \section{Resources} \sptablerule TAP-sync & /sync & must \\ TAP-async & /async & must \\ -VOSI-availability & service specific & must (must be anonymous) \\ VOSI-capabilities & /capabilities & must (must be anonymous) \\ VOSI-tables & /tables & should \\ DALI-examples & /examples & should \\ @@ -299,11 +307,11 @@ \section{Resources} A TAP service provides a single base URL with child resources for the various features in the table above. As required by DALI \citep{2017ivoa.spec.0517D}, all resources (including the optional VOSI-tables -resource) except the VOSI-availability must be siblings of the VOSI-capabilities resource. +resource) must be siblings of the VOSI-capabilities resource. -The fixed name resources above (async, sync, tables, and examples) are used for both anonymous and +The fixed name resources above (async, sync, capabilities, tables, and examples) are used for both anonymous and authenticated access to the service; the consequences of having a single base URL are detailed below in -section~\ref{sec:vosi-capabilities}. The VOSI-availability and VOSI-capabilities resources must allow anonymous access as they can be used by clients to determine if the service is available and which resources to use with +section~\ref{sec:vosi-capabilities}. The VOSI-capabilities resources must allow anonymous access as they can be used by clients to determine if the service is available and which resources to use with available security (authentication) methods. The web resource at the root of the tree represents the service as a whole. @@ -403,11 +411,6 @@ \subsection{TAP-async} data rows. Details on interacting with these resources are specified in the UWS standard; for examples specific to TAP see section~\ref{sec:examples} below. -\subsection{VOSI-availability} -\label{sec:vosi-availability} - -The use of the VOSI-availability resource is described in DALI. - \subsection{VOSI-capabilities} \label{sec:vosi-capabilities} @@ -460,7 +463,7 @@ \subsection{VOSI-capabilities} As a consequence of using a single base URL with fixed name child resources, all supported authentication methods must be able to co-exist on the same URL. -In the example above, the VOSI-availability and VOSI-capabilities interfaces are anonymous (no securityMethod). +In the example above, the VOSI-capabilities interface is anonymous (no securityMethod). The VOSI-tables interface is typically also anonymous-only, but in the example we show that it may also support anonymous and/or authenticated access. In general, clients that support authentication should be prepared to discover and use anonymous-only endpoints for some requests. @@ -1360,6 +1363,15 @@ \subsection{Example: DALI-examples Document} \appendix +\section{Changes from TAP-1.1 to TAP-1.2} +\begin{itemize} + +\item Core API specification is provided as using OpenAPI (yaml syntax). + +\item VOSI-availability removed. + +\end{itemize} + \section{Changes from TAP-1.0 to TAP-1.1} \subsection{General Improvements and Clarifications} diff --git a/openapi-linter b/openapi-linter new file mode 100755 index 0000000..c4b88e3 --- /dev/null +++ b/openapi-linter @@ -0,0 +1,8 @@ +#!/bin/bash + +if [[ "$1" == "-it" ]]; then + docker run -it --rm -v $PWD:/work:ro dshanley/vacuum /bin/bash +else + docker run --rm -v $PWD:/work:ro dshanley/vacuum lint --no-clip -d openapi.yaml +fi + diff --git a/openapi.yaml b/openapi.yaml new file mode 100644 index 0000000..87ea014 --- /dev/null +++ b/openapi.yaml @@ -0,0 +1,51 @@ +openapi: 3.1.0 +info: + title: Draft TAP-1.2 with user managed tables + version: "1.2" + description: | + This is the WD-TAP-1.2 API specification with components imported from + DALI and VOSI. + +servers: +- url: /tap-service + +tags: + - name: TAP Query + - name: VOSI Capabilities + - name: VOSI Table Metadata + - name: VOSI Table Operations + + +## any of the $ref values below can be URLs to external (standard) OpenAPI components +## openapi/dali and openapi/vosi need to be copied from ivoa-std/DALI and ivoa-std/VOSI +## openapi/uws is currently here in ivoa-std/TAP -- TBD +## openapi/tap is here in ivoa-std/TAP + +paths: + /sync: + $ref: ./openapi/tap/tap-sync.yaml + /async: + $ref: ./openapi/tap/tap-async.yaml + /async/{jobID}: + $ref: ./openapi/tap/tap-async-job.yaml + /async/{jobID}/phase: + $ref: ./openapi/tap/tap-async-job-phase.yaml + /async/{jobID}/results/result: + $ref: ./openapi/tap/tap-result.yaml + + /tables: + $ref: ./openapi/vosi/vosi-tableset.yaml + /tables/{name}: + $ref: ./openapi/vosi/vosi-table.yaml + + /capabilities: + $ref: ./openapi/vosi/vosi-capabilities.yaml + +# optional endpoint: allowed to return 404 or 405 + /table-ops: + $ref: ./openapi/vosi/vosi-table-ops.yaml + /table-ops/{jobID}: + $ref: ./openapi/vosi/vosi-table-ops-job.yaml + /table-ops/{jobID}/phase: + $ref: ./openapi/vosi/vosi-table-ops-job-phase.yaml + diff --git a/openapi/tap/tap-async-job-phase.yaml b/openapi/tap/tap-async-job-phase.yaml new file mode 100644 index 0000000..79ca646 --- /dev/null +++ b/openapi/tap/tap-async-job-phase.yaml @@ -0,0 +1,37 @@ +get: + operationId: tap-job-phase-get + tags: + - TAP Query + summary: get a UWS job phase + description: job phase resource + parameters: + - $ref: ../uws/uws-jobid-param.yaml + responses: + '200': + description: successful response + content: + text/plain: + schema: + type: string + example: "PENDING" +post: + operationId: tap-job-phase-update + tags: + - TAP Query + summary: change a UWS job phase + description: update job + parameters: + - $ref: ../uws/uws-jobid-param.yaml + - $ref: ../uws/uws-phase-change-param.yaml + responses: + '200': + description: phase change successful + '400': + $ref: ../uws/uws-responses.yaml#/invalid-phase-change + '401': + $ref: ../vosi/vosi-std-responses.yaml#/not-authenticated + '403': + $ref: ../vosi/vosi-std-responses.yaml#/permission-denied + '404': + $ref: ../vosi/vosi-std-responses.yaml#/not-found + diff --git a/openapi/tap/tap-async-job.yaml b/openapi/tap/tap-async-job.yaml new file mode 100644 index 0000000..aeba4de --- /dev/null +++ b/openapi/tap/tap-async-job.yaml @@ -0,0 +1,18 @@ +get: + operationId: tap-job-get + tags: + - TAP Query + summary: get a UWS job description + description: job resource + parameters: + - $ref: ../uws/uws-jobid-param.yaml + - $ref: ../uws/uws-wait-param.yaml + responses: + '200': + $ref: ../uws/uws-responses.yaml#/job + '401': + $ref: ../vosi/vosi-std-responses.yaml#/not-authenticated + '403': + $ref: ../vosi/vosi-std-responses.yaml#/permission-denied + '404': + $ref: ../vosi/vosi-std-responses.yaml#/not-found diff --git a/openapi/tap/tap-async.yaml b/openapi/tap/tap-async.yaml new file mode 100644 index 0000000..78ccf60 --- /dev/null +++ b/openapi/tap/tap-async.yaml @@ -0,0 +1,37 @@ +get: + operationId: query-job-list + tags: + - TAP Query + summary: list queries (UWS jobs) + description: TAP asynchronous query endpoint (list UWS Jobs) + parameters: + - $ref: ../uws/uws-phase-param.yaml + - $ref: ../uws/uws-after-param.yaml + - $ref: ../uws/uws-last-param.yaml + responses: + '200': + $ref: ../uws/uws-responses.yaml#/job-listing + '401': + $ref: ../vosi/vosi-std-responses.yaml#/not-authenticated + '403': + $ref: ../vosi/vosi-std-responses.yaml#/permission-denied +post: + operationId: query-job-create + tags: + - TAP Query + summary: create async query (UWS job) + description: TAP asynchronous query endpoint (create UWS Job) + parameters: + - $ref: ./tap-lang.yaml + - $ref: ./tap-query.yaml + - $ref: ./tap-format.yaml + - $ref: ../dali/dali-maxrec.yaml + - $ref: ../dali/dali-responseformat.yaml + - $ref: ../dali/dali-upload.yaml + responses: + '303': + $ref: ../uws/uws-responses.yaml#/created + '401': + $ref: ../vosi/vosi-std-responses.yaml#/not-authenticated + '403': + $ref: ../vosi/vosi-std-responses.yaml#/permission-denied diff --git a/openapi/tap/tap-format.yaml b/openapi/tap/tap-format.yaml new file mode 100644 index 0000000..40c253b --- /dev/null +++ b/openapi/tap/tap-format.yaml @@ -0,0 +1,5 @@ +name: FORMAT +in: query +description: "supported for backwards compatibility to 1.0 (see: RESPONSEFORMAT)" +schema: + type: string diff --git a/openapi/tap/tap-lang.yaml b/openapi/tap/tap-lang.yaml new file mode 100644 index 0000000..99d3bb4 --- /dev/null +++ b/openapi/tap/tap-lang.yaml @@ -0,0 +1,7 @@ +name: LANG +in: query +description: specify the query language used in the QUERY parameter +required: true +schema: + type: string + example: ADQL diff --git a/openapi/tap/tap-query.yaml b/openapi/tap/tap-query.yaml new file mode 100644 index 0000000..934213a --- /dev/null +++ b/openapi/tap/tap-query.yaml @@ -0,0 +1,7 @@ +name: QUERY +in: query +description: specify the query +required: true +schema: + type: string + example: select * from tap_schema.tables diff --git a/openapi/tap/tap-responses.yaml b/openapi/tap/tap-responses.yaml new file mode 100644 index 0000000..14ccd46 --- /dev/null +++ b/openapi/tap/tap-responses.yaml @@ -0,0 +1,17 @@ +# UWS create job response +invalid-query: + description: the requested TAP query job is invalid; this is usually the ADQL query + content: + application/x-votable+xml: + schema: + type: object + xml: + name: VOTABLE + namespace: http://www.ivoa.net/xml/VOTable/v1.3 + example: | + + + Table 'Y' is not found in TAP schema + + diff --git a/openapi/tap/tap-result.yaml b/openapi/tap/tap-result.yaml new file mode 100644 index 0000000..7b70047 --- /dev/null +++ b/openapi/tap/tap-result.yaml @@ -0,0 +1,52 @@ +get: + operationId: tap-result + tags: + - TAP Query + summary: async query result + description: download async query result for a completed job + parameters: + - $ref: ../uws/uws-jobid-param.yaml + responses: + '200': + $ref: '#/components/schemas/queryResult' + '303': + description: redirect to the stored result (optional) + '401': + $ref: ../vosi/vosi-std-responses.yaml#/not-authenticated + '403': + $ref: ../vosi/vosi-std-responses.yaml#/permission-denied + '404': + $ref: ../vosi/vosi-std-responses.yaml#/not-found + +components: + schemas: + queryResult: + description: query result (table) + content: + application/x-votable+xml: + schema: + type: object + xml: + name: VOTABLE + namespace: http://www.ivoa.net/xml/VOTable/v1.3 + example: | + + + + + schema name for reference to tap_schema.schemas + + + lists the utypes of schemas in the tableset + + + describes schemas in the tableset + + + recommended sort order when listing schemas + + +
+ + diff --git a/openapi/tap/tap-sync.yaml b/openapi/tap/tap-sync.yaml new file mode 100644 index 0000000..d385a2d --- /dev/null +++ b/openapi/tap/tap-sync.yaml @@ -0,0 +1,49 @@ +get: + operationId: sync-query-job-get + tags: + - TAP Query + summary: execute sync query + description: TAP synchronous query endpoint + parameters: + - $ref: ./tap-lang.yaml + - $ref: ./tap-query.yaml + - $ref: ./tap-format.yaml + - $ref: ../dali/dali-maxrec.yaml + - $ref: ../dali/dali-responseformat.yaml + - $ref: ../dali/dali-upload.yaml + responses: + '200': + $ref: './tap-result.yaml#/components/schemas/queryResult' + '303': + description: redirect to the job result (optional) + '400': + $ref: ./tap-responses.yaml#/invalid-query + '401': + $ref: ../vosi/vosi-std-responses.yaml#/not-authenticated + '403': + $ref: ../vosi/vosi-std-responses.yaml#/permission-denied +post: + operationId: sync-query-job-create + tags: + - TAP Query + summary: execute sync query + description: TAP synchronous query endpoint + parameters: + - $ref: ./tap-lang.yaml + - $ref: ./tap-query.yaml + - $ref: ./tap-format.yaml + - $ref: ../dali/dali-maxrec.yaml + - $ref: ../dali/dali-responseformat.yaml + - $ref: ../dali/dali-upload.yaml + responses: + '200': + $ref: './tap-result.yaml#/components/schemas/queryResult' + '303': + description: redirect to the job result (optional) + '400': + $ref: ./tap-responses.yaml#/invalid-query + '401': + $ref: ../vosi/vosi-std-responses.yaml#/not-authenticated + '403': + $ref: ../vosi/vosi-std-responses.yaml#/permission-denied + diff --git a/rsync-openapi-components.sh b/rsync-openapi-components.sh new file mode 100755 index 0000000..f6a4c5e --- /dev/null +++ b/rsync-openapi-components.sh @@ -0,0 +1,19 @@ +#!/bin/bash + +# source locations for external components +VOSI=../VOSI +DALI=../DALI +test -d $VOSI || VOSI=../VOSI.git +test -d $DALI || DALI=../DALI.git + +ARGS="$1 -avc --delete" + +echo "found VOSI: $VOSI" +rsync $ARGS $VOSI/openapi/vosi openapi/ + +echo "get UWS from $VOSI" +rsync $ARGS $VOSI/openapi/uws openapi/ + +echo "found DALI: $DALI" +rsync $ARGS $DALI/openapi/dali openapi/ + diff --git a/sample-capability.xml b/sample-capability.xml index 1f32794..45f11be 100644 --- a/sample-capability.xml +++ b/sample-capability.xml @@ -27,14 +27,6 @@ - - - - https://example.net/srv/availability - - - -