java -jar causal-rest-api.jar
+java -jar causal-rest-api.jar
-
API Usage and Examples
In the following sections, we'll demonstrate the API usage with examples using the API server that is running on Pittsburgh Super Computing. The API base URI is https://ccd4.vm.bridges.psc.edu/ccd-api.
This API requires user to be authenticated. Before using this API, the user will need to go to Causal-Web App and create an account.
@@ -241,13 +266,11 @@ Getting JSON Web Token(JWT)
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/jwt
-
In basic auth, the user provides the username and password, which the HTTP client concatenates (username + ":" + password), and base64 encodes it. This encoded string is then sent using a Authorization header with the "Basic" schema. For instance user email demo@pitt.edu whose password is 123.
POST /ccd-api/jwt HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Basic ZGVtb0BwaXR0LmVkdToxMjM=
-
Once the request is processed successfully, the user ID together with a JWT will be returned in the response for further API queries.
{
"userId": 22,
@@ -262,7 +285,6 @@ Getting JSON Web Token(JWT)
]
}
-
We'll need to use this userId in the URI path of all subsequent requests. And this jwt expires in 3600 seconds(1 hour), so the API consumer will need to request for another JWT otherwise the API query to other API endpoints will be denied. And this JWT will need to be sent via the HTTP Authorization header as well, but using the Bearer schema.
The wallTime field is designed for users who want to specify the the maximum CPU time when Slurm handles the jobs on PSC. Normally, a job is expected to finish before the specified maximum walltime. After the walltime reaches the maximum, the job terminates regardless whether the job processes are still running or not. In this example, you can pick 1 hour, 3 or 6 hours as the wallTime.
Note: querying the JWT endpoint again before the current JWT expires will generate a new JWT, which makes the old JWT expired automatically. And this newly generated JWT will be valid in another hour unless there's another new JWT being queried.
@@ -280,7 +302,6 @@ Upload small data file
API Endpoint URI pattern:
POST https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/dataset/upload
-
This is a multipart file upload via an HTML form, and the client is required to use name="file" to name their file upload field in their form.
Generated HTTP request code example:
POST /ccd-api/22/dataset/upload HTTP/1.1
@@ -295,9 +316,8 @@ Upload small data file
----WebKitFormBoundary7MA4YWxkTrZu0gW
-
If the Authorization header is not provided, the response will look like this:
-{
+{
"timestamp": 1465414501443,
"status": 401,
"error": "Unauthorized",
@@ -305,9 +325,8 @@ Upload small data file
"path": "/22/dataset/upload"
}
-
This POST request will upload the dataset file to the target server location and add corresponding records into database. And the response will contain the following pieces:
-{
+{
"id": 6,
"name": "Lung-tetrad_hv.txt",
"creationTime": 1466622267000,
@@ -322,13 +341,11 @@ Upload small data file
}
}
-
The prior knowledge file upload uses a similar API endpoint:
POST https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/priorknowledge/upload
-
Due to there's no need to summarize a prior knowledge file, the response of a successful prior knowledge file upload will look like:
-{
+{
"id": 6,
"name": "Lung-tetrad_hv.txt",
"creationTime": 1466622267000,
@@ -337,7 +354,6 @@ Upload small data file
"md5checkSum": "ugdb7511rt293d29ke3055d9a7b46c9k"
}
-
Resumable data file upload
In addition to the regular file upload described in Example 6, we also provide the option of stable and resumable large file upload. It requires the client side to have a resumable upload implementation. We currently support client integrated with Resumable.js, whihc provides multiple simultaneous, stable
and resumable uploads via the HTML5 File API. You can also create your own client as long as al the following parameters are set correctly.
@@ -346,7 +362,6 @@ Resumable data file upload
POST https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/chunkupload
-
In this example, the data file is splited into 3 chunks. The upload of each chunk consists of a GET request and a POST request. To handle the state of upload chunks, a number of extra parameters are sent along with all requests:
resumableChunkNumber: The index of the chunk in the current upload. First chunk is 1 (no base-0 counting here).Resumable data file upload
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
-
This GET request checks if the data chunk is already on the server side. If the target file chunk is not found on the server, the client will issue a POST request to upload the actual data.
Generated HTTP request code example:
POST /ccd-api/22/chunkupload HTTP/1.1
@@ -415,26 +429,22 @@ Resumable data file upload
------WebKitFormBoundaryMFjgApg56XGyeTnZ--
-
Each chunk upload POST will get a 200 status code from response if everything works fine.
And finally the md5checkSum string of the reassemabled file will be returned once the whole file has been uploaded successfully. In this example, the POST request that uploads the third chunk will response this:
b1db7511ee293d297e3055d9a7b46c5e
-
List all dataset files of a user
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/dataset
-
Generated HTTP request code example:
GET /ccd-api/22/dataset HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
Accept: application/json
-
A JSON formatted list of all the input dataset files that are associated with user 22 will be returned.
-[
+[
{
"id": 8,
"name": "data_small.txt",
@@ -479,7 +489,6 @@ List all dataset files of a user
}
]
-
You can also specify the response format as XML in your request
Generated HTTP request code example:
GET /ccd-api/22/dataset HTTP/1.1
@@ -487,9 +496,8 @@ List all dataset files of a user
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
Accept: application/xml
-
And the response will look like this:
-<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
+<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<datasetFileDTOes>
<datasetFile>
<id>8</id>
@@ -535,21 +543,18 @@ List all dataset files of a user
</datasetFile>
</datasetFileDTOes>
-
Form the above output, we can also tell that data file with ID 10 doesn't have all the fileSummary field values set, we'll cover this in the dataset summarization section.
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/dataset/{id}
-
Generated HTTP request code example:
GET /ccd-api/22/dataset/8 HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
-
And the resulting response looks like this:
-{
+{
"id": 8,
"name": "data_small.txt",
"creationTime": 1467132449000,
@@ -564,18 +569,15 @@
-
Delete physical dataset file and all records from database for a given file ID
API Endpoint URI pattern:
DELETE https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/dataset/{id}
-
Generated HTTP request code example:
DELETE /ccd-api/22/dataset/8 HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
-
And this will result a HTTP 204 No Content status in response on success, which means the server successfully processed the deletion request but there's no content to response.
Summarize dataset file
So from the first example we can tell that file with ID 10 doesn't have variableType, fileDelimiter, numOfRows, and numOfColumns specified under fileSummary. Among these attributes, variableTypeandfileDelimiter` are the ones that users will need to provide during this summarization process.
@@ -605,7 +607,6 @@ Summarize dataset file
API Endpoint URI pattern:
POST https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/dataset/summarize
-
Generated HTTP request code example:
POST /ccd-api/22/dataset/summarize HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
@@ -618,9 +619,8 @@ Summarize dataset file
"fileDelimiter": "comma"
}
-
This POST request will summarize the dataset file and generate a response (JSON or XML) like below:
-{
+{
"id": 10,
"name": "large-data.txt",
"creationTime": 1467134048000,
@@ -635,21 +635,18 @@ Summarize dataset file
}
}
-
List all prior knowledge files of a given user
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/priorknowledge
-
Generated HTTP request code example:
GET /ccd-api/22/priorknowledge HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
Accept: application/json
-
A JSON formatted list of all the input dataset files that are associated with user 22 will be returned.
-[
+[
{
"id": 9,
"name": "data_small.prior",
@@ -668,20 +665,17 @@ List all prior knowledge
}
]
-
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/priorknowledge/{id}
-
Generated HTTP request code example:
GET /ccd-api/22/priorknowledge/9 HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
-
And the resulting response looks like this:
-{
+{
"id": 9,
"name": "data_small.prior",
"creationTime": 1467132449000,
@@ -690,18 +684,15 @@
-
Delete physical prior knowledge file and all records from database for a given file ID
API Endpoint URI pattern:
DELETE https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/priorknowledge/{id}
-
Generated HTTP request code example:
DELETE /ccd-api/22/priorknowledge/9 HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
-
And this will result a HTTP 204 No Content status in response on success, which means the server successfully processed the deletion request but there's no content to response.
2. Causal Discovery
Once the data file is uploaded and summaried, you can start running a Causal Discovery Algorithm on the uploaded data file.
@@ -709,14 +700,12 @@ List all the availab
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/algorithms
-
Generated HTTP request code example:
GET /ccd-api/22/algorithms HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
-
-[
+[
{
"id": 1,
"name": "FGESc",
@@ -739,7 +728,6 @@ List all the availab
}
]
-
Currently we support "FGES continuous", "FGES discrete", "GFCI continuous", and "GFCI discrete". They also share a common JSON structure as of their input, for example:
@@ -1021,7 +1009,6 @@ Add a n
API Endpoint URI pattern:
POST https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/jobs/FGESc
-
Generated HTTP request code example:
POST /ccd-api/22/jobs/FGESc HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
@@ -1050,7 +1037,6 @@ Add a n
]
}
-
In this example, we are running the "FGES continuous" algorithm on the file of ID 8. We also set the wallTime as 1 hour. And this call will return the job info with a 201 Created response status code.
{
"id": 5,
@@ -1061,13 +1047,11 @@ Add a n
"errorResultFileName": "error_FGESc_data_small.txt_1472742564353.txt"
}
-
From this response we can tell that the job ID is 5, and the result file name will be FGESc_data_small.txt_1472742564353.txt if everything goes well. If something is wrong an error result file with name error_FGEsc_data_small.txt_1472742564353.txt will be created.
When you need to run "FGES discrete", just send the request to a different endpont URI:
API Endpoint URI pattern:
POST https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/jobs/FGESd
-
Generated HTTP request code example:
POST /ccd-api/22/jobs/FGESd HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
@@ -1091,12 +1075,10 @@ Add a n
}
}
-
List all running jobs
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/jobs
-
Generated HTTP request code example:
GET /ccd-api/22/jobs/ HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
@@ -1104,9 +1086,8 @@ List all running jobs
Content-Type: application/json
-
Then you'll see the information of all jobs that are currently running:
-[
+[
{
"id": 32,
"algorithmName": "FGESc",
@@ -1119,47 +1100,40 @@ List all running jobs
}
]
-
Check the job status for a given job ID
Once the new job is submitted, it takes time and resources to run the algorithm on the server. During the waiting, you can check the status of a given job ID:
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/jobs/{id}
-
Generated HTTP request code example:
GET /ccd-api/22/jobs/32 HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
-
This will either return "Pending" or "Completed".
Cancel a running job
Sometimes you may want to cancel a submitted job.
API Endpoint URI pattern:
DELETE https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/jobs/{id}
-
Generated HTTP request code example:
DELETE /ccd-api/22/jobs/8 HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
-
This call will response either "Job 8 has been canceled" or "Unable to cancel job 8". It's not guranteed that the system can always cencal a job successfully.
3. Result Management
List all result files generated by the algorithm
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/results
-
Generated HTTP request code example:
GET /ccd-api/22/results HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
-
The response to this request will look like this:
-[
+[
{
"name": "FGESc_sim_data_20vars_100cases.csv_1466171729046.txt",
"creationTime": 1466171732000,
@@ -1174,21 +1148,18 @@ List all result files
}
]
-
Download a specific result file generated by the algorithm based on file name
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/results/{result_file_name}
-
Generated HTTP request code example:
GET /ccd-api/22/results/FGESc_data_small.txt_1466172140585.txt HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
-
On success, you will get the result file back as text file content. If there's a typo in file name of the that file doesn't exist, you'll get either a JSON or XML message based on the accept header in your request:
The response to this request will look like this:
-{
+{
"timestamp": 1467210996233,
"status": 404,
"error": "Not Found",
@@ -1196,13 +1167,11 @@ Compare algorithm result files
Since we can list all the algorithm result files, based on the results, we can also choose multiple files and run a comparison.
API Endpoint URI pattern:
POST https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/results/compare
-
The request body is a JSON that contains an array of result files to be compared.
Generated HTTP request code example:
POST /ccd-api/22/results/compare HTTP/1.1
@@ -1216,7 +1185,6 @@ Compare algorithm result files
]
}
-
When you specify multiple file names, use the !! as a delimiter. This request will generate a result comparison file with the following content (shortened version):
FGESc_sim_data_20vars_100cases.csv_1466171729046.txt FGESc_data_small.txt_1467305104859.txt
Edges In All Same End Point
@@ -1233,21 +1201,18 @@ Compare algorithm result files
SERPINI2,FGG 0 0
hsa_miR_451,hsa_miR_136_ 0 0
-
From this comparison, you can see if the two algorithm graphs have common edges and endpoints.
List all the comparison files
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/results/comparisons
-
Generated HTTP request code example:
GET /ccd-api/22/results/comparisons HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
-
The response will show a list of comparison files:
-[
+[
{
"name": "result_comparison_1467385923407.txt",
"creationTime": 1467385923000,
@@ -1268,18 +1233,15 @@ List all the comparison files
}
]
-
Download a specific comparison file based on file name
API Endpoint URI pattern:
GET https://ccd4.vm.bridges.psc.edu/ccd-api/{userId}/results/comparisons/{comparison_file_name}
-
Generated HTTP request code example:
GET /ccd-api/22/results/comparisons/result_comparison_1467388042261.txt HTTP/1.1
Host: ccd4.vm.bridges.psc.edu
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJodHRwczovL2Nsb3VkLmNjZC5waXR0LmVkdS8iLCJuYW1lIjoiemh5MTkiLCJleHAiOjE0NzU4NTA2NzY4MDQsImlhdCI6MTQ3NTg0NzA3NjgwNH0.8azVEoNPfETczXb-vn7dfyDd98eRt7iiLBXehGpPGzY
-
Then it returns the content of that comparison file (shorted version):
FGESc_sim_data_20vars_100cases.csv_1466171729046.txt FGESc_data_small.txt_1467305104859.txt
Edges In All Same End Point
@@ -1297,29 +1259,21 @@ Download a speci
-
-