From 84f412503df96c56551d418f34ffb802817ea66b Mon Sep 17 00:00:00 2001 From: Vladimir Sudilovsky Date: Wed, 22 Jul 2015 14:22:13 +0000 Subject: [PATCH 01/39] README: andycasey/ads -> adsabs/ads --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 32aedba..0a1d24f 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,9 @@ Announcements and discussion related to the Developer API are available via the The unofficial python client for the API can be found here: -* https://github.com/andycasey/ads +* https://github.com/adsabs/ads + +(Note that this is a temporary fork of andycasey's client; please install from the git repo and not pypi if you use this fork) ## Access From 10de3e51ecf52cede72897a59a97242e51a7bbef Mon Sep 17 00:00:00 2001 From: Alberto Accomazzi Date: Wed, 23 Sep 2015 16:36:10 -0400 Subject: [PATCH 02/39] Update README.md --- README.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 0a1d24f..83f7f11 100644 --- a/README.md +++ b/README.md @@ -18,11 +18,10 @@ Announcements and discussion related to the Developer API are available via the ## Clients -The unofficial python client for the API can be found here: +The unofficial python client for the API is maintained by Andy Casey and can be found here: -* https://github.com/adsabs/ads +* https://github.com/andycasey/ads -(Note that this is a temporary fork of andycasey's client; please install from the git repo and not pypi if you use this fork) ## Access @@ -33,7 +32,7 @@ To obtain access to the ADS Developer API you must do two things: 1. Create an account and log in to the latest version of the [ADS](https://ui.adsabs.harvard.edu). 1. Push the "Generate a new key" button under the [user profile](https://ui.adsabs.harvard.edu/#user/settings/token) -All API requests must pass your token in an `Authorization: Bearer ` HTTP header. +All API requests must pass your token in an `Authorization: Bearer ` HTTP header (where is the key you just generated). ## Access Settings From 163c1dbdbc56d5088e27e972edad0464c4cdbc2e Mon Sep 17 00:00:00 2001 From: Alberto Accomazzi Date: Wed, 23 Sep 2015 16:43:04 -0400 Subject: [PATCH 03/39] Update README.md --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index 83f7f11..bbe4087 100644 --- a/README.md +++ b/README.md @@ -22,6 +22,9 @@ The unofficial python client for the API is maintained by Andy Casey and can be * https://github.com/andycasey/ads +Geert Barentsen has built an application to support the Kepler publication database which uses the ADS API to discover relevant papers: + +* https://github.com/KeplerGO/kpub ## Access From e112fb24b0934de637d7f4c6560b81c6e46e548b Mon Sep 17 00:00:00 2001 From: Roman Chyla Date: Mon, 5 Oct 2015 15:11:09 -0400 Subject: [PATCH 04/39] Update README.md --- README.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index bbe4087..34c287a 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,10 @@ To obtain access to the ADS Developer API you must do two things: 1. Create an account and log in to the latest version of the [ADS](https://ui.adsabs.harvard.edu). 1. Push the "Generate a new key" button under the [user profile](https://ui.adsabs.harvard.edu/#user/settings/token) -All API requests must pass your token in an `Authorization: Bearer ` HTTP header (where is the key you just generated). +All API requests must pass your token in an `Authorization: Bearer ` HTTP header (where is the key you just generated), e.g. + + curl -H 'Authorization: Bearer ' 'https://api.adsabs.harvard.edu/v1/search/query?q=star' + ## Access Settings From 347ac7d316b42b56c763c5aa896efb16a5ec3a81 Mon Sep 17 00:00:00 2001 From: Alberto Accomazzi Date: Mon, 5 Oct 2015 17:48:08 -0400 Subject: [PATCH 05/39] Update search.md Removed note about aqp parser for fq --- search.md | 1 - 1 file changed, 1 deletion(-) diff --git a/search.md b/search.md index 55250dc..3123e9b 100644 --- a/search.md +++ b/search.md @@ -89,7 +89,6 @@ Assuming your access token is `my_token`, all queries can be executed by any HTT # Search for "dark energy", filter by author, sort by citation count ?q=dark+energy&fq=author:"Civano,+F"&sort=citation_count+desc - (@roman fq won't work in this example until !aqp is the default) # Same search but only return *bibcode* and *property* values ?q=dark+energy&fq=author:"Civano,+F"&sort=citation_count+desc&fl=bibcode,property From ecd3ad899d773f44a9fdce0aa3579a17304c8407 Mon Sep 17 00:00:00 2001 From: Jonny Elliott Date: Sun, 18 Oct 2015 14:16:17 -0400 Subject: [PATCH 06/39] Update README.md Closes #16. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 34c287a..3660ad3 100644 --- a/README.md +++ b/README.md @@ -28,7 +28,7 @@ Geert Barentsen has built an application to support the Kepler publication datab ## Access -Access to the ADS data holdings is regulated by the the ADS terms of use, as described in the ADS [Terms of Use](http://doc.adsabs.harvard.edu/abs_doc/help_pages/overview.html#use). +Access to the ADS data holdings is regulated by the the ADS terms of use, as described in the ADS [Terms of Use](http://adsabs.github.io/help/terms/). To obtain access to the ADS Developer API you must do two things: From ac6c631811de043eb230af5a5e29e9a87de8d48b Mon Sep 17 00:00:00 2001 From: Edwin Henneken Date: Mon, 19 Oct 2015 11:14:27 -0400 Subject: [PATCH 07/39] Documentation for metrics --- metrics.md | 252 ++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 251 insertions(+), 1 deletion(-) diff --git a/metrics.md b/metrics.md index 71ab335..77e95eb 100644 --- a/metrics.md +++ b/metrics.md @@ -4,4 +4,254 @@ To retrieve metrics data for a particular search result set append `/metrics/` t ``` https://api.adsabs.harvard.edu/v1/metrics -``` \ No newline at end of file +``` + +If you do a request from the command line like + + curl -H "Authorization: Bearer " -H "Content-Type: application/json" -X POST -d '{"bibcodes":["1980ApJS...44..137K","1980ApJS...44..489B"]}' https://api.adsabs.harvard.edu/v1/metrics + +and you should get back all metrics data. If you access the API using the Python `requests` module, you would do something like + + r = requests.post(queryURL, data=json.dumps(payload), headers=headers) + +where + + payload = {'bibcodes': [bibcode1, bibcode2, ...]} + headers = {'Content-type': 'application/json', 'Accept': 'text/plain', 'Authorization':'Bearer %s' % token} + +and `token` is your API token. + +You can also request a specific set of metrics by adding the `types` parameter in the JSON header. For example: + + curl -H "Authorization: Bearer " -H "Content-Type: application/json" -X POST -d '{"bibcodes":["1980ApJS...44..137K","1980ApJS...44..489B"],'types':['basic']}' https://api.adsabs.harvard.edu/v1/metrics + +which will return just the basic statistics for the set of bibcodes. The following values are allowed for the `types` list: + + * 'basic': publication and usage stats (all papers, and just refereed papers) + * 'citations': citation stats + * 'indicators': indicators, like the h-index, g-index, m-index etc etc + * 'histograms': publication, citation, reads and downloads histograms + * 'timeseries': time series for a set of indicators + +If you just want a specific histogram, you can add a `histograms` parameter to the JSON header, specifying that histogram. For example: + + curl -H "Authorization: Bearer " -H "Content-Type: application/json" -X POST -d '{"bibcodes":["1980ApJS...44..137K","1980ApJS...44..489B"],'types':['histograms'], 'histograms':['publications']}' https://api.adsabs.harvard.edu/v1/metrics + +to get just the publications histogram. The service returns JSON with attributes determined by the contents of the `types` list (no such list means that everything gets returned). It always contains an attribute 'skipped bibcodes', which is a list of bibcodes for which no metrics data could be found. The following overview shows the JSON attributes returned by each type: + + * 'basic': 'basic stats' and 'basic stats refereed' + * 'citations': 'citation stats' and 'citation stats refereed' + * 'indicators': 'indicators' and 'indicators refereed' + * 'histograms': 'histograms' + * 'timeseries': 'time series' + +Each of these attributes contains specific data elements. The following overview shows what each section contains (the phrases listed are the attribute names). + +### 'basic stats' and 'basic stats refereed' + * number of papers + * normalized paper count + * median number of reads + * average number of reads + * total number of reads + * normalized number of reads + * median number of downloads + * average number of downloads + * total number of downloads + * normalized number of downloads + +### 'citation stats' and 'citation stats refereed' + * number of citing papers + * median number of citations + * average number of citations + * total number of citations + * normalized number of citations + * median number of refereed citations + * average number of refereed citations + * total number of refereed citations + * normalized number of refereed citations + * number of self-citations + +### 'indicators' + * i10 + * i100 + * h + * m + * g + * read10 + * tori + +### 'histograms' +For all histograms, the actual histogram values are stored as follows: + + { + 'histogram name': {year: value, year: value, ...} + } + +where values can be either integers or floats, depending on the type of histograms. The histogram name is an attribute of the histogram type. For example, the `publications` type has 4 separate histograms: `all publications`, `refereed publications`, `all publications normalized` and `refereed publications normalized`, which are organized as follows in the JSON output: + + { + 'histograms': { + 'publications': { + 'all publications': {}, + 'refereed publications': {}, + 'all publications normalized': {}, + 'refereed publications normalized': {}, + + } + } + } + +The following overview lists for all histogram types the names of the actual histograms stored in them: + + * 'publications': 'all publications', 'refereed publications', 'all publications normalized', 'refereed publications normalized' + * 'reads': 'all reads', 'refereed reads', 'all reads normalized', 'refereed reads normalized' + * 'downloads': 'all downloads', 'refereed downloads', 'all downloads normalized', 'refereed downloads normalized' + * 'citations': 'refereed to refereed', 'nonrefereed to refereed', 'refereed to nonrefereed', 'nonrefereed to nonrefereed', 'refereed to refereed normalized', 'nonrefereed to refereed normalized', 'refereed to nonrefereed normalized', 'nonrefereed to nonrefereed normalized' + +Note that 'refereed reads' means: 'number of reads for the refereed publications in the set'. + +### 'timeseries' +The `timeseries` attribute contains time series data for: `h`, `g`, `i10`, `i100`, `read10` and `tori`. The output is structured as + + { + 'time series': { + 'h': { values }, + 'g': { values }, + 'i10': { values }, + 'i100': { values }, + 'read10': { values }, + 'tori': { values }, + + } + } + +where the 'values' are organized in the same way as for the histograms: + + { + 'indicator name': {year: value, year: value, ...} + } + +## Everything together +So, returning everything, the output looks like + + { + 'skipped bibcodes': [ ... ], + 'basic stats': + { + 'number of papers': 1, + 'normalized paper count': 0.1, + 'median number of reads': 1.1, + 'average number of reads': 1.1, + 'total number of reads': 1, + 'normalized number of reads': 0.1, + 'median number of downloads': 1.1, + 'average number of downloads': 1.1, + 'total number of downloads': 1, + 'normalized number of downloads': 0.1, + }, + 'basic stats refereed': + { + 'number of papers': 1, + 'normalized paper count': 0.1, + 'median number of reads': 1.1, + 'average number of reads': 1.1, + 'total number of reads': 1, + 'normalized number of reads': 0.1, + 'median number of downloads': 1.1, + 'average number of downloads': 1.1, + 'total number of downloads': 1, + 'normalized number of downloads': 0.1, + }, + 'citation stats': + { + 'number of citing papers': 1, + 'median number of citations': 0.1, + 'average number of citations': 1.1, + 'total number of citations': 1, + 'normalized number of citations': 0.1, + 'median number of refereed citations': 0.1, + 'average number of refereed citations': 1.1, + 'total number of refereed citations': 1, + 'normalized number of refereed citations': 0.1, + 'number of self-citations': 1, + }, + 'citations stats refereed': + { + 'number of citing papers': 1, + 'median number of citations': 0.1, + 'average number of citations': 1.1, + 'total number of citations': 1, + 'normalized number of citations': 0.1, + 'median number of refereed citations': 0.1, + 'average number of refereed citations': 1.1, + 'total number of refereed citations': 1, + 'normalized number of refereed citations': 0.1, + 'number of self-citations': 1, + }, + 'indicators': + { + 'i10': 1, + 'i100': 1, + 'h': 1, + 'm': 1.1, + 'g': 1, + 'read10': 0.1, + 'tori': 1.1, + 'riq': 1, + }, + 'indicators refereed': + { + 'i10': 1, + 'i100': 1, + 'h': 1, + 'm': 1.1, + 'g': 1, + 'read10': 0.1, + 'tori': 1.1, + 'riq': 1, + }, + 'histograms': + { + 'publications': { + 'all publications': { values }, + 'refereed publications': { values }, + 'all publications normalized': { values }, + 'refereed publications normalized': { values }, + }, + 'reads': { + 'all reads': { values }, + 'refereed reads': { values }, + 'all reads normalized': { values }, + 'refereed reads normalized': { values }, + }, + 'downloads': { + 'all downloads': { values }, + 'refereed downloads': { values }, + 'all downloads normalized': { values }, + 'refereed downloads normalized': { values }, + }, + 'citations': { + 'refereed to refereed': { values }, + 'nonrefereed to refereed': { values }, + 'refereed to nonrefereed': { values }, + 'nonrefereed to nonrefereed': { values }, + 'refereed to refereed normalized': { values }, + 'nonrefereed to refereed normalized': { values }, + 'refereed to nonrefereed normalized': { values }, + 'nonrefereed to nonrefereed normalized': { values }, + }, + }, + 'time series': + { + 'h': { values }, + 'g': { values }, + 'i10': { values }, + 'i100': { values }, + 'read10': { values }, + 'tori': { values }, + } + } + +where, as before, the 'values' have the form + + {year: value, year: value, ...} \ No newline at end of file From b2dfef11a2398bd7e24d938c48e081a77a4f9d21 Mon Sep 17 00:00:00 2001 From: Edwin Henneken Date: Mon, 19 Oct 2015 11:24:34 -0400 Subject: [PATCH 08/39] Documentation for export --- export.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/export.md b/export.md index b7fe439..0966218 100644 --- a/export.md +++ b/export.md @@ -3,4 +3,20 @@ ``` https://api.adsabs.harvard.edu/v1/export/:format ``` +Currently, our API supports export in the following formats: AASTeX, BibTeX and EndNote. To get e.g. BibTeX for a set of records you do a POST request to the endpoint + https://api.adsabs.harvard.edu/v1/export/bibtex + +with in the POST header + + {"bibcode":["","", ...]} + +Using `curl`, to get the BibTeX for e.g. the record with bibcode `2015ApJS..219...21Z`, you would do + + curl -H "Authorization: Bearer " -H "Content-Type: application/json" -X POST -d '{"bibcodes":["2015ApJS..219...21Z"]}' https://api.adsabs.harvard.edu/v1/export/bibtex + +and the API then responds in JSON with + + {u'msg': u'Retrieved 1 abstracts, starting with number 1. Total number selected: 1.', u'export': u'@ARTICLE{2015ApJS..219...21Z,\n author = {{Zhang}, M. and {Fang}, M. and {Wang}, H. and {Sun}, J. and \n\t{Wang}, M. and {Jiang}, Z. and {Anathipindika}, S.},\n title = "{A Deep Near-infrared Survey toward the Aquila Molecular Cloud. I. Molecular Hydrogen Outflows}",\n journal = {\\apjs},\narchivePrefix = "arXiv",\n eprint = {1506.08372},\n primaryClass = "astro-ph.SR",\n keywords = {infrared: ISM, ISM: jets and outflows, shock waves, stars: formation, stars: winds, outflows},\n year = 2015,\n month = aug,\n volume = 219,\n eid = {21},\n pages = {21},\n doi = {10.1088/0067-0049/219/2/21},\n adsurl = {http://adsabs.harvard.edu/abs/2015ApJS..219...21Z},\n adsnote = {Provided by the SAO/NASA Astrophysics Data System}\n}\n\n'} + +The get AASTeX back, you use `aastex` as format parameter, and `endnote` for EndNote. \ No newline at end of file From 6239f4d3ca201ced2b48395fa36ae3aa8254d961 Mon Sep 17 00:00:00 2001 From: Edwin Henneken Date: Thu, 22 Oct 2015 13:46:29 -0400 Subject: [PATCH 09/39] Typo correction --- export.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/export.md b/export.md index 0966218..30af052 100644 --- a/export.md +++ b/export.md @@ -13,10 +13,10 @@ with in the POST header Using `curl`, to get the BibTeX for e.g. the record with bibcode `2015ApJS..219...21Z`, you would do - curl -H "Authorization: Bearer " -H "Content-Type: application/json" -X POST -d '{"bibcodes":["2015ApJS..219...21Z"]}' https://api.adsabs.harvard.edu/v1/export/bibtex + curl -H "Authorization: Bearer " -H "Content-Type: application/json" -X POST -d '{"bibcode":["2015ApJS..219...21Z"]}' https://api.adsabs.harvard.edu/v1/export/bibtex and the API then responds in JSON with {u'msg': u'Retrieved 1 abstracts, starting with number 1. Total number selected: 1.', u'export': u'@ARTICLE{2015ApJS..219...21Z,\n author = {{Zhang}, M. and {Fang}, M. and {Wang}, H. and {Sun}, J. and \n\t{Wang}, M. and {Jiang}, Z. and {Anathipindika}, S.},\n title = "{A Deep Near-infrared Survey toward the Aquila Molecular Cloud. I. Molecular Hydrogen Outflows}",\n journal = {\\apjs},\narchivePrefix = "arXiv",\n eprint = {1506.08372},\n primaryClass = "astro-ph.SR",\n keywords = {infrared: ISM, ISM: jets and outflows, shock waves, stars: formation, stars: winds, outflows},\n year = 2015,\n month = aug,\n volume = 219,\n eid = {21},\n pages = {21},\n doi = {10.1088/0067-0049/219/2/21},\n adsurl = {http://adsabs.harvard.edu/abs/2015ApJS..219...21Z},\n adsnote = {Provided by the SAO/NASA Astrophysics Data System}\n}\n\n'} -The get AASTeX back, you use `aastex` as format parameter, and `endnote` for EndNote. \ No newline at end of file +The get AASTeX back, you use `aastex` as format parameter, and `endnote` for EndNote. From 9ee834f6352c6d7f3d792548449d8a5f5e36d946 Mon Sep 17 00:00:00 2001 From: Jonny Elliott Date: Tue, 3 Nov 2015 12:41:43 -0500 Subject: [PATCH 10/39] Closes issue #20 biblib api documentation --- libraries.md | 302 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 302 insertions(+) create mode 100644 libraries.md diff --git a/libraries.md b/libraries.md new file mode 100644 index 0000000..8311159 --- /dev/null +++ b/libraries.md @@ -0,0 +1,302 @@ +# ADS Bibliographic Libraries API + +The source code of this microservice can be viewed [here](https://github.com/adsabs/biblib-service). + +## API end point +`https://api.adsabs.harvard.edu/v1/biblib` + +### End points and methods + +#### /libraries + +End point to create a library or view all libraries for a given user. + +##### GET +Obtains all the libraries that belong to the requesting user, along with some additional metadata. + +*Return data*: + * name: + * **string** + * Name of the library + * id: + * **string** + * ID of the library + * description: + * **string** + * Description of the library + * num_documents: + * **int** + * Number of documents in the library + * date_created: + * **string** + * ISO date library was created + * date_last_modified: + * **string** + * ISO date library was last modified + * permission: + * **string** + * Permission type, can be: 'read', 'write', 'admin', or 'owner' + * public: + * **boolean** + * True means it is a public library + * num_users: + * **int** + * Number of users with permissions to this library + * owner: + * **string** + * Identifier of the user who created the library + +*Permissions*: +Must be authenticated via the API. + +##### POST +Creates a library for the requesting user. Returns metadata on the library created. + +*Post body*: + * *name*: + * **string** + * name of the library (must be unique for that user) + * *description*: + * **string** + * description of the library + * *public*: + * **boolean** + * is the library public to view + * *bibcode (OPTIONAL)*: + * **list** + * list of bibcodes to add + +*Return data*: + * **name**: + * Name of the library + * **id**: + * + * ID of the library + * **description**: + * + * Description of the library + +*Permissions*: +Must be authenticated via the API. + +#### /libraries/<library_id> + +End point to interact with a specific library, only returns library content if the user has the correct privileges. See the document end point for deleting a library. + + * **library_id** [**string**] is the unique identifier of the library, as specified in the `/libraries` GET response. + +##### GET +Obtains all the documents of a given library. + +*Return data*: + + * documents: + * **list** + * Currently, a list containing the bibcodes. + * solr: + * **dict** + * The response from the solr bigquery end point + * metadata: + * **dict** + * contains the following: + * name: + * **string** + * Name of the library + * id: + * **string** + * ID of the library + * description: + * **string** + * Description of the library + * num_documents: + * **int** + * Number of documents in the library + * date_created: + * **string** + * ISO date library was created + * date_last_modified: + * **string** + * ISO date library was last modified + * permission: + * **sting** + * Permission type, can be: 'read', 'write', 'admin', or 'owner' + * public: + * **boolean** + * True means it is public + * num_users: + * **int** + * Number of users with permissions to this library + * owner: + * **string** + * Identifier of the user who created the library + * updates: + * **dict** + * contains the following + * num_updated: + * **int** + * Number of documents modified based on the response from solr + * duplicates_removed: + * **int** + * Number of files removed because they are duplications + * update_list: + * **list**[**dict**] + * List of dictionaries such that a single element described the original bibcode (key) and the updated bibcode now being stored (item) + +*Permissions*: +The following type of user can read a library: + - owner + - admin + - write + - read + +#### /documents/<library_id> + +End point to interact with a specific library, by adding documents and removing documents. You also use this endpoint to delete the entire library. + + * **library_id** [**string**] is the unique identifier of the library, as specified in the `/libraries` GET response. + +##### POST + +Request that adds a document to a library for a given user. + +*Post body*: + + * bibcode: + * **list** + * List of bibcodes to be added + * action: + * **string** + * 'add' or 'remove' + * add - adds a bibcode + * remove - removes a bibcode + +*Return data*: + * number_added: + * **int** + * number of documents added (if 'add' is used) + * number_removed: + * **int** + * number of documents removed (if 'remove' is used) + +*Permissions*: +The following type of user can add documents: + - owner + - admin + - write + +##### PUT + +Request that updates the meta-data of the library + +*Post-body*: + * name: + * **string** + * name of the library + * description: + * **string** + * description of the library + * public: + * **boolean** + * if the library should be publicly viewable + +*Note: The above are optional. Leave fields out if they do not need to be updated.* + +*Return data*: + * name: + * **string** + * name of the library + * description: + * **string** + * description of the library + * public: + * **boolean** + * if the library should be publicly viewable + +*Note: returns the key/value that was requested to be updated* + +*Permissions*: +The following type of user can update the 'name', 'library', and 'public' attributes: + - owner + - admin + +##### DELETE + +Request that deletes a library. + +*Permissions*: + +The following type of user can update a library: + - owner + +#### /permissions/<library_id> + +End point to manipulate the permissions between a user and a library. + + * **library_id** [**string**] is the unique identifier of the library, as specified in the `/libraries` GET response. + +##### GET +Request that returns the permissions for a given library. + +*Return data*: + * list of dictionaries, where each dictionary is for a specific user. The dictionary contains a list of all the permissions of the user. + * **list** + * user dictionary + * **dict** + * **key**: + * **string** + * user e-mail + * **item**: + * **list** + * list of permissions: 'read', 'write', 'admin', 'owner' + +*Permissions*: +The following type of user can access permissions of a library: + - owner + - admin + +##### POST +Request that modifies the permissions of a library + +*Post data*: + + * username: + * **string** + * specifies which user's permissions to be modified + * permission: + * **string** + * 'read', 'write', 'admin' or 'owner' + * value: + * **boolean** + * whether the user has this permission (True=yes, False=no) + + +*Permissions*: +The following type of user can update a permission for a user for a given library: + - owner + - admin + +#### /transfer/<library_id> + +End point to transfer a the ownership of a library to another ADS user. + + * **library_id** [**string**] is the unique identifier of the library, as specified in the `/libraries` GET response. + +##### POST +Request that transfers the ownership of a library from one user to another. + +*Post body*: + * transfer_user: + * **string** + * username of the user the account should be transfered to + +*Permissions*: +The following type of user can transfer libraries: + - owner + +## Permissions + * **owner**: Owns the library. Can read libraries, add/remove bibcodes, delete the library, add/remove permissions, transfer ownership of the library. + * **admin**: Can read libraries, add/remove bibcodes, add/remove permissions. + * **write**: Can read libraries, add/remove bibcdes. + * ** read**: Can read libraries. + +Any other user that does not have a permission to the library they try to visit, can only see it if it is public. If the user has no permissions, they cannot view this library. From bd5cca68e7e4325e8afae9091ae30efb18c411e2 Mon Sep 17 00:00:00 2001 From: Jonny Elliott Date: Tue, 3 Nov 2015 12:43:15 -0500 Subject: [PATCH 11/39] Update README.md --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 3660ad3..2adc2bb 100644 --- a/README.md +++ b/README.md @@ -11,6 +11,7 @@ For bugs, feature requests or even random questions feel free to use the [issues - [Search API](search.md) - [Metrics API](metrics.md) - [Export API](export.md) +- [Libraries API](libraries.md) ## Mailing List From 42946f4f32f77add0291a0041b1c853a461f3b1a Mon Sep 17 00:00:00 2001 From: Andy Casey Date: Wed, 4 Nov 2015 20:36:59 +0000 Subject: [PATCH 12/39] Fixed typo Also, this is **awesome**. I'm going to start adding code to make use of this functionality on the client end. I was just about to email you guys if there was going to be API access to libraries in the future, and then here we are! --- libraries.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libraries.md b/libraries.md index 8311159..910d633 100644 --- a/libraries.md +++ b/libraries.md @@ -297,6 +297,6 @@ The following type of user can transfer libraries: * **owner**: Owns the library. Can read libraries, add/remove bibcodes, delete the library, add/remove permissions, transfer ownership of the library. * **admin**: Can read libraries, add/remove bibcodes, add/remove permissions. * **write**: Can read libraries, add/remove bibcdes. - * ** read**: Can read libraries. + * **read**: Can read libraries. Any other user that does not have a permission to the library they try to visit, can only see it if it is public. If the user has no permissions, they cannot view this library. From d08de5dd9945b04297e3ee0e1e9b9d3addb4d0c6 Mon Sep 17 00:00:00 2001 From: Edwin Henneken Date: Wed, 18 Nov 2015 08:45:30 -0500 Subject: [PATCH 13/39] metrics doc update: proper quoting sample JSON output --- metrics.md | 220 ++++++++++++++++++++++++++--------------------------- 1 file changed, 110 insertions(+), 110 deletions(-) diff --git a/metrics.md b/metrics.md index 77e95eb..65349e3 100644 --- a/metrics.md +++ b/metrics.md @@ -84,18 +84,18 @@ Each of these attributes contains specific data elements. The following overview For all histograms, the actual histogram values are stored as follows: { - 'histogram name': {year: value, year: value, ...} + "histogram name": {year: value, year: value, ...} } where values can be either integers or floats, depending on the type of histograms. The histogram name is an attribute of the histogram type. For example, the `publications` type has 4 separate histograms: `all publications`, `refereed publications`, `all publications normalized` and `refereed publications normalized`, which are organized as follows in the JSON output: { - 'histograms': { - 'publications': { - 'all publications': {}, - 'refereed publications': {}, - 'all publications normalized': {}, - 'refereed publications normalized': {}, + "histograms": { + "publications": { + "all publications": {}, + "refereed publications": {}, + "all publications normalized": {}, + "refereed publications normalized": {}, } } @@ -114,13 +114,13 @@ Note that 'refereed reads' means: 'number of reads for the refereed publications The `timeseries` attribute contains time series data for: `h`, `g`, `i10`, `i100`, `read10` and `tori`. The output is structured as { - 'time series': { - 'h': { values }, - 'g': { values }, - 'i10': { values }, - 'i100': { values }, - 'read10': { values }, - 'tori': { values }, + "time series": { + "h": { values }, + "g": { values }, + "i10": { values }, + "i100": { values }, + "read10": { values }, + "tori": { values }, } } @@ -128,127 +128,127 @@ The `timeseries` attribute contains time series data for: `h`, `g`, `i10`, `i100 where the 'values' are organized in the same way as for the histograms: { - 'indicator name': {year: value, year: value, ...} + "indicator name": {year: value, year: value, ...} } ## Everything together So, returning everything, the output looks like { - 'skipped bibcodes': [ ... ], - 'basic stats': + "skipped bibcodes": [ ... ], + "basic stats": { - 'number of papers': 1, - 'normalized paper count': 0.1, - 'median number of reads': 1.1, - 'average number of reads': 1.1, - 'total number of reads': 1, - 'normalized number of reads': 0.1, - 'median number of downloads': 1.1, - 'average number of downloads': 1.1, - 'total number of downloads': 1, - 'normalized number of downloads': 0.1, + "number of papers": 1, + "normalized paper count": 0.1, + "median number of reads": 1.1, + "average number of reads": 1.1, + "total number of reads": 1, + "normalized number of reads": 0.1, + "median number of downloads": 1.1, + "average number of downloads": 1.1, + "total number of downloads": 1, + "normalized number of downloads": 0.1, }, - 'basic stats refereed': + "basic stats refereed": { - 'number of papers': 1, - 'normalized paper count': 0.1, - 'median number of reads': 1.1, - 'average number of reads': 1.1, - 'total number of reads': 1, - 'normalized number of reads': 0.1, - 'median number of downloads': 1.1, - 'average number of downloads': 1.1, - 'total number of downloads': 1, - 'normalized number of downloads': 0.1, + "number of papers": 1, + "normalized paper count": 0.1, + "median number of reads": 1.1, + "average number of reads": 1.1, + "total number of reads": 1, + "normalized number of reads": 0.1, + "median number of downloads": 1.1, + "average number of downloads": 1.1, + "total number of downloads": 1, + "normalized number of downloads": 0.1, }, - 'citation stats': + "citation stats": { - 'number of citing papers': 1, - 'median number of citations': 0.1, - 'average number of citations': 1.1, - 'total number of citations': 1, - 'normalized number of citations': 0.1, - 'median number of refereed citations': 0.1, - 'average number of refereed citations': 1.1, - 'total number of refereed citations': 1, - 'normalized number of refereed citations': 0.1, - 'number of self-citations': 1, + "number of citing papers": 1, + "median number of citations": 0.1, + "average number of citations": 1.1, + "total number of citations": 1, + "normalized number of citations": 0.1, + "median number of refereed citations": 0.1, + "average number of refereed citations": 1.1, + "total number of refereed citations": 1, + "normalized number of refereed citations": 0.1, + "number of self-citations": 1, }, - 'citations stats refereed': + "citations stats refereed": { - 'number of citing papers': 1, - 'median number of citations': 0.1, - 'average number of citations': 1.1, - 'total number of citations': 1, - 'normalized number of citations': 0.1, - 'median number of refereed citations': 0.1, - 'average number of refereed citations': 1.1, - 'total number of refereed citations': 1, - 'normalized number of refereed citations': 0.1, - 'number of self-citations': 1, + "number of citing papers": 1, + "median number of citations": 0.1, + "average number of citations": 1.1, + "total number of citations": 1, + "normalized number of citations": 0.1, + "median number of refereed citations": 0.1, + "average number of refereed citations": 1.1, + "total number of refereed citations": 1, + "normalized number of refereed citations": 0.1, + "number of self-citations": 1, }, - 'indicators': + "indicators": { - 'i10': 1, - 'i100': 1, - 'h': 1, - 'm': 1.1, - 'g': 1, - 'read10': 0.1, - 'tori': 1.1, - 'riq': 1, + "i10": 1, + "i100": 1, + "h": 1, + "m": 1.1, + "g": 1, + "read10": 0.1, + "tori": 1.1, + "riq": 1, }, - 'indicators refereed': + "indicators refereed": { - 'i10': 1, - 'i100': 1, - 'h': 1, - 'm': 1.1, - 'g': 1, - 'read10': 0.1, - 'tori': 1.1, - 'riq': 1, + "i10": 1, + "i100": 1, + "h": 1, + "m": 1.1, + "g": 1, + "read10": 0.1, + "tori": 1.1, + "riq": 1, }, - 'histograms': + "histograms": { - 'publications': { - 'all publications': { values }, - 'refereed publications': { values }, - 'all publications normalized': { values }, - 'refereed publications normalized': { values }, + "publications": { + "all publications": { values }, + "refereed publications": { values }, + "all publications normalized": { values }, + "refereed publications normalized": { values }, }, - 'reads': { - 'all reads': { values }, - 'refereed reads': { values }, - 'all reads normalized': { values }, - 'refereed reads normalized': { values }, + "reads": { + "all reads": { values }, + "refereed reads": { values }, + "all reads normalized": { values }, + "refereed reads normalized": { values }, }, - 'downloads': { - 'all downloads': { values }, - 'refereed downloads': { values }, - 'all downloads normalized': { values }, - 'refereed downloads normalized': { values }, + "downloads": { + "all downloads": { values }, + "refereed downloads": { values }, + "all downloads normalized": { values }, + "refereed downloads normalized": { values }, }, - 'citations': { - 'refereed to refereed': { values }, - 'nonrefereed to refereed': { values }, - 'refereed to nonrefereed': { values }, - 'nonrefereed to nonrefereed': { values }, - 'refereed to refereed normalized': { values }, - 'nonrefereed to refereed normalized': { values }, - 'refereed to nonrefereed normalized': { values }, - 'nonrefereed to nonrefereed normalized': { values }, + "citations": { + "refereed to refereed": { values }, + "nonrefereed to refereed": { values }, + "refereed to nonrefereed": { values }, + "nonrefereed to nonrefereed": { values }, + "refereed to refereed normalized": { values }, + "nonrefereed to refereed normalized": { values }, + "refereed to nonrefereed normalized": { values }, + "nonrefereed to nonrefereed normalized": { values }, }, }, - 'time series': + "time series": { - 'h': { values }, - 'g': { values }, - 'i10': { values }, - 'i100': { values }, - 'read10': { values }, - 'tori': { values }, + "h": { values }, + "g": { values }, + "i10": { values }, + "i100": { values }, + "read10": { values }, + "tori": { values }, } } From 8de66d2b630f1841ac3ff3b8796a0fdc00ea78a6 Mon Sep 17 00:00:00 2001 From: Roman Chyla Date: Tue, 24 Nov 2015 18:23:16 -0500 Subject: [PATCH 14/39] Improved search.md --- search.md | 65 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/search.md b/search.md index 3123e9b..489fe5c 100644 --- a/search.md +++ b/search.md @@ -11,10 +11,75 @@ Base URL: `https://api.adsabs.harvard.edu/v1/search` +## Get search results + + GET /query?q=value&fl=value2..... + All text values shoud be UTF-8 and url-encoded. The response body will be json encoded. Note that the search API uses the same syntax as [Apache Solr](http://lucene.apache.org/solr/). For a full reference of query possibilities, please refer to the Solr documentation and [ADS Search Help](http://adsabs.github.io/help/search/). The sections below present useful parameters and patterns for the vast majority of use cases, but are not meant to be exhaustive. + +## Parse a query + + GET /qtree?q=this+OR+that + +Returns a `query tree` (Abstract Syntax Tree - AST) as understood by our query parser. Useful if you want to modify and/or enhance +queries. + +Example response (value in the `qtree` is a string (JSON) serialized version of the AST): + +``` +{ + "qtree": "\n{\"name\":\"OPERATOR\", \"label\":\"DEFOP\", \"children\": [\n {\"name\":\"MODIFIER\", \"label\":\"MODIFIER\", \"children\": [\n {\"name\":\"TMODIFIER\", \"label\":\"TMODIFIER\", \"children\": [\n {\"name\":\"FIELD\", \"label\":\"FIELD\", \"children\": [\n {\"name\":\"QNORMAL\", \"label\":\"QNORMAL\", \"children\": [\n {\"name\":\"TERM_NORMAL\", \"input\":\"star\", \"start\":0, \"end\":3}]\n }]\n }]\n }]\n }]\n}", + "responseHeader": { + "status": 0, + "QTime": 6, + "params": { + "q": "star", + "wt": "json", + "fl": "id" + } + } + } +``` + +## Post a large query + + POST /bigquery[?urlparams] + +Returns standard search results, but accepts as input very large query (i.e. a query that can be expressed only as a list of search +criteria, typically IDs). There is currently no limit to the size of the submitted data (besides buffer/time limits imposed by our API +frontend), however there are severe limits on how often you can call this enpoint. Typically, only 100 requests per day are allowed. + +### Parameters + +Name | Type | Description +--------------|--------|-------------- +`urlparams` |`string`| **Required**. Url escaped/serialized query parameters if used in the URL (ie: `q=star&fl=bibcode,title`) or form encoded search paramaters. +`payload` |`string`| **Required**. Newline separated list of values, the first line specifies index that will be used for search (ie: `bibcode\n1907AN....174...59.\n1908PA.....16..445.\n1989LNP...334..242S`) + + +Currently, we allow to search in `bibcode` index only. You can submit `canonical` as well as `alternate` bibcodes; the search will automatically match both. In the future, the list of available indexes may be extended. (We do not plan to support search with other values than IDs). + +The bigquery filter is *applied only after* the main search (ie: it limits results of the main search). + +Example python session - get all papers from ADS and filter them using several IDs: + +```python +import requests +bibcodes="bibcode\n1907AN....174...59.\n1908PA.....16..445.\n1989LNP...334..242S" +r = requests.post('https://api.adsabs.harvard.edu/v1/search/bigquery', + params={'q':'*:*', 'wt':'json', 'fq':'{!bitset}', 'fl':'bibcode'}, + headers={'Authorization': 'Bearer: TOKEN'}, + data=bibcodes) +``` + +Output is exactly the same as from `/query` endpoint. + + + + ##### Basic search https://api.adsabs.harvard.edu/v1/search/query The basic interface to the ADS data holdings. Query our search engine and receive the cooresponding documents back. From 49d7faf2203fd7e6535102a02dff043a4cfa5cb6 Mon Sep 17 00:00:00 2001 From: Roman Chyla Date: Tue, 24 Nov 2015 18:26:33 -0500 Subject: [PATCH 15/39] Improved search.md --- search.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/search.md b/search.md index 489fe5c..d76d325 100644 --- a/search.md +++ b/search.md @@ -11,9 +11,10 @@ Base URL: `https://api.adsabs.harvard.edu/v1/search` + ## Get search results - GET /query?q=value&fl=value2..... + GET /query?q=value&fl=value2..... All text values shoud be UTF-8 and url-encoded. The response body will be json encoded. @@ -22,7 +23,7 @@ Note that the search API uses the same syntax as [Apache Solr](http://lucene.apa ## Parse a query - GET /qtree?q=this+OR+that + GET /qtree?q=this+OR+that Returns a `query tree` (Abstract Syntax Tree - AST) as understood by our query parser. Useful if you want to modify and/or enhance queries. @@ -46,7 +47,7 @@ Example response (value in the `qtree` is a string (JSON) serialized version of ## Post a large query - POST /bigquery[?urlparams] + POST /bigquery[?urlparams] Returns standard search results, but accepts as input very large query (i.e. a query that can be expressed only as a list of search criteria, typically IDs). There is currently no limit to the size of the submitted data (besides buffer/time limits imposed by our API @@ -79,10 +80,7 @@ Output is exactly the same as from `/query` endpoint. - -##### Basic search - https://api.adsabs.harvard.edu/v1/search/query -The basic interface to the ADS data holdings. Query our search engine and receive the cooresponding documents back. + ## Query parameters From 27bd8b15f8e154e1b8b52d70e7516538682eb2a3 Mon Sep 17 00:00:00 2001 From: Alberto Accomazzi Date: Tue, 15 Dec 2015 09:31:16 -0500 Subject: [PATCH 16/39] Update README.md --- README.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 2adc2bb..73be269 100644 --- a/README.md +++ b/README.md @@ -43,10 +43,20 @@ All API requests must pass your token in an `Authorization: Bearer ` HTTP ## Access Settings -Each endpoint is individually rate-limited. API Responses advertise these limits in their response headers: +Each endpoint is individually rate-limited. API Responses advertise these limits in their HTTP response headers. A useful way to see these values is to issue a curl request to the desired endpoint with the verbose flag, e.g.: + + curl -v -H "Authorization: Bearer " 'https://api.adsabs.harvard.edu/v1/search/query?q=star' + +And then noting the following values: X-RateLimit-Limit: 5000 X-RateLimit-Remaining: 4999 X-RateLimit-Reset: 1435190400 +The Limit variable indicates the amount of daily queries allowed to the user (in this case 5000). The Remaining variable indicates how many queries are still available. The Reset variable provides a UTC timestamp corresponding to the time the rate limits will be reset. To see its value in human-readable format, you can use the UNIX "date" command: + + date -r 1435190400 + Wed Jun 24 20:00:00 EDT 2015 + +(the rate resetting happens at midnight UTC). To increase rate limits, please contact us directly at `adshelp@cfa.harvard.edu`. From 562db8b953c9b84dfba255908079e91dcc752260 Mon Sep 17 00:00:00 2001 From: Alberto Accomazzi Date: Thu, 18 Feb 2016 11:26:01 -0500 Subject: [PATCH 17/39] Update search.md --- search.md | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/search.md b/search.md index d76d325..10816d2 100644 --- a/search.md +++ b/search.md @@ -84,6 +84,8 @@ Output is exactly the same as from `/query` endpoint. ## Query parameters +All query parameters appearing in the search URL must be UTF-8, url-encoded strings. Please note that due to the requirements of the authentication library used for validating requests, most non-ascii characters appearing in the URL need to be hex-encoded, e.g. a double quote character (") must be encoded as %22. In most programming languages the libraries used to retrieve content from web services will perform this encoding for you, but if you are using your own curl-based request you will need to take care of this. + #### q *required:* a UTF-8, url-encoded string of <= 1000 characters representing the search query. `q` can be used for both fielded (`title:exoplanets`), and unfielded (`exoplanets`) search. See the list of fields [below](#fields). @@ -94,10 +96,10 @@ number of results to return. Default is 10. starting point for returned results (for pagination). Default is 0. #### fl -Specify the fields contained in each returned document. Value should be a comma-separated list of field names. +Fields list: specify the fields contained in each returned document. Value should be a comma-separated list of field names. #### fq -Filter your query results using a particular `field:value` condition. This parameter is repeatable. +Filter query: filter your results using a particular `field:value` condition. This parameter is repeatable. * `field` can be any field listed in the field descriptions below * `value` should be a UTF-8, url-encoded string @@ -151,19 +153,20 @@ Assuming your access token is `my_token`, all queries can be executed by any HTT ?q=black+holes&fq=database:astronomy # Search for "dark energy", filter by author, sort by citation count - ?q=dark+energy&fq=author:"Civano,+F"&sort=citation_count+desc + # (double quotes encoded as %22) + ?q=dark+energy&fq=author:%22Civano,+F%22&sort=citation_count+desc # Same search but only return *bibcode* and *property* values - ?q=dark+energy&fq=author:"Civano,+F"&sort=citation_count+desc&fl=bibcode,property + ?q=dark+energy&fq=author:%22Civano,+F%22&sort=citation_count+desc&fl=bibcode,property # Limit a search to only refereed articles - ?q=author:"Kurtz,+M"&fq=property:refereed + ?q=author:%22Kurtz,+M%22&fq=property:refereed - # Search for "transiting exoplanets", get 200 rows - ?q=transiting+exoplanets&rows=200 + # Search for the phrase "transiting exoplanets", get 200 rows + ?q=%22transiting+exoplanets%22&rows=200 # Same search but get the next 200 rows - ?q=transiting+exoplanets&rows=200&start=201 + ?q=%22transiting+exoplanets%22&rows=200&start=201 ## Example record response From 54eb976aa96ac81bdb5e44e9f17c74ca1fb96b61 Mon Sep 17 00:00:00 2001 From: Jonny Elliott Date: Fri, 27 May 2016 14:23:05 -0400 Subject: [PATCH 18/39] Updated the list of fields --- README.md | 4 ++++ search.md | 50 +++++++++++++++++++++++++++++++++++++------------- 2 files changed, 41 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 73be269..996cb0e 100644 --- a/README.md +++ b/README.md @@ -27,6 +27,10 @@ Geert Barentsen has built an application to support the Kepler publication datab * https://github.com/KeplerGO/kpub +The ADS built an application to compare author's with a wrestling theme at AAS 227: + +* https://authorsmackdown.herokuapp.com/ + ## Access Access to the ADS data holdings is regulated by the the ADS terms of use, as described in the ADS [Terms of Use](http://adsabs.github.io/help/terms/). diff --git a/search.md b/search.md index 10816d2..955995f 100644 --- a/search.md +++ b/search.md @@ -113,26 +113,50 @@ Some useful fields to sort by may be `pubdate`, `citation_count`, or `num_reader ## Fields -Below are some fields noted here for their high usefulness that are available to all API users: +Below are the fields available to all API users: +* `abstract` - the abstract of the record +* `ack` - the acknowledgements section of an article +* `aff` - an array of the authors' affiliations +* `alternate_bibcode` - list of alternate bibcodes for a single record +* `alternate_title` - list of alternate titles for a single record (usually if they are in multiple languages) +* `arxiv_class` - the arXiv class the pre-print was submitted to +* `author` - an array of the author names associated with the record +* `author_facet` * `bibcode` - the canonical ADS bibcode identifier for this record +* `bibgroup` - the bibliographic groups that the bibcode has been associated with +* `bibgroup_facet` * `bibstem` - the abbreviated name of the journal or publication, e.g., *ApJ*. -* `id` - a unique integer for this record +* `bibstem_facet` +* `body`\* - the full text content of the article +* `citation_count` - number of citations the item has received +* `copyright` - the copyright applied to the article +* `data` - the list of sources that have data related to this bibcode +* `database` - Which database the record is associated with. +* `doi`- the digital object identifier of the article +* `first_author` - the first author of the article +* `grant` - the list of grant IDs and agencies noted within an article +* `id` - a (**non-persistent**) unique integer for this record, used for fast look-up of a document * `identifier` - an array of alternative identifiers for the record. May contain alternative bibcodes, DOIs and/or arxiv ids. -* `title` - the title of the record -* `author` - an array of the author names associated with the record -* `pub` - the canonical name of the publication the record appeared in -* `keyword` - an array of normalized and un-normalized keyword values associated with the record -* `abstract` - the abstract of the record -* `aff` - an array of the authors' affiliations -* `property` - an array of miscellaneous flags associated with the record. Possible values include: ARTICLE, REFEREED, NOT_REFEREED, INPROCEEDINGS, OPENACCESS, NONARTICLE, EPRINT, BOOK, PROCEEDINGS, CATALOG, SOFTWARE (@aacommazzi didn't we remove/clean up some of these? Could you tell me which, if any, don't belong here) -* `volume` - volume the record appeared in +* `indexstamp` - time at which the document was (last) indexed * `issue` - issue the record appeared in +* `keyword` - an array of normalized and un-normalized keyword values associated with the record +* `lang`\* - the language of the article's title +* `orcid_pub` - ORCiD iDs supplied by publishers +* `orcid_user` - ORCiD iDs supplied by knonwn users in the ADS +* `orcid_other` - ORCiD iDs supplied by anonymous users in the ADS * `page` - starting page -* `citation_count` - number of citations the item has received -* `read_count` - number of times the record has been viewed within in a 90-day windows (ads and arxiv) +* `property` - an array of miscellaneous flags associated with the record. Possible values include: `ARTICLE`, `REFEREED`, `NOT\_REFEREED`, `INPROCEEDINGS`, `OPENACCESS`, `NONARTICLE`, `EPRINT`, `BOOK`, `PROCEEDINGS`, `CATALOG`,`SOFTWARE +* `pub` - the canonical name of the publication the record appeared in * `pubdate` - publication date in the form YYYY-MM-DD (DD value will always be "00") -* `database` - Which database the record is associated with. +* `read_count` - number of times the record has been viewed within in a 90-day windows (ads and arxiv) +* `title` - the title of the record +* `vizier` - the subject tags given to the article by VizieR +* `vizier_facet` +* `volume` - volume the record appeared in +* `year` - the year the article was published + +\* *These fields are only indexed and so are only searchable - they are not stored or returned by the search end point, if requested* #### databases From edd8962c0c4b85ac0b846f1c2f4175911f707ec1 Mon Sep 17 00:00:00 2001 From: Jonny Elliott Date: Fri, 27 May 2016 14:24:22 -0400 Subject: [PATCH 19/39] Added another URL --- README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/README.md b/README.md index 996cb0e..f67199d 100644 --- a/README.md +++ b/README.md @@ -23,6 +23,10 @@ The unofficial python client for the API is maintained by Andy Casey and can be * https://github.com/andycasey/ads +Examples of how to use Andy Casey's Python client can be found here: + +* https://github.com/jonnybazookatone/ads-examples + Geert Barentsen has built an application to support the Kepler publication database which uses the ADS API to discover relevant papers: * https://github.com/KeplerGO/kpub @@ -31,6 +35,7 @@ The ADS built an application to compare author's with a wrestling theme at AAS 2 * https://authorsmackdown.herokuapp.com/ + ## Access Access to the ADS data holdings is regulated by the the ADS terms of use, as described in the ADS [Terms of Use](http://adsabs.github.io/help/terms/). From 7b918e2e3f8f7afb700aea570e120ecfe3db93d1 Mon Sep 17 00:00:00 2001 From: Jonny Elliott Date: Fri, 27 May 2016 15:18:31 -0400 Subject: [PATCH 20/39] Updated fields, with links --- search.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/search.md b/search.md index 955995f..c73ed34 100644 --- a/search.md +++ b/search.md @@ -122,18 +122,16 @@ Below are the fields available to all API users: * `alternate_title` - list of alternate titles for a single record (usually if they are in multiple languages) * `arxiv_class` - the arXiv class the pre-print was submitted to * `author` - an array of the author names associated with the record -* `author_facet` * `bibcode` - the canonical ADS bibcode identifier for this record * `bibgroup` - the bibliographic groups that the bibcode has been associated with -* `bibgroup_facet` * `bibstem` - the abbreviated name of the journal or publication, e.g., *ApJ*. -* `bibstem_facet` * `body`\* - the full text content of the article * `citation_count` - number of citations the item has received * `copyright` - the copyright applied to the article * `data` - the list of sources that have data related to this bibcode * `database` - Which database the record is associated with. * `doi`- the digital object identifier of the article +* `doctype` - the type of document it is (see [here](https://adsabs.github.io/help/search/search-syntax) for a list of doctypes) * `first_author` - the first author of the article * `grant` - the list of grant IDs and agencies noted within an article * `id` - a (**non-persistent**) unique integer for this record, used for fast look-up of a document @@ -146,13 +144,12 @@ Below are the fields available to all API users: * `orcid_user` - ORCiD iDs supplied by knonwn users in the ADS * `orcid_other` - ORCiD iDs supplied by anonymous users in the ADS * `page` - starting page -* `property` - an array of miscellaneous flags associated with the record. Possible values include: `ARTICLE`, `REFEREED`, `NOT\_REFEREED`, `INPROCEEDINGS`, `OPENACCESS`, `NONARTICLE`, `EPRINT`, `BOOK`, `PROCEEDINGS`, `CATALOG`,`SOFTWARE +* `property` - an array of miscellaneous flags associated with the record (see [here](https://adsabs.github.io/help/search/search-syntax) for a list of properties * `pub` - the canonical name of the publication the record appeared in * `pubdate` - publication date in the form YYYY-MM-DD (DD value will always be "00") * `read_count` - number of times the record has been viewed within in a 90-day windows (ads and arxiv) * `title` - the title of the record * `vizier` - the subject tags given to the article by VizieR -* `vizier_facet` * `volume` - volume the record appeared in * `year` - the year the article was published From 47b1b2dd561c2930df3aec7a5df1f12408499826 Mon Sep 17 00:00:00 2001 From: Jonny Elliott Date: Wed, 8 Jun 2016 15:02:41 -0400 Subject: [PATCH 21/39] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index f67199d..83d2691 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,7 @@ The unofficial python client for the API is maintained by Andy Casey and can be Examples of how to use Andy Casey's Python client can be found here: -* https://github.com/jonnybazookatone/ads-examples +* https://github.com/adsabs/ads-examples Geert Barentsen has built an application to support the Kepler publication database which uses the ADS API to discover relevant papers: From 80363286d2b40dcede81c90007dc865c2fd87973 Mon Sep 17 00:00:00 2001 From: Alberto Accomazzi Date: Mon, 27 Mar 2017 09:23:46 -0400 Subject: [PATCH 22/39] Point to readthedocs site for ads python client --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 83d2691..d545643 100644 --- a/README.md +++ b/README.md @@ -19,9 +19,9 @@ Announcements and discussion related to the Developer API are available via the ## Clients -The unofficial python client for the API is maintained by Andy Casey and can be found here: +The unofficial python client for the API is maintained by Andy Casey described here: -* https://github.com/andycasey/ads +* http://ads.readthedocs.io/ Examples of how to use Andy Casey's Python client can be found here: From f16010b5809bc46e061ed0a392b9142ac15af6ff Mon Sep 17 00:00:00 2001 From: Roman Chyla Date: Mon, 10 Apr 2017 16:34:14 -0400 Subject: [PATCH 23/39] Update search.md --- search.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/search.md b/search.md index c73ed34..52b8c6d 100644 --- a/search.md +++ b/search.md @@ -163,11 +163,11 @@ The ADS content is divided into a few separate databases: "astronomy", "physics" Assuming your access token is `my_token`, all queries can be executed by any HTTP client. An example `curl` request is: -`curl -H "Authorization: Bearer my_token" https://api.adsabs.harvard.edu/v1/search/query` +`curl -H "Authorization: Bearer my_token" "https://api.adsabs.harvard.edu/v1/search/query?........"` ...and by adding the following query params, the response will contain relevant documents: - # Search by bibcode + # Search by bibcode (make sure you enclose these arguments in quotes!) ?q=bibcode:2011ApJ...737..103S # Search for "black holes", restricted to astronomy content From 3c6a6cab4bcd4f14482e1eb6c8bcddcd02e24aff Mon Sep 17 00:00:00 2001 From: Alberto Accomazzi Date: Fri, 12 May 2017 11:34:48 -0400 Subject: [PATCH 24/39] Update search.md --- search.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/search.md b/search.md index 52b8c6d..b629a40 100644 --- a/search.md +++ b/search.md @@ -86,9 +86,10 @@ Output is exactly the same as from `/query` endpoint. All query parameters appearing in the search URL must be UTF-8, url-encoded strings. Please note that due to the requirements of the authentication library used for validating requests, most non-ascii characters appearing in the URL need to be hex-encoded, e.g. a double quote character (") must be encoded as %22. In most programming languages the libraries used to retrieve content from web services will perform this encoding for you, but if you are using your own curl-based request you will need to take care of this. -#### q -*required:* a UTF-8, url-encoded string of <= 1000 characters representing the search query. `q` can be used for both fielded (`title:exoplanets`), and unfielded (`exoplanets`) search. See the list of fields [below](#fields). +[Below](#fields) you can find the list of fields available to all API users. A more comprehensive list is [available in our help pages](https://adsabs.github.io/help/search/comprehensive-solr-term-list). +#### q +*required:* a UTF-8, url-encoded string of <= 1000 characters representing the search query. `q` can be used for both fielded (`title:exoplanets`), and unfielded (`exoplanets`) search. #### rows number of results to return. Default is 10. @@ -96,7 +97,7 @@ number of results to return. Default is 10. starting point for returned results (for pagination). Default is 0. #### fl -Fields list: specify the fields contained in each returned document. Value should be a comma-separated list of field names. +Fields list: specify the fields contained in each returned document. Value should be a comma-separated list of field names. Default is just the document id (field `id`). #### fq Filter query: filter your results using a particular `field:value` condition. This parameter is repeatable. @@ -109,7 +110,7 @@ Indicate how you wish the result set sorted. The format is `field direction` whe Example of a properly formated & encoded sort param is `sort=read_count+desc` -Some useful fields to sort by may be `pubdate`, `citation_count`, or `num_readers`. +Some useful fields to sort by may be `pubdate`, `citation_count`, or `first_author`. ## Fields From f1e79c145a7807bcd9c3b189ee88f4747322caf0 Mon Sep 17 00:00:00 2001 From: Roman Chyla Date: Tue, 13 Jun 2017 15:49:39 -0400 Subject: [PATCH 25/39] Fixed extra space in the Authorization header example --- search.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/search.md b/search.md index b629a40..f320e76 100644 --- a/search.md +++ b/search.md @@ -72,7 +72,7 @@ import requests bibcodes="bibcode\n1907AN....174...59.\n1908PA.....16..445.\n1989LNP...334..242S" r = requests.post('https://api.adsabs.harvard.edu/v1/search/bigquery', params={'q':'*:*', 'wt':'json', 'fq':'{!bitset}', 'fl':'bibcode'}, - headers={'Authorization': 'Bearer: TOKEN'}, + headers={'Authorization': 'Bearer:TOKEN'}, data=bibcodes) ``` From 2815a69b037727da483eeb8bc6ec43eeeffc5fb1 Mon Sep 17 00:00:00 2001 From: Alberto Accomazzi Date: Mon, 26 Jun 2017 15:04:54 -0400 Subject: [PATCH 26/39] Update search.md --- search.md | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/search.md b/search.md index f320e76..a579cf6 100644 --- a/search.md +++ b/search.md @@ -11,6 +11,11 @@ Base URL: `https://api.adsabs.harvard.edu/v1/search` +Please note that all queries to the API need to include an HTTP header specifying an access token, e.g. + + curl -H 'Authorization: Bearer ' 'https://api.adsabs.harvard.edu/v1/search/query?q=star' + +For more information on this please see the top-level [README](README.md) file. ## Get search results @@ -45,11 +50,11 @@ Example response (value in the `qtree` is a string (JSON) serialized version of } ``` -## Post a large query +## Post a large identifier query POST /bigquery[?urlparams] -Returns standard search results, but accepts as input very large query (i.e. a query that can be expressed only as a list of search +Returns standard search results, but accepts as input a very large query (i.e. a query that can be expressed only as a list of search criteria, typically IDs). There is currently no limit to the size of the submitted data (besides buffer/time limits imposed by our API frontend), however there are severe limits on how often you can call this enpoint. Typically, only 100 requests per day are allowed. @@ -65,21 +70,19 @@ Currently, we allow to search in `bibcode` index only. You can submit `canonical The bigquery filter is *applied only after* the main search (ie: it limits results of the main search). -Example python session - get all papers from ADS and filter them using several IDs: +Example python session - get all papers from ADS and filter them using several IDs (thus returning only records for the list of input bibcodes): ```python import requests bibcodes="bibcode\n1907AN....174...59.\n1908PA.....16..445.\n1989LNP...334..242S" r = requests.post('https://api.adsabs.harvard.edu/v1/search/bigquery', params={'q':'*:*', 'wt':'json', 'fq':'{!bitset}', 'fl':'bibcode'}, - headers={'Authorization': 'Bearer:TOKEN'}, + headers={'Authorization': 'Bearer my_token'}, data=bibcodes) ``` Output is exactly the same as from `/query` endpoint. - - ## Query parameters From b65914b6138e35ea952cd701a242121e679d2854 Mon Sep 17 00:00:00 2001 From: Roman Chyla Date: Tue, 22 May 2018 15:57:22 -0400 Subject: [PATCH 27/39] Update search.md --- search.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/search.md b/search.md index a579cf6..015b2bf 100644 --- a/search.md +++ b/search.md @@ -83,6 +83,13 @@ r = requests.post('https://api.adsabs.harvard.edu/v1/search/bigquery', Output is exactly the same as from `/query` endpoint. +An example of a bigquery using curl: + +``` +curl -H "Content-Type: big-query/csv" -H "Authorization: Bearer {token}" "https://api.adsabs.harvard.edu/v1/search/bigquery?q=*:*&wt=json&fl=bibcode" -X POST -d $'bibcode\n1907AN....174...59.\n1908PA.....16..445.\n1989LNP...334..242S\n' + +``` + ## Query parameters From 93b5bdaecf8c70d7cc9e8c308ac934923290284d Mon Sep 17 00:00:00 2001 From: Kelly Lockhart <2926089+kelockhart@users.noreply.github.com> Date: Tue, 12 Jun 2018 12:23:51 -0400 Subject: [PATCH 28/39] Fix quotes typo in curl statement in docs --- metrics.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/metrics.md b/metrics.md index 65349e3..5c15534 100644 --- a/metrics.md +++ b/metrics.md @@ -23,7 +23,7 @@ and `token` is your API token. You can also request a specific set of metrics by adding the `types` parameter in the JSON header. For example: - curl -H "Authorization: Bearer " -H "Content-Type: application/json" -X POST -d '{"bibcodes":["1980ApJS...44..137K","1980ApJS...44..489B"],'types':['basic']}' https://api.adsabs.harvard.edu/v1/metrics + curl -H "Authorization: Bearer " -H "Content-Type: application/json" -X POST -d '{"bibcodes":["1980ApJS...44..137K","1980ApJS...44..489B"],"types":["basic"]}' https://api.adsabs.harvard.edu/v1/metrics which will return just the basic statistics for the set of bibcodes. The following values are allowed for the `types` list: @@ -254,4 +254,4 @@ So, returning everything, the output looks like where, as before, the 'values' have the form - {year: value, year: value, ...} \ No newline at end of file + {year: value, year: value, ...} From bca6224571e345d7a2ac5ba59c6a4af9f82a125e Mon Sep 17 00:00:00 2001 From: Kelly Lockhart <2926089+kelockhart@users.noreply.github.com> Date: Fri, 29 Jun 2018 14:25:21 -0400 Subject: [PATCH 29/39] Updated documentation and converted it to Jupyter notebooks --- Converting_curl_to_python.ipynb | 327 +++++++++++++ Export_API.ipynb | 183 ++++++++ Libraries_API.ipynb | 693 ++++++++++++++++++++++++++++ Metrics_API.ipynb | 790 ++++++++++++++++++++++++++++++++ README.md | 15 +- Search_API.ipynb | 613 +++++++++++++++++++++++++ export.md | 22 - libraries.md | 302 ------------ metrics.md | 257 ----------- search.md | 268 ----------- 10 files changed, 2616 insertions(+), 854 deletions(-) create mode 100644 Converting_curl_to_python.ipynb create mode 100644 Export_API.ipynb create mode 100644 Libraries_API.ipynb create mode 100644 Metrics_API.ipynb create mode 100644 Search_API.ipynb delete mode 100644 export.md delete mode 100644 libraries.md delete mode 100644 metrics.md delete mode 100644 search.md diff --git a/Converting_curl_to_python.ipynb b/Converting_curl_to_python.ipynb new file mode 100644 index 0000000..2a77099 --- /dev/null +++ b/Converting_curl_to_python.ipynb @@ -0,0 +1,327 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Converting `curl` requests to Python requests\n", + "All examples in these notebooks are given using `curl` statements from the shell command line. However, using the `requests` package in Python, it is possible to rewrite any of these `curl` statements and run them in Python. Some examples are given here. For more information about the `requests` package, please see their [official documentation](http://docs.python-requests.org/en/master/). If you're looking for a more robust Python package for ADS requests, see the [unofficial ADS Python package](http://ads.readthedocs.io/).\n", + "\n", + "Note: all examples are taken from other notebooks in this repository and converted to Python. See the original notebooks for more information about each query and for more query examples.\n", + "\n", + "### Contents\n", + "* [Search API examples](#Examples---Search-API)\n", + "* [Metrics API examples](#Examples---Metrics-API)\n", + "* [Export API examples](#Examples---Export-API)\n", + "* [Libraries API examples](#Examples---Library-API)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Examples - Search API\n", + "[Full Search API notebook and examples](Search_API.ipynb)\n", + "#### Get search results" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{u'responseHeader': {u'status': 0, u'QTime': 283, u'params': {u'x-amzn-trace-id': u'Root=1-5b3552fa-ab284eb0f5fc9060ec6a6920', u'rows': u'1', u'q': u'author:mart\\xednez neutron star', u'start': u'0', u'wt': u'json', u'fl': u'author'}}, u'response': {u'start': 0, u'numFound': 168, u'docs': [{u'author': [u'P\\xe9rez Rojas, Hugo', u'Mart\\xednez, Aurora P\\xe9rez', u'Mesquita, Alexandre', u'Razeira, Mois\\xe9s', u'Gomes, Rosana O.', u'Vasconcellos, C\\xe9sar A. Z.']}]}}\n" + ] + } + ], + "source": [ + "import requests\n", + "\n", + "token=\"your-token-here\"\n", + "# the query parameters can be included as part of the URL\n", + "r = requests.get(\"https://api.adsabs.harvard.edu/v1/search/query?q=author%3Amart%C3%ADnez+neutron+star&fl=author&rows=1\",\\\n", + " headers={'Authorization': 'Bearer ' + token})\n", + "# the requests package returns an object; to get just the JSON API response, you have to specify this\n", + "print(r.json())\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Post a large identifier query" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{u'responseHeader': {u'status': 0, u'QTime': 42, u'params': {u'x-amzn-trace-id': u'Root=1-5b35547f-f290c88fb5e9c182bab8287f', u'rows': u'2000', u'fq': u'{!bitset}', u'q': u'*:*', u'start': u'0', u'wt': u'json', u'fl': u'bibcode,title'}}, u'response': {u'start': 0, u'numFound': 3, u'docs': [{u'bibcode': u'1908PA.....16..445.', u'title': [u'Variable Stars']}, {u'bibcode': u'1907AN....174...59.', u'title': [u'Kleine Mitteilungen']}, {u'bibcode': u'1989LNP...334..242S', u'title': [u'The Optical and Radio Properties of X-Ray Selected Bl-Lacertae Objects']}]}}\n" + ] + } + ], + "source": [ + "# the query parameters can also be passed separately as a dictionary with the `params` keyword\n", + "bibcodes = \"bibcode\\n1907AN....174...59.\\n1908PA.....16..445.\\n1989LNP...334..242S\"\n", + "r = requests.post(\"https://api.adsabs.harvard.edu/v1/search/bigquery\",\\\n", + " params={\"q\":\"*:*\", \"fl\": \"bibcode,title\", \"rows\":2000},\n", + " headers={'Authorization': 'Bearer ' + token},\n", + " data=bibcodes)\n", + "print(r.json())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Examples - Metrics API\n", + "[Full Metrics API notebook and examples](Metrics_API.ipynb)\n", + "\n", + "**Get all metrics for a single bibcode**" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{u'basic stats': {u'median number of downloads': 8134.0, u'average number of reads': 19652.0, u'normalized paper count': 0.058823529411764705, u'recent number of reads': 454, u'number of papers': 1, u'recent number of downloads': 144, u'total number of reads': 19652, u'median number of reads': 19652.0, u'total number of downloads': 8134, u'average number of downloads': 8134.0}, u'citation stats refereed': {u'normalized number of citations': 513.5294117647059, u'average number of refereed citations': 7500.0, u'median number of citations': 8730.0, u'median number of refereed citations': 7500.0, u'number of citing papers': 8730, u'average number of citations': 8730.0, u'total number of refereed citations': 7500, u'normalized number of refereed citations': 441.1764705882353, u'number of self-citations': 0, u'total number of citations': 8730}, u'histograms': {u'reads': {u'all reads normalized': {u'2018': 26.705882352941178, u'2015': 43.705882352941174, u'2014': 54.05882352941177, u'2017': 46.294117647058826, u'2016': 46.88235294117647, u'2011': 80.47058823529412, u'2010': 79.70588235294117, u'2013': 54.64705882352941, u'2012': 79.29411764705883, u'1997': 0.0, u'1996': 0.0, u'1999': 0.0, u'1998': 0.0, u'2002': 0.0, u'2003': 49.35294117647059, u'2000': 0.0, u'2001': 0.0, u'2006': 112.29411764705883, u'2007': 101.11764705882354, u'2004': 117.58823529411765, u'2005': 108.11764705882354, u'2008': 82.6470588235294, u'2009': 73.11764705882354}, u'refereed reads': {u'2018': 454, u'2015': 743, u'2014': 919, u'2017': 787, u'2016': 797, u'2011': 1368, u'2010': 1355, u'2013': 929, u'2012': 1348, u'1997': 0, u'1996': 0, u'1999': 0, u'1998': 0, u'2002': 0, u'2003': 839, u'2000': 0, u'2001': 0, u'2006': 1909, u'2007': 1719, u'2004': 1999, u'2005': 1838, u'2008': 1405, u'2009': 1243}, u'all reads': {u'2018': 454, u'2015': 743, u'2014': 919, u'2017': 787, u'2016': 797, u'2011': 1368, u'2010': 1355, u'2013': 929, u'2012': 1348, u'1997': 0, u'1996': 0, u'1999': 0, u'1998': 0, u'2002': 0, u'2003': 839, u'2000': 0, u'2001': 0, u'2006': 1909, u'2007': 1719, u'2004': 1999, u'2005': 1838, u'2008': 1405, u'2009': 1243}, u'refereed reads normalized': {u'2018': 26.705882352941178, u'2015': 43.705882352941174, u'2014': 54.05882352941177, u'2017': 46.294117647058826, u'2016': 46.88235294117647, u'2011': 80.47058823529412, u'2010': 79.70588235294117, u'2013': 54.64705882352941, u'2012': 79.29411764705883, u'1997': 0.0, u'1996': 0.0, u'1999': 0.0, u'1998': 0.0, u'2002': 0.0, u'2003': 49.35294117647059, u'2000': 0.0, u'2001': 0.0, u'2006': 112.29411764705883, u'2007': 101.11764705882354, u'2004': 117.58823529411765, u'2005': 108.11764705882354, u'2008': 82.6470588235294, u'2009': 73.11764705882354}}, u'citations': {u'refereed to nonrefereed': {u'2016': 0, u'2017': 0, u'2018': 0, u'2002': 0, u'2003': 0, u'2001': 0, u'2006': 0, u'2007': 0, u'2004': 0, u'2005': 0, u'2015': 0, u'2014': 0, u'2008': 0, u'2009': 0, u'2011': 0, u'2010': 0, u'2013': 0, u'2012': 0}, u'nonrefereed to nonrefereed': {u'2016': 0, u'2017': 0, u'2018': 0, u'2002': 0, u'2003': 0, u'2001': 0, u'2006': 0, u'2007': 0, u'2004': 0, u'2005': 0, u'2015': 0, u'2014': 0, u'2008': 0, u'2009': 0, u'2011': 0, u'2010': 0, u'2013': 0, u'2012': 0}, u'nonrefereed to nonrefereed normalized': {u'2009': 0, u'2014': 0, u'2010': 0, u'2002': 0, u'2003': 0, u'2001': 0, u'2006': 0, u'2018': 0, u'2004': 0, u'2005': 0, u'2015': 0, u'2012': 0, u'2017': 0, u'2016': 0, u'2011': 0, u'2008': 0, u'2013': 0, u'2007': 0}, u'nonrefereed to refereed': {u'2016': 35, u'2017': 39, u'2018': 42, u'2002': 2, u'2003': 95, u'2001': 0, u'2006': 152, u'2007': 101, u'2004': 148, u'2005': 237, u'2015': 34, u'2014': 30, u'2008': 75, u'2009': 70, u'2011': 35, u'2010': 52, u'2013': 36, u'2012': 47}, u'refereed to refereed normalized': {u'2009': 27.176470588235205, u'2014': 16.588235294117595, u'2010': 24.176470588235215, u'2002': 0, u'2003': 22.29411764705875, u'2001': 0.058823529411764705, u'2006': 58.64705882353082, u'2018': 5.176470588235283, u'2004': 58.35294117647198, u'2005': 60.35294117647209, u'2015': 14.705882352941131, u'2012': 20.117647058823465, u'2017': 10.999999999999968, u'2016': 11.999999999999964, u'2011': 21.294117647058755, u'2008': 31.352941176470484, u'2013': 18.647058823529353, u'2007': 39.23529411764736}, u'refereed to nonrefereed normalized': {u'2009': 0, u'2014': 0, u'2010': 0, u'2002': 0, u'2003': 0, u'2001': 0, u'2006': 0, u'2018': 0, u'2004': 0, u'2005': 0, u'2015': 0, u'2012': 0, u'2017': 0, u'2016': 0, u'2011': 0, u'2008': 0, u'2013': 0, u'2007': 0}, u'refereed to refereed': {u'2016': 204, u'2017': 187, u'2018': 88, u'2002': 0, u'2003': 379, u'2001': 1, u'2006': 997, u'2007': 667, u'2004': 992, u'2005': 1026, u'2015': 250, u'2014': 282, u'2008': 533, u'2009': 462, u'2011': 362, u'2010': 411, u'2013': 317, u'2012': 342}, u'nonrefereed to refereed normalized': {u'2009': 4.117647058823522, u'2014': 1.7647058823529413, u'2010': 3.058823529411761, u'2002': 0.11764705882352941, u'2003': 5.588235294117634, u'2001': 0, u'2006': 8.94117647058821, u'2018': 2.470588235294116, u'2004': 8.705882352941153, u'2005': 13.941176470588193, u'2015': 2.0, u'2012': 2.7647058823529385, u'2017': 2.2941176470588225, u'2016': 2.0588235294117645, u'2011': 2.0588235294117645, u'2008': 4.411764705882344, u'2013': 2.117647058823529, u'2007': 5.941176470588221}}}, u'citation stats': {u'number of self-citations': 0, u'average number of refereed citations': 7500.0, u'median number of citations': 8730.0, u'self-citations': [], u'number of citing papers': 8730, u'average number of citations': 8730.0, u'total number of refereed citations': 7500, u'normalized number of refereed citations': 441.1764705882353, u'median number of refereed citations': 7500.0, u'total number of citations': 8730, u'normalized number of citations': 513.5294117647059}, u'basic stats refereed': {u'median number of downloads': 8134.0, u'average number of reads': 19652.0, u'normalized paper count': 0.058823529411764705, u'recent number of reads': 454, u'number of papers': 1, u'recent number of downloads': 144, u'total number of reads': 19652, u'median number of reads': 19652.0, u'total number of downloads': 8134, u'average number of downloads': 8134.0}, u'skipped bibcodes': []}\n" + ] + } + ], + "source": [ + "import json\n", + "\n", + "# to pass a dictionary in the request payload, convert it to a string first using the json package\n", + "bibcode = {\"bibcodes\":[\"2003ApJS..148..175S\"]}\n", + "r = requests.post(\"https://api.adsabs.harvard.edu/v1/metrics\", \\\n", + " headers={\"Authorization\": \"Bearer \" + token, \"Content-type\": \"application/json\"}, \\\n", + " data=json.dumps(bibcode))\n", + "print(r.json())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "** Get data for a single histogram for multiple bibcodes**" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{u'skipped bibcodes': [], u'histograms': {u'citations': {u'refereed to nonrefereed': {u'2016': 0, u'2017': 0, u'2018': 0, u'2002': 0, u'2003': 0, u'2001': 0, u'2006': 0, u'2007': 0, u'2004': 0, u'2005': 0, u'2015': 0, u'2014': 0, u'2008': 0, u'2009': 0, u'2011': 0, u'2010': 0, u'2013': 0, u'2012': 0}, u'nonrefereed to nonrefereed': {u'2016': 0, u'2017': 0, u'2018': 0, u'2002': 0, u'2003': 0, u'2001': 0, u'2006': 0, u'2007': 0, u'2004': 0, u'2005': 0, u'2015': 0, u'2014': 0, u'2008': 0, u'2009': 0, u'2011': 0, u'2010': 0, u'2013': 0, u'2012': 0}, u'nonrefereed to nonrefereed normalized': {u'2009': 0, u'2014': 0, u'2010': 0, u'2002': 0, u'2003': 0, u'2001': 0, u'2006': 0, u'2018': 0, u'2004': 0, u'2005': 0, u'2015': 0, u'2012': 0, u'2017': 0, u'2016': 0, u'2011': 0, u'2008': 0, u'2013': 0, u'2007': 0}, u'nonrefereed to refereed': {u'2016': 56, u'2017': 74, u'2018': 74, u'2002': 2, u'2003': 95, u'2001': 0, u'2006': 273, u'2007': 294, u'2004': 148, u'2005': 238, u'2015': 57, u'2014': 53, u'2008': 239, u'2009': 184, u'2011': 85, u'2010': 119, u'2013': 66, u'2012': 83}, u'refereed to refereed normalized': {u'2009': 57.085561497326964, u'2014': 25.815508021390585, u'2010': 47.54010695187224, u'2002': 0, u'2003': 22.29411764705875, u'2001': 0.058823529411764705, u'2006': 78.69251336898593, u'2018': 7.994652406417126, u'2004': 58.35294117647198, u'2005': 60.35294117647209, u'2015': 22.56951871657767, u'2012': 34.16310160427841, u'2017': 16.954545454545396, u'2016': 18.181818181818166, u'2011': 40.157754010695655, u'2008': 83.21657754010832, u'2013': 31.147058823529708, u'2007': 87.82620320855783}, u'refereed to nonrefereed normalized': {u'2009': 0, u'2014': 0, u'2010': 0, u'2002': 0, u'2003': 0, u'2001': 0, u'2006': 0, u'2018': 0, u'2004': 0, u'2005': 0, u'2015': 0, u'2012': 0, u'2017': 0, u'2016': 0, u'2011': 0, u'2008': 0, u'2013': 0, u'2007': 0}, u'refereed to refereed': {u'2016': 340, u'2017': 318, u'2018': 150, u'2002': 0, u'2003': 379, u'2001': 1, u'2006': 1438, u'2007': 1736, u'2004': 992, u'2005': 1026, u'2015': 423, u'2014': 485, u'2008': 1674, u'2009': 1120, u'2011': 777, u'2010': 925, u'2013': 592, u'2012': 651}, u'nonrefereed to refereed normalized': {u'2009': 9.299465240641725, u'2014': 2.810160427807486, u'2010': 6.104278074866325, u'2002': 0.11764705882352941, u'2003': 5.588235294117634, u'2001': 0, u'2006': 14.441176470588152, u'2018': 3.9251336898395692, u'2004': 8.705882352941153, u'2005': 13.986631016042738, u'2015': 3.0454545454545445, u'2012': 4.401069518716577, u'2017': 3.885026737967912, u'2016': 3.013368983957218, u'2011': 4.331550802139039, u'2008': 11.866310160427789, u'2013': 3.4812834224598914, u'2007': 14.713903743315441}}}}\n" + ] + } + ], + "source": [ + "# to pass a dictionary in the request payload, convert it to a string first using the json package\n", + "payload = {\"bibcodes\":[\"2003ApJS..148..175S\", \"2007ApJS..170..377S\"], \\\n", + " \"types\":[\"histograms\"], \"histograms\":[\"citations\"]}\n", + "r = requests.post(\"https://api.adsabs.harvard.edu/v1/metrics\", \\\n", + " headers={\"Authorization\": \"Bearer \" + token, \"Content-type\": \"application/json\"}, \\\n", + " data=json.dumps(payload))\n", + "print(r.json())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Examples - Export API \n", + "[Full Export API notebook and examples](Export_API.ipynb)\n", + "\n", + "**Get the BibTeX entry for a single bibcode**" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{u'msg': u'Retrieved 1 abstracts, starting with number 1.', u'export': u'@ARTICLE{2015RaSc...50..916A,\\n author = {{Ads}, A.~G. and {Bergad{\\\\`a}}, P. and {Regu{\\\\\\'e}}, J.~R. and {Alsina-\\n Pag{\\\\`e}s}, R.~M. and {Pijoan}, J.~L. and {Altadill}, D. and\\n {Badia}, D. and {Graells}, S.},\\n title = \"{Vertical and oblique ionospheric soundings over the long haul HF link\\n between Antarctica and Spain}\",\\n journal = {Radio Science},\\n keywords = {VIS, OIS, ionosphere},\\n year = 2015,\\n month = Sep,\\n volume = {50},\\n pages = {916-930},\\n doi = {10.1002/2015RS005773},\\n adsurl = {https://ui.adsabs.harvard.edu/#abs/2015RaSc...50..916A},\\n adsnote = {Provided by the SAO/NASA Astrophysics Data System}\\n}\\n\\n'}\n" + ] + } + ], + "source": [ + "bibcode = {\"bibcode\":[\"2015RaSc...50..916A\"]}\n", + "r = requests.post(\"https://api.adsabs.harvard.edu/v1/export/bibtex\", \\\n", + " headers={\"Authorization\": \"Bearer \" + token, \"Content-type\": \"application/json\"}, \\\n", + " data=json.dumps(bibcode))\n", + "print(r.json())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "** Get the AASTeX entries for multiple bibcodes**" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{u'msg': u'Retrieved 3 abstracts, starting with number 1.', u'export': u'\\\\bibitem[Accomazzi, et al.(2000)]{2000A&AS..143...85A} Accomazzi, A., Eichhorn, G., Kurtz, M.~J., et al.\\\\ 2000, Astronomy and Astrophysics Supplement Series, 143, 85.\\n\\\\bibitem[Grant, et al.(2000)]{2000A&AS..143..111G} Grant, C.~S., Accomazzi, A., Eichhorn, G., et al.\\\\ 2000, Astronomy and Astrophysics Supplement Series, 143, 111.\\n\\\\bibitem[Kurtz, et al.(2000)]{2000A&AS..143...41K} Kurtz, M.~J., Eichhorn, G., Accomazzi, A., et al.\\\\ 2000, Astronomy and Astrophysics Supplement Series, 143, 41.\\n'}\n" + ] + } + ], + "source": [ + "# get the AASTeX entries for multiple bibcodes\n", + "payload = {\"bibcode\":[\"2000A&AS..143...41K\", \"2000A&AS..143...85A\", \"2000A&AS..143..111G\"],\\\n", + " \"sort\": \"first_author asc\"}\n", + "r = requests.post(\"https://api.adsabs.harvard.edu/v1/export/aastex\", \\\n", + " headers={\"Authorization\": \"Bearer \" + token, \"Content-type\": \"application/json\"}, \\\n", + " data=json.dumps(payload))\n", + "print(r.json())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Get custom formatted entries for multiple bibcodes**" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{u'msg': u'Retrieved 3 abstracts, starting with number 1.', u'export': u'Accomazzi, Eichhorn, Kurtz, Grant \\\\& Murray 2000\\nGrant, Accomazzi, Eichhorn, Kurtz \\\\& Murray 2000\\nKurtz, Eichhorn, Accomazzi, Grant, Murray \\\\& Watson 2000\\n\\n'}\n" + ] + } + ], + "source": [ + "payload = {\"bibcode\":[\"2000A&AS..143...41K\", \"2000A&AS..143...85A\", \"2000A&AS..143..111G\"],\\\n", + " \"sort\": \"first_author asc\",\\\n", + " \"format\": \"%m %Y\"}\n", + "r = requests.post(\"https://api.adsabs.harvard.edu/v1/export/custom\", \\\n", + " headers={\"Authorization\": \"Bearer \" + token, \"Content-type\": \"application/json\"}, \\\n", + " data=json.dumps(payload))\n", + "print(r.json())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Examples - Library API\n", + "[Full Libraries API notebook and examples](Libraries_API.ipynb)\n", + "\n", + "**Get all your libraries**" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{u'libraries': [{u'num_documents': 163, u'description': u'Papers about ADS', u'name': u'ADS', u'permission': u'owner', u'id': u'ZX9Mgv67QSyS6GfuYLjB7w', u'num_users': 1, u'owner': u'test.user-api', u'date_created': u'2018-06-28T15:30:24.764684', u'public': False, u'date_last_modified': u'2018-06-28T15:33:01.545962'}, {u'num_documents': 100, u'description': u'My ADS library', u'name': u'WMAP', u'permission': u'owner', u'id': u'HNHFavGNS3izENGEbGDJCA', u'num_users': 1, u'owner': u'test.user-api', u'date_created': u'2018-06-28T15:31:12.741894', u'public': False, u'date_last_modified': u'2018-06-28T15:32:10.641190'}]}\n" + ] + } + ], + "source": [ + "lib_token = \"your-token-here\"\n", + "r = requests.get(\"https://api.adsabs.harvard.edu/v1/biblib/libraries\", \\\n", + " headers={\"Authorization\": \"Bearer \" + lib_token})\n", + "print(r.json())" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Get data for a specific library**" + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{u'documents': [u'2018AAS...23221409A', u'2018arXiv180303598K', u'2018AAS...23130709A', u'2018AAS...23136217A', u'2017arXiv171008505A', u'2017arXiv170909566R', u'2016arXiv160107858A', u'2015scop.confE...3A', u'2015ASPC..495..401C', u'2015IAUGA..2257639R', u'2015IAUGA..2257982A', u'2015IAUGA..2257768A', u'2015ASPC..492..208G', u'2015ASPC..492...85E', u'2015ASPC..492..150T', u'2015ASPC..492..189A', u'2015ASPC..492...80H', u'2015AAS...22533655A', u'2014AAS...22325503A', u'2014AAS...22325525A'], u'solr': {u'responseHeader': {u'status': 0, u'QTime': 22, u'params': {u'sort': u'date desc', u'fq': u'{!bitset}', u'rows': u'20', u'q': u'*:*', u'start': u'0', u'wt': u'json', u'fl': u'bibcode,alternate_bibcode'}}, u'response': {u'start': 0, u'numFound': 163, u'docs': [{u'bibcode': u'2018AAS...23221409A'}, {u'bibcode': u'2018arXiv180303598K'}, {u'bibcode': u'2018AAS...23130709A'}, {u'bibcode': u'2018AAS...23136217A'}, {u'bibcode': u'2017arXiv171008505A'}, {u'bibcode': u'2017arXiv170909566R'}, {u'bibcode': u'2016arXiv160107858A'}, {u'bibcode': u'2015scop.confE...3A'}, {u'alternate_bibcode': [u'2015arXiv150305881C', u'2015adass..24..401C'], u'bibcode': u'2015ASPC..495..401C'}, {u'bibcode': u'2015IAUGA..2257639R'}, {u'bibcode': u'2015IAUGA..2257982A'}, {u'bibcode': u'2015IAUGA..2257768A'}, {u'bibcode': u'2015ASPC..492..208G'}, {u'bibcode': u'2015ASPC..492...85E'}, {u'bibcode': u'2015ASPC..492..150T'}, {u'alternate_bibcode': [u'2015arXiv150304194A'], u'bibcode': u'2015ASPC..492..189A'}, {u'alternate_bibcode': [u'2014arXiv1406.4542H'], u'bibcode': u'2015ASPC..492...80H'}, {u'bibcode': u'2015AAS...22533655A'}, {u'bibcode': u'2014AAS...22325503A'}, {u'bibcode': u'2014AAS...22325525A'}]}}, u'updates': {u'update_list': [], u'num_updated': 0, u'duplicates_removed': 0}, u'metadata': {u'num_documents': 163, u'description': u'Papers about ADS', u'name': u'ADS', u'permission': u'owner', u'id': u'ZX9Mgv67QSyS6GfuYLjB7w', u'num_users': 1, u'owner': u'test.user-api', u'date_created': u'2018-06-28T15:30:24.764684', u'public': False, u'date_last_modified': u'2018-06-28T15:33:01.545962'}}\n" + ] + } + ], + "source": [ + "# get the data for a specific library\n", + "r = requests.get(\"https://api.adsabs.harvard.edu/v1/biblib/libraries/ZX9Mgv67QSyS6GfuYLjB7w\", \\\n", + " headers={\"Authorization\": \"Bearer \" + lib_token})\n", + "print(r.json())" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 2", + "language": "python", + "name": "python2" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 2 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython2", + "version": "2.7.14" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/Export_API.ipynb b/Export_API.ipynb new file mode 100644 index 0000000..0ceb304 --- /dev/null +++ b/Export_API.ipynb @@ -0,0 +1,183 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Export service\n", + "This notebook explains how to retrieve data in BibTeX and other formats for one or more bibcodes.\n", + "\n", + "The base URL for export queries is:\n", + "```\n", + "https://api.adsabs.harvard.edu/v1/export/\n", + "```\n", + "\n", + "where `` should be replaced by the desired export format. For example, to get BibTeX for a set of records, send a request with the POST method to the URL `https://api.adsabs.harvard.edu/v1/export/bibtex`. The following formats are currently supported:\n", + "* BibTeX (`/bibtex`)\n", + "* BibTeX ABS (`/bibtexabs`)\n", + "* ADS (`/ads`)\n", + "* EndNote (`/endnote`)\n", + "* ProCite (`/procite`)\n", + "* RIS (`/ris`)\n", + "* RefWorks (`/refworks`)\n", + "* RSS (`/rss`)\n", + "* MEDLARS (`/medlars`)\n", + "* DC-XML (`/dcxml`)\n", + "* REF-XML (`/refxml`)\n", + "* REFABS-XML (`/refabsxml`)\n", + "* AASTeX (`/aastex`)\n", + "* Icarus (`/icarus`)\n", + "* MNRAS (`/mnras`)\n", + "* Solar Physics (`/soph`)\n", + "* VOTable (`/votable`)\n", + "* Custom format (`/custom`)\n", + "\n", + "The POST payload (set via the `-d` flag) should be in JSON format. To indicate the payload format, you must add the following header to your `curl` request: `-H \"Content-Type: application/json\"`. For the export, the payload accepts the following keys: \n", + "* `bibcode`: **required**; set the value to a comma-separated list of the bibcodes to export\n", + "* `sort`: optional; set the value to a combination of the field to sort on (see the Fields section in the Search API notebook), followed by a space plus `asc` or `desc` to indicate ascending or descending order. If the sort order is not specified, the default sort is by date, then by bibcode \n", + "* `format`: **required when using custom format**; set the value to a string with the [desired formatting codes](http://adsabs.github.io/help/actions/export)\n", + "\n", + "For example, for most formats the POST payload would take this format: `{\"bibcode\": [\"\",\"\", ...], \"sort\": \" \"}`. For custom format exports, the POST payload would take this format: `{\"bibcode\": [\"\",\"\", ...], \"sort\": \" \", \"format\": \"\"}`\n", + "\n", + "The complete `curl` request should also include your personal API access token, as described in the top-level [README](https://github.com/adsabs/adsabs-dev-api/blob/master/README.md). The complete `curl` request follows this format:\n", + "```\n", + " curl -H \"Authorization: Bearer \" -H \"Content-Type: application/json\" https://api.adsabs.harvard.edu/v1/export/ -X POST -d '{\"bibcode\":[\"\",\"\", ...]}' \n", + "```\n", + "\n", + "Note: the API examples in this notebook assume you're using `curl` statements from the shell command line--you'll see `%%bash` at the top of a cell if its contents should be run in the shell instead of in Python. Guidelines for translating bash `curl` statements into Python can be found in the [converting curl to python notebook](Converting_curl_to_python.ipynb). If you prefer, you can use the unofficial Python package instead; see the top-level [README](https://github.com/adsabs/adsabs-dev-api/blob/master/README.md) for more information." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Examples" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"msg\": \"Retrieved 1 abstracts, starting with number 1.\", \"export\": \"@ARTICLE{2015RaSc...50..916A,\\n author = {{Ads}, A.~G. and {Bergad{\\\\`a}}, P. and {Regu{\\\\'e}}, J.~R. and {Alsina-\\n Pag{\\\\`e}s}, R.~M. and {Pijoan}, J.~L. and {Altadill}, D. and\\n {Badia}, D. and {Graells}, S.},\\n title = \\\"{Vertical and oblique ionospheric soundings over the long haul HF link\\n between Antarctica and Spain}\\\",\\n journal = {Radio Science},\\n keywords = {VIS, OIS, ionosphere},\\n year = 2015,\\n month = Sep,\\n volume = {50},\\n pages = {916-930},\\n doi = {10.1002/2015RS005773},\\n adsurl = {https://ui.adsabs.harvard.edu/#abs/2015RaSc...50..916A},\\n adsnote = {Provided by the SAO/NASA Astrophysics Data System}\\n}\\n\\n\"}" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 833 100 798 100 35 798 35 0:00:01 --:--:-- 0:00:01 3062\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# get the BibTeX entry for a single bibcode\n", + "curl -H \"Authorization: Bearer $token\" -H \"Content-Type: application/json\" \\\n", + " https://api.adsabs.harvard.edu/v1/export/bibtex \\\n", + " -X POST -d '{\"bibcode\":[\"2015RaSc...50..916A\"]}'" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"msg\": \"Retrieved 3 abstracts, starting with number 1.\", \"export\": \"\\\\bibitem[Accomazzi, et al.(2000)]{2000A&AS..143...85A} Accomazzi, A., Eichhorn, G., Kurtz, M.~J., et al.\\\\ 2000, Astronomy and Astrophysics Supplement Series, 143, 85.\\n\\\\bibitem[Grant, et al.(2000)]{2000A&AS..143..111G} Grant, C.~S., Accomazzi, A., Eichhorn, G., et al.\\\\ 2000, Astronomy and Astrophysics Supplement Series, 143, 111.\\n\\\\bibitem[Kurtz, et al.(2000)]{2000A&AS..143...41K} Kurtz, M.~J., Eichhorn, G., Accomazzi, A., et al.\\\\ 2000, Astronomy and Astrophysics Supplement Series, 143, 41.\\n\"}" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 684 100 574 100 110 574 110 0:00:01 --:--:-- 0:00:01 915\r", + "100 684 100 574 100 110 574 110 0:00:01 --:--:-- 0:00:01 914\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# get the AASTeX entries for multiple bibcodes, sorting by first author\n", + "curl -H \"Authorization: Bearer $token\" -H \"Content-Type: application/json\" \\\n", + " https://api.adsabs.harvard.edu/v1/export/aastex \\\n", + " -X POST \\\n", + " -d '{\"bibcode\": [\"2000A&AS..143...41K\", \"2000A&AS..143...85A\", \"2000A&AS..143..111G\"], \"sort\": \"first_author asc\"}'" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"msg\": \"Retrieved 3 abstracts, starting with number 1.\", \"export\": \"Accomazzi, Eichhorn, Kurtz, Grant \\\\& Murray 2000\\nGrant, Accomazzi, Eichhorn, Kurtz \\\\& Murray 2000\\nKurtz, Eichhorn, Accomazzi, Grant, Murray \\\\& Watson 2000\\n\\n\"}" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 363 100 234 100 129 234 129 0:00:01 --:--:-- 0:00:01 1354\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# same as above, but exported using a custom format\n", + "curl -H \"Authorization: Bearer $token\" -H \"Content-Type: application/json\" \\\n", + " https://api.adsabs.harvard.edu/v1/export/custom \\\n", + " -X POST \\\n", + " -d '{\"format\": \"%m %Y\", \"bibcode\": [\"2000A&AS..143...41K\", \"2000A&AS..143...85A\", \"2000A&AS..143..111G\"], \"sort\": \"first_author asc\"}'" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 2", + "language": "python", + "name": "python2" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 2 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython2", + "version": "2.7.14" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/Libraries_API.ipynb b/Libraries_API.ipynb new file mode 100644 index 0000000..5f41c40 --- /dev/null +++ b/Libraries_API.ipynb @@ -0,0 +1,693 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Libraries\n", + "This notebook explains how to retrieve data for and edit private and public ADS libraries.\n", + "\n", + "The base URL for library queries is\n", + "```\n", + "https://api.adsabs.harvard.edu/v1/biblib/\n", + "```\n", + "where the allowed `` values are:\n", + "* `/libraries`: create or view all libraries for a given user; GET or POST methods allowed\n", + "* `/libraries/`: view a specific library; GET method is allowed\n", + "* `/documents/`: add and remove records from a specific library, update the metadata for a specific library, or delete an entire library; POST, PUT, and DELETE methods are allowed\n", + "* `/permissions/`: view or modify the permissions for a specific library; GET and POST methods are allowed\n", + "* `/transfer/`: transfer ownership of a library to another ADS user; GET and POST methods are allowed\n", + "\n", + "These endpoints are discussed further below, with examples.\n", + "\n", + "Note that the complete curl request should also include your personal API access token, as described in the top-level [README](https://github.com/adsabs/adsabs-dev-api/blob/master/README.md).\n", + "\n", + "The API examples in this notebook generally assume you're using `curl` statements from the shell command line--you'll see `%%bash` at the top of a cell if its contents should be run in the shell instead of in Python. Guidelines for translating bash `curl` statements into Python can be found in the [converting curl to python notebook](Converting_curl_to_python.ipynb). If you prefer, you can use the unofficial Python package instead; see the top-level [README](https://github.com/adsabs/adsabs-dev-api/blob/master/README.md) for more information." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Permissions\n", + " * **owner**: Owns the library. Can read libraries, add/remove bibcodes, delete the library, add/remove permissions, transfer ownership of the library.\n", + " * **admin**: Can read libraries, add/remove bibcodes, add/remove permissions.\n", + " * **write**: Can read libraries, add/remove bibcdes.\n", + " * **read**: Can read libraries.\n", + "\n", + "Any other user that does not have a permission to the library they try to visit, can only see it if it is public. If the user has no permissions, they cannot view this library." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `/libraries`\n", + "Create a library or view all libraries for a given user, depending on the method used.\n", + "\n", + "#### GET method\n", + "View all libraries for the user associated with the given API token.\n", + "\n", + "*Return data*:\n", + " * name: (string) name of the library\n", + " * id: (string) ID of the library\n", + " * description: (string) description of the library\n", + " * num_documents: (integer) number of documents in the library\n", + " * date_created: (string) ISO date library was created\n", + " * date_last_modified: (string) ISO date library was last modified\n", + " * permission: (string) permission type, can be: 'read', 'write', 'admin', or 'owner'\n", + " * public: (boolean) true means it is a public library\n", + " * num_users: (integer) number of users with permissions to this library\n", + " * owner: (string) ID of the user who created the library" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"libraries\": [{\"num_documents\": 163, \"date_last_modified\": \"2018-06-28T15:30:24.764691\", \"permission\": \"owner\", \"description\": \"My ADS library\", \"public\": false, \"num_users\": 1, \"owner\": \"test.user-api\", \"date_created\": \"2018-06-28T15:30:24.764684\", \"id\": \"ZX9Mgv67QSyS6GfuYLjB7w\", \"name\": \"ADS\"}, {\"num_documents\": 100, \"date_last_modified\": \"2018-06-28T15:32:10.641190\", \"permission\": \"owner\", \"description\": \"My ADS library\", \"public\": false, \"num_users\": 1, \"owner\": \"test.user-api\", \"date_created\": \"2018-06-28T15:31:12.741894\", \"id\": \"HNHFavGNS3izENGEbGDJCA\", \"name\": \"WMAP\"}]}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 595 100 595 0 0 595 0 0:00:01 --:--:-- 0:00:01 1882\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "curl -H \"Authorization: Bearer $token\" https://api.adsabs.harvard.edu/v1/biblib/libraries" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### POST method\n", + "Create a library for the requesting user. Returns the metadata of the library created.\n", + "\n", + "In addition to the usual token header and request URL, you must specify that this is a POST request using `-X POST`. The POST payload must also be passed using the `-d` flag and should be in JSON format. The payload should include the following keys:\n", + " * name: (string) optional; name of the library (must be unique for that user). The default is `Untitled Library`, with an incremented integer starting with 1 appended\n", + " * description: (string) optional; description of the library. The default is \"My ADS library\"\n", + " * public: (boolean) optional; set to true to allow public access. Note: the Boolean value is case senstitive, and must be entered in lowercase. The default is false, or a private library\n", + " * bibcode: (list) optional; list of bibcodes to add. The default is an empty library\n", + " \n", + "To indicate that the payload is in JSON format, you must include a header specifying the format: `-H \"Content-Type: application/json\"`.\n", + "\n", + "The metadata of the library you just created will be returned after the request:\n", + " * name: (string) name of the library\n", + " * id: (string) ID of the library\n", + " * description: (string) description of the library" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"id\": \"hHGU1Ef-TpacAhicI3J8kQ\", \"bibcode\": [\"1975CMaPh..43..199H\", \"1973PhRvD...7.2333B\"], \"description\": \"This library is created via API\", \"name\": \"API library\"}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 308 100 165 100 143 165 143 0:00:01 --:--:-- 0:00:01 980\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "curl -H \"Authorization: Bearer $token\" -H \"Content-Type: application/json\" \\\n", + " https://api.adsabs.harvard.edu/v1/biblib/libraries \\\n", + " -X POST \\\n", + " -d '{\"name\":\"API library\", \"description\":\"This library is created via API\", \"public\":true, \"bibcode\":[\"1975CMaPh..43..199H\",\"1973PhRvD...7.2333B\"]}'\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### /libraries/<library_id>\n", + "View a specific library. The `library_id` (string) is the unique identifier of the library, given via the `/libraries` GET method detailed above.\n", + "\n", + "To edit or delete a library, see the `/document/<library_id>` endpoint section below.\n", + "\n", + "#### GET\n", + "Obtains all the documents in a given library. \n", + "\n", + "*Return data*:\n", + " * documents: (list) a list containing the bibcodes in the library\n", + " * solr: (dict) the response from the solr bigquery endpoint\n", + " * metadata: (dict) Contains the following:\n", + " * name: (string) name of the library\n", + " * id: (string) ID of the library\n", + " * description: (string) description of the library\n", + " * num_documents: (integer) number of documents in the library\n", + " * date_created: (string) ISO date library was created\n", + " * date_last_modified: (string) ISO date library was last modified\n", + " * permission: (sting) Permission type, can be 'read', 'write', 'admin', or 'owner'\n", + " * public: (boolean) true means it is public\n", + " * num_users: (integer) number of users with permissions to this library\n", + " * owner: (string) identifier of the user who created the library\n", + " * updates: (dict) Contains the following\n", + " * num_updated: (integer) number of documents modified based on the response from solr\n", + " * duplicates_removed: (integer) number of files removed because they are duplications\n", + " * update_list: (list, dict) list of dictionaries such that a single element described the original bibcode (key) and the updated bibcode now being stored (item)\n", + "\n", + "The following type of user can read a library:\n", + " - owner\n", + " - admin\n", + " - write\n", + " - read" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\n", + " \"documents\": [\n", + " \"1975CMaPh..43..199H\",\n", + " \"1973PhRvD...7.2333B\"\n", + " ],\n", + " \"metadata\": {\n", + " \"date_created\": \"2018-06-28T17:50:14.641552\",\n", + " \"date_last_modified\": \"2018-06-28T17:50:14.641558\",\n", + " \"description\": \"This library is created via API\",\n", + " \"id\": \"hHGU1Ef-TpacAhicI3J8kQ\",\n", + " \"name\": \"API library\",\n", + " \"num_documents\": 2,\n", + " \"num_users\": 1,\n", + " \"owner\": \"test.user-api\",\n", + " \"permission\": \"owner\",\n", + " \"public\": true\n", + " },\n", + " \"solr\": {\n", + " \"response\": {\n", + " \"docs\": [\n", + " {\n", + " \"bibcode\": \"1975CMaPh..43..199H\"\n", + " },\n", + " {\n", + " \"bibcode\": \"1973PhRvD...7.2333B\"\n", + " }\n", + " ],\n", + " \"numFound\": 2,\n", + " \"start\": 0\n", + " },\n", + " \"responseHeader\": {\n", + " \"QTime\": 22,\n", + " \"params\": {\n", + " \"fl\": \"bibcode,alternate_bibcode\",\n", + " \"fq\": \"{!bitset}\",\n", + " \"q\": \"*:*\",\n", + " \"rows\": \"20\",\n", + " \"sort\": \"date desc\",\n", + " \"start\": \"0\",\n", + " \"wt\": \"json\"\n", + " },\n", + " \"status\": 0\n", + " }\n", + " },\n", + " \"updates\": {\n", + " \"duplicates_removed\": 0,\n", + " \"num_updated\": 0,\n", + " \"update_list\": []\n", + " }\n", + "}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 780 100 780 0 0 780 0 0:00:01 --:--:-- 0:00:01 1115\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# get the data for the library created above\n", + "# note: add | python -m json.tool to the end of the request to print the response in a more readable format\n", + "curl -H \"Authorization: Bearer $token\" https://api.adsabs.harvard.edu/v1/biblib/libraries/hHGU1Ef-TpacAhicI3J8kQ | python -m json.tool" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### /documents/<library_id>\n", + "Interact with a specific library. The `library_id` (string) is the unique identifier of the library, as specified in the `/libraries` GET response.\n", + "\n", + "Functions allowed: add records to a library (POST), delete records from a library (POST), update the metadata of a library (PUT), delete the entire library (DELETE)\n", + "\n", + "#### POST\n", + "Add or delete a document to a given library. Returns the number of documents added or removed\n", + "\n", + "Specify that the request is a POST method by adding `-X POST` to your request. You must include a payload, passed via the `-d` flag, in JSON format. Add this header to your request to indicate your payload is in the appropriate format: `-H \"Content-Type: application/json\"`. The payload must include:\n", + " * bibcode: (list) list of bibcodes to be added or removed\n", + " * action: (string) 'add' or 'remove' to add or remove the given bibcodes from the specified library\n", + "\n", + "*Return data*:\n", + " * number_added: (integer) number of documents added (if 'add' is used)\n", + " * number_removed: (integer) number of documents removed (if 'remove' is used)\n", + "\n", + "*Permissions*:\n", + "The following type of user can add or remove documents:\n", + " * owner\n", + " * admin\n", + " * write" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"number_added\": 1}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 73 100 20 100 53 20 53 0:00:01 --:--:-- 0:00:01 244\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# add a bibcode\n", + "curl -H \"Authorization: Bearer $token\" -H \"Content-Type: application/json\" \\\n", + " https://api.adsabs.harvard.edu/v1/biblib/documents/hHGU1Ef-TpacAhicI3J8kQ \\\n", + " -X POST \\\n", + " -d '{\"bibcode\": [\"1974Natur.248...30H\"], \"action\": \"add\"}'" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"number_removed\": 1}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 78 100 22 100 56 22 56 0:00:01 --:--:-- 0:00:01 295\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# remove it again\n", + "curl -H \"Authorization: Bearer $token\" -H \"Content-Type: application/json\" \\\n", + " https://api.adsabs.harvard.edu/v1/biblib/documents/hHGU1Ef-TpacAhicI3J8kQ \\\n", + " -X POST \\\n", + " -d '{\"bibcode\": [\"1974Natur.248...30H\"], \"action\": \"remove\"}'" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### PUT\n", + "Update the metadata of a given library. Returns updated metadata of the library\n", + "\n", + "As with the POST method, you must specify that the request is a PUT method by adding `-X PUT` to your request. You must also include a payload, passed via the `-d` flag, in JSON format. Add this header to your request to indicate your payload is in the appropriate format: `-H \"Content-Type: application/json\"`. The payload can include:\n", + "\n", + "*Payload*:\n", + " * name: (string) name of the library\n", + " * description: (string) description of the library\n", + " * public: (boolean) true if the library should be publicly viewable, false if it should be private (note that this is case sensitive and should be lower case)\n", + "\n", + "*Note: The above are optional. Leave fields out if they do not need to be updated.*\n", + "\n", + "*Return data*:\n", + " * name: (string) name of the library\n", + " * description: (string) description of the library\n", + " * public:(boolean) true if the library is publicly viewable, false if it is private\n", + "\n", + "*Note: returns the key/value that was requested to be updated*\n", + "\n", + "*Permissions*:\n", + "The following type of user can update the 'name', 'library', and 'public' attributes:\n", + " - owner\n", + " - admin" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"description\": \"New API library description\", \"name\": \"New API library name\", \"public\": false}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 191 100 96 100 95 96 95 0:00:01 --:--:-- 0:00:01 313\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# change the metadata of a library\n", + "curl -H \"Authorization: Bearer $token\" -H \"Content-Type: application/json\" \\\n", + " https://api.adsabs.harvard.edu/v1/biblib/documents/hHGU1Ef-TpacAhicI3J8kQ \\\n", + " -X PUT \\\n", + " -d '{\"name\": \"New API library name\", \"description\": \"New API library description\", \"public\": false}'" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##### DELETE\n", + "Delete the given library\n", + "\n", + "Make sure to specify the DELETE method using `-X DELETE`. No payload is required, as the library ID is passed in the URL. There is no return response to the request.\n", + "\n", + "*Permissions*: The following type of user can update a library:\n", + " - owner" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"id\": \"GPdVMFKwTmC87ysCk3JK3g\", \"description\": \"My ADS library\", \"name\": \"temp library\"}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 113 100 90 100 23 90 23 0:00:01 --:--:-- 0:00:01 379\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# create a temporary library\n", + "curl -H \"Authorization: Bearer $token\" -H \"Content-Type: application/json\" \\\n", + " https://api.adsabs.harvard.edu/v1/biblib/libraries \\\n", + " -X POST \\\n", + " -d '{\"name\":\"temp library\"}'\n" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 3 100 3 0 0 3 0 0:00:01 --:--:-- 0:00:01 10\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# delete the temporary library\n", + "curl -H \"Authorization: Bearer $token\" -H \"Content-Type: application/json\" \\\n", + " https://api.adsabs.harvard.edu/v1/biblib/documents/GPdVMFKwTmC87ysCk3JK3g \\\n", + " -X DELETE" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### /permissions/<library_id>\n", + "Edit the permissions of a library. The `library_id` (string) is the unique identifier of the library, as specified in the `/libraries` GET response.\n", + "\n", + "#### GET\n", + "Get the permissions for a given library.\n", + "\n", + "The response is a list of dictionaries, where each dictionary is for a specific user. The dictionary contains a list of all the permissions of the user:\n", + " \t* user dictionary (dict) contains:\n", + " \t\t* key (string): user e-mail\n", + " \t\t* item (list): list of permissions ('read', 'write', 'admin', 'owner')\n", + "\n", + "*Permissions*:\n", + "The following type of user can access permissions of a library:\n", + " - owner\n", + " - admin" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "scrolled": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[{\"test.user-api@example.email.com\": [\"owner\"]}]\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 52 100 52 0 0 52 0 0:00:01 --:--:-- 0:00:01 201\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# get the library permissions\n", + "curl -H \"Authorization: Bearer $token\" \\\n", + " https://api.adsabs.harvard.edu/v1/biblib/permissions/hHGU1Ef-TpacAhicI3J8kQ\n", + " " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### POST\n", + "Edit the permissions of a library.\n", + "\n", + "Specify that the request is a POST method by adding `-X POST` to your request. You must include a payload, passed via the `-d` flag, in JSON format. Add this header to your request to indicate your payload is in the appropriate format: `-H \"Content-Type: application/json\"`. The payload must include:\n", + " * email: (string) specifies which user's permissions to be modified\n", + " * permission: (string) 'read', 'write', 'admin' or 'owner'\n", + " * value: (boolean) whether the user has this permission (true=yes, false=no)\n", + "\n", + "There is no response to the request.\n", + "\n", + "*Permissions*:\n", + "The following type of user can update a permission for a user for a given library:\n", + " - owner\n", + " - admin" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 81 100 3 100 78 3 78 0:00:01 --:--:-- 0:00:01 465\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# edit the library permissions\n", + "curl -H \"Authorization: Bearer $token\" -H \"Content-Type: application/json\" \\\n", + " https://api.adsabs.harvard.edu/v1/biblib/permissions/hHGU1Ef-TpacAhicI3J8kQ \\\n", + " -X POST \\\n", + " -d '{\"email\":\"test.user@example.email.com\", \"permission\":\"read\", \"value\": true}'" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### /transfer/<library_id>\n", + "Transfer a the ownership of a library to another ADS user. The `library_id` (string) is the unique identifier of the library, as specified in the `/libraries` GET response.\n", + "\n", + "#### POST\n", + "Transfer the ownership of a library from one user to another. \n", + "\n", + "Specify that the request is a POST method by adding `-X POST` to your request. You must include a payload, passed via the `-d` flag, in JSON format. Add this header to your request to indicate your payload is in the appropriate format: `-H \"Content-Type: application/json\"`. The payload must include:\n", + " * email: (string) email of the user the account should be transfered to\n", + "\n", + "There is no response to the request.\n", + "\n", + "*Permissions*:\n", + "The following type of user can transfer libraries:\n", + " - owner" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 46 100 3 100 43 3 43 0:00:01 --:--:-- 0:00:01 133\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# transfer the library ownership\n", + "curl -H \"Authorization: Bearer $token\" -H \"Content-Type: application/json\" \\\n", + " https://api.adsabs.harvard.edu/v1/biblib/transfer/hHGU1Ef-TpacAhicI3J8kQ \\\n", + " -X POST \\\n", + " -d '{\"email\": \"test.user@example.email.com\"}'" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 2", + "language": "python", + "name": "python2" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 2 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython2", + "version": "2.7.14" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/Metrics_API.ipynb b/Metrics_API.ipynb new file mode 100644 index 0000000..750a818 --- /dev/null +++ b/Metrics_API.ipynb @@ -0,0 +1,790 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Metrics Search Requests\n", + "This notebook explains how to retrieve citation data and other metrics for a given bibcode(s).\n", + "\n", + "The base URL for metrics queries is `https://api.adsabs.harvard.edu/v1/metrics`. \n", + "\n", + "All queries to the API must include an HTTP header that specifies your API access token, e.g.\n", + "\n", + "`curl -H 'Authorization: Bearer ' ...`\n", + "\n", + "In the examples below, {token_header} is used to denote `-H 'Authorization: Bearer '`, where `` is your personal API access token as described in the top-level [README](https://github.com/adsabs/adsabs-dev-api/blob/master/README.md).\n", + "\n", + "Note: the API examples in this notebook generally assume you're using `curl` statements from the shell command line--you'll see `%%bash` at the top of a cell if its contents should be run in the shell instead of in Python. Guidelines for translating bash `curl` statements into Python can be found in the [converting curl to python notebook](Converting_curl_to_python.ipynb). If you prefer, you can use the unofficial Python package instead; see the top-level [README](https://github.com/adsabs/adsabs-dev-api/blob/master/README.md) for more information." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Sending a metrics request\n", + "All metrics requests use the default HTTP method POST. POST requests require a payload to be specified using the `-d` flag. For metrics, this payload will be JSON formatted, with the key `bibcodes` and a value set to a comma-separated list of the bibcodes to retrieve metrics for. For example:\n", + "\n", + "`curl {token_header} https://api.adsabs.harvard.edu/v1/metrics -X POST -d '{\"bibcodes\": [\"bibcode1\",\"bibcode2\"]}'`\n", + "\n", + "You can request a specific set of metrics by adding the `types` parameter in the JSON payload. For example:\n", + "\n", + "`curl {token_header} -H \"Content-Type: application/json\" https://api.adsabs.harvard.edu/v1/metrics -X POST -d '{\"bibcodes\": [\"bibcode1\",\"bibcode2\"], \"types\":[\"basic\"]}'`\n", + "\n", + "This will return just the basic statistics for a set of bibcodes. The following values are allowed for the `types` list:\n", + "\n", + " * 'basic': publication and usage stats (all papers, and just refereed papers)\n", + " * 'citations': citation stats\n", + " * 'indicators': indicators, like the h-index, g-index, m-index etc etc\n", + " * 'histograms': publication, citation, reads and downloads histograms\n", + " * 'timeseries': time series for a set of indicators\n", + " \n", + "To only return a specific histogram, add a `histograms` parameter to the JSON payload, specifying that histogram. For example:\n", + "\n", + "`curl {token_header} -H \"Content-Type: application/json\" https://api.adsabs.harvard.edu/v1/metrics -X POST -d '{\"bibcodes\":[\"bibcode1\",\"bibcode2\"], \"types\":[\"histograms\"], \"histograms\":[\"publications\"]}'`\n", + "\n", + "to get just the publications histogram. " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Response from a metrics request\n", + "The service returns JSON with attributes determined by the contents of the `types` list (no such list means that everything gets returned). It always contains an attribute `skipped bibcodes`, which is a list of bibcodes for which no metrics data could be found. The following JSON attributes are returned by each type:\n", + "\n", + " * 'basic': 'basic stats' and 'basic stats refereed'\n", + " * 'citations': 'citation stats' and 'citation stats refereed'\n", + " * 'indicators': 'indicators' and 'indicators refereed'\n", + " * 'histograms': 'histograms'\n", + " * 'timeseries': 'time series'\n", + "\n", + "Each of these attributes contains specific data elements. The following sections shows what each type contains (each bullet point is an attribute name, or JSON key).\n", + "\n", + "#### 'basic stats' and 'basic stats refereed'\n", + " * number of papers\n", + " * normalized paper count\n", + " * median number of reads\n", + " * average number of reads\n", + " * total number of reads\n", + " * normalized number of reads\n", + " * median number of downloads\n", + " * average number of downloads\n", + " * total number of downloads\n", + " * normalized number of downloads\n", + " \n", + "#### 'citation stats' and 'citation stats refereed'\n", + " * number of citing papers\n", + " * median number of citations\n", + " * average number of citations\n", + " * total number of citations\n", + " * normalized number of citations\n", + " * median number of refereed citations\n", + " * average number of refereed citations\n", + " * total number of refereed citations\n", + " * normalized number of refereed citations\n", + " * number of self-citations\n", + "\n", + "#### 'indicators'\n", + " * i10\n", + " * i100\n", + " * h\n", + " * m\n", + " * g\n", + " * read10\n", + " * tori\n", + "\n", + "#### 'histograms'\n", + "For all histograms, the actual histogram values are stored as follows:\n", + "```\n", + " {\n", + "\t \"histogram name\": {year: value, year: value, ...}\n", + "\t}\n", + "```\n", + "where values can be either integers or floats, depending on the type of histograms. The histogram name is an attribute of the histogram type. For example, the `publications` type has 4 separate histograms: `all publications`, `refereed publications`, `all publications normalized` and `refereed publications normalized`, which are organized as follows in the JSON output:\n", + "```\n", + " {\n", + "\t \"histograms\": {\n", + "\t\t\t\"publications\": {\n", + "\t\t\t\t\"all publications\": {},\n", + "\t\t\t\t\"refereed publications\": {},\n", + "\t\t\t\t\"all publications normalized\": {},\n", + "\t\t\t\t\"refereed publications normalized\": {},\n", + "\t\t\t\n", + "\t\t\t}\n", + "\t\t}\n", + "\t}\n", + "```\n", + "The following lists all histogram types and the names of the actual histograms stored in them:\n", + "\n", + " * 'publications': 'all publications', 'refereed publications', 'all publications normalized', 'refereed publications normalized'\n", + " * 'reads': 'all reads', 'refereed reads', 'all reads normalized', 'refereed reads normalized'\n", + " * 'downloads': 'all downloads', 'refereed downloads', 'all downloads normalized', 'refereed downloads normalized'\n", + " * 'citations': 'refereed to refereed', 'nonrefereed to refereed', 'refereed to nonrefereed', 'nonrefereed to nonrefereed', 'refereed to refereed normalized', 'nonrefereed to refereed normalized', 'refereed to nonrefereed normalized', 'nonrefereed to nonrefereed normalized'\n", + "\n", + "Note that 'refereed reads' means: 'number of reads for the refereed publications in the set'.\n", + "\n", + "#### 'timeseries'\n", + "The `timeseries` attribute contains time series data for: `h`, `g`, `i10`, `i100`, `read10` and `tori`. The output is structured as\n", + "```\n", + " {\n", + "\t\t\"time series\": {\n", + "\t\t\t\"h\": { values },\n", + "\t\t\t\"g\": { values },\n", + "\t\t\t\"i10\": { values },\n", + "\t\t\t\"i100\": { values },\n", + "\t\t\t\"read10\": { values },\n", + "\t\t\t\"tori\": { values },\n", + "\t\t\n", + "\t\t}\n", + "\t}\n", + "```\n", + "where the 'values' are organized in the same way as for the histograms: \n", + "```\n", + " {\n", + "\t \"indicator name\": {year: value, year: value, ...}\n", + "\t}\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Examples" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\n", + " \"basic stats\": {\n", + " \"average number of downloads\": 8134.0,\n", + " \"average number of reads\": 19644.0,\n", + " \"median number of downloads\": 8134.0,\n", + " \"median number of reads\": 19644.0,\n", + " \"normalized paper count\": 0.058823529411764705,\n", + " \"number of papers\": 1,\n", + " \"recent number of downloads\": 144,\n", + " \"recent number of reads\": 446,\n", + " \"total number of downloads\": 8134,\n", + " \"total number of reads\": 19644\n", + " },\n", + " \"basic stats refereed\": {\n", + " \"average number of downloads\": 8134.0,\n", + " \"average number of reads\": 19644.0,\n", + " \"median number of downloads\": 8134.0,\n", + " \"median number of reads\": 19644.0,\n", + " \"normalized paper count\": 0.058823529411764705,\n", + " \"number of papers\": 1,\n", + " \"recent number of downloads\": 144,\n", + " \"recent number of reads\": 446,\n", + " \"total number of downloads\": 8134,\n", + " \"total number of reads\": 19644\n", + " },\n", + " \"citation stats\": {\n", + " \"average number of citations\": 8730.0,\n", + " \"average number of refereed citations\": 7501.0,\n", + " \"median number of citations\": 8730.0,\n", + " \"median number of refereed citations\": 7501.0,\n", + " \"normalized number of citations\": 513.5294117647059,\n", + " \"normalized number of refereed citations\": 441.2352941176471,\n", + " \"number of citing papers\": 8730,\n", + " \"number of self-citations\": 0,\n", + " \"self-citations\": [],\n", + " \"total number of citations\": 8730,\n", + " \"total number of refereed citations\": 7501\n", + " },\n", + " \"citation stats refereed\": {\n", + " \"average number of citations\": 8730.0,\n", + " \"average number of refereed citations\": 7501.0,\n", + " \"median number of citations\": 8730.0,\n", + " \"median number of refereed citations\": 7501.0,\n", + " \"normalized number of citations\": 513.5294117647059,\n", + " \"normalized number of refereed citations\": 441.2352941176471,\n", + " \"number of citing papers\": 8730,\n", + " \"number of self-citations\": 0,\n", + " \"total number of citations\": 8730,\n", + " \"total number of refereed citations\": 7501\n", + " },\n", + " \"histograms\": {\n", + " \"citations\": {\n", + " \"nonrefereed to nonrefereed\": {\n", + " \"2001\": 0,\n", + " \"2002\": 0,\n", + " \"2003\": 0,\n", + " \"2004\": 0,\n", + " \"2005\": 0,\n", + " \"2006\": 0,\n", + " \"2007\": 0,\n", + " \"2008\": 0,\n", + " \"2009\": 0,\n", + " \"2010\": 0,\n", + " \"2011\": 0,\n", + " \"2012\": 0,\n", + " \"2013\": 0,\n", + " \"2014\": 0,\n", + " \"2015\": 0,\n", + " \"2016\": 0,\n", + " \"2017\": 0,\n", + " \"2018\": 0\n", + " },\n", + " \"nonrefereed to nonrefereed normalized\": {\n", + " \"2001\": 0,\n", + " \"2002\": 0,\n", + " \"2003\": 0,\n", + " \"2004\": 0,\n", + " \"2005\": 0,\n", + " \"2006\": 0,\n", + " \"2007\": 0,\n", + " \"2008\": 0,\n", + " \"2009\": 0,\n", + " \"2010\": 0,\n", + " \"2011\": 0,\n", + " \"2012\": 0,\n", + " \"2013\": 0,\n", + " \"2014\": 0,\n", + " \"2015\": 0,\n", + " \"2016\": 0,\n", + " \"2017\": 0,\n", + " \"2018\": 0\n", + " },\n", + " \"nonrefereed to refereed\": {\n", + " \"2001\": 0,\n", + " \"2002\": 2,\n", + " \"2003\": 95,\n", + " \"2004\": 148,\n", + " \"2005\": 237,\n", + " \"2006\": 152,\n", + " \"2007\": 101,\n", + " \"2008\": 75,\n", + " \"2009\": 70,\n", + " \"2010\": 52,\n", + " \"2011\": 35,\n", + " \"2012\": 47,\n", + " \"2013\": 36,\n", + " \"2014\": 30,\n", + " \"2015\": 34,\n", + " \"2016\": 35,\n", + " \"2017\": 39,\n", + " \"2018\": 41\n", + " },\n", + " \"nonrefereed to refereed normalized\": {\n", + " \"2001\": 0,\n", + " \"2002\": 0.11764705882352941,\n", + " \"2003\": 5.588235294117634,\n", + " \"2004\": 8.705882352941153,\n", + " \"2005\": 13.941176470588193,\n", + " \"2006\": 8.94117647058821,\n", + " \"2007\": 5.941176470588221,\n", + " \"2008\": 4.411764705882344,\n", + " \"2009\": 4.117647058823522,\n", + " \"2010\": 3.058823529411761,\n", + " \"2011\": 2.0588235294117645,\n", + " \"2012\": 2.7647058823529385,\n", + " \"2013\": 2.117647058823529,\n", + " \"2014\": 1.7647058823529413,\n", + " \"2015\": 2.0,\n", + " \"2016\": 2.0588235294117645,\n", + " \"2017\": 2.2941176470588225,\n", + " \"2018\": 2.4117647058823515\n", + " },\n", + " \"refereed to nonrefereed\": {\n", + " \"2001\": 0,\n", + " \"2002\": 0,\n", + " \"2003\": 0,\n", + " \"2004\": 0,\n", + " \"2005\": 0,\n", + " \"2006\": 0,\n", + " \"2007\": 0,\n", + " \"2008\": 0,\n", + " \"2009\": 0,\n", + " \"2010\": 0,\n", + " \"2011\": 0,\n", + " \"2012\": 0,\n", + " \"2013\": 0,\n", + " \"2014\": 0,\n", + " \"2015\": 0,\n", + " \"2016\": 0,\n", + " \"2017\": 0,\n", + " \"2018\": 0\n", + " },\n", + " \"refereed to nonrefereed normalized\": {\n", + " \"2001\": 0,\n", + " \"2002\": 0,\n", + " \"2003\": 0,\n", + " \"2004\": 0,\n", + " \"2005\": 0,\n", + " \"2006\": 0,\n", + " \"2007\": 0,\n", + " \"2008\": 0,\n", + " \"2009\": 0,\n", + " \"2010\": 0,\n", + " \"2011\": 0,\n", + " \"2012\": 0,\n", + " \"2013\": 0,\n", + " \"2014\": 0,\n", + " \"2015\": 0,\n", + " \"2016\": 0,\n", + " \"2017\": 0,\n", + " \"2018\": 0\n", + " },\n", + " \"refereed to refereed\": {\n", + " \"2001\": 1,\n", + " \"2002\": 0,\n", + " \"2003\": 379,\n", + " \"2004\": 992,\n", + " \"2005\": 1026,\n", + " \"2006\": 997,\n", + " \"2007\": 667,\n", + " \"2008\": 533,\n", + " \"2009\": 462,\n", + " \"2010\": 411,\n", + " \"2011\": 362,\n", + " \"2012\": 342,\n", + " \"2013\": 317,\n", + " \"2014\": 282,\n", + " \"2015\": 250,\n", + " \"2016\": 204,\n", + " \"2017\": 187,\n", + " \"2018\": 89\n", + " },\n", + " \"refereed to refereed normalized\": {\n", + " \"2001\": 0.058823529411764705,\n", + " \"2002\": 0,\n", + " \"2003\": 22.29411764705875,\n", + " \"2004\": 58.35294117647198,\n", + " \"2005\": 60.35294117647209,\n", + " \"2006\": 58.64705882353082,\n", + " \"2007\": 39.23529411764736,\n", + " \"2008\": 31.352941176470484,\n", + " \"2009\": 27.176470588235205,\n", + " \"2010\": 24.176470588235215,\n", + " \"2011\": 21.294117647058755,\n", + " \"2012\": 20.117647058823465,\n", + " \"2013\": 18.647058823529353,\n", + " \"2014\": 16.588235294117595,\n", + " \"2015\": 14.705882352941131,\n", + " \"2016\": 11.999999999999964,\n", + " \"2017\": 10.999999999999968,\n", + " \"2018\": 5.235294117647047\n", + " }\n", + " },\n", + " \"reads\": {\n", + " \"all reads\": {\n", + " \"1996\": 0,\n", + " \"1997\": 0,\n", + " \"1998\": 0,\n", + " \"1999\": 0,\n", + " \"2000\": 0,\n", + " \"2001\": 0,\n", + " \"2002\": 0,\n", + " \"2003\": 839,\n", + " \"2004\": 1999,\n", + " \"2005\": 1838,\n", + " \"2006\": 1909,\n", + " \"2007\": 1719,\n", + " \"2008\": 1405,\n", + " \"2009\": 1243,\n", + " \"2010\": 1355,\n", + " \"2011\": 1368,\n", + " \"2012\": 1348,\n", + " \"2013\": 929,\n", + " \"2014\": 919,\n", + " \"2015\": 743,\n", + " \"2016\": 797,\n", + " \"2017\": 787,\n", + " \"2018\": 446\n", + " },\n", + " \"all reads normalized\": {\n", + " \"1996\": 0.0,\n", + " \"1997\": 0.0,\n", + " \"1998\": 0.0,\n", + " \"1999\": 0.0,\n", + " \"2000\": 0.0,\n", + " \"2001\": 0.0,\n", + " \"2002\": 0.0,\n", + " \"2003\": 49.35294117647059,\n", + " \"2004\": 117.58823529411765,\n", + " \"2005\": 108.11764705882354,\n", + " \"2006\": 112.29411764705883,\n", + " \"2007\": 101.11764705882354,\n", + " \"2008\": 82.6470588235294,\n", + " \"2009\": 73.11764705882354,\n", + " \"2010\": 79.70588235294117,\n", + " \"2011\": 80.47058823529412,\n", + " \"2012\": 79.29411764705883,\n", + " \"2013\": 54.64705882352941,\n", + " \"2014\": 54.05882352941177,\n", + " \"2015\": 43.705882352941174,\n", + " \"2016\": 46.88235294117647,\n", + " \"2017\": 46.294117647058826,\n", + " \"2018\": 26.235294117647058\n", + " },\n", + " \"refereed reads\": {\n", + " \"1996\": 0,\n", + " \"1997\": 0,\n", + " \"1998\": 0,\n", + " \"1999\": 0,\n", + " \"2000\": 0,\n", + " \"2001\": 0,\n", + " \"2002\": 0,\n", + " \"2003\": 839,\n", + " \"2004\": 1999,\n", + " \"2005\": 1838,\n", + " \"2006\": 1909,\n", + " \"2007\": 1719,\n", + " \"2008\": 1405,\n", + " \"2009\": 1243,\n", + " \"2010\": 1355,\n", + " \"2011\": 1368,\n", + " \"2012\": 1348,\n", + " \"2013\": 929,\n", + " \"2014\": 919,\n", + " \"2015\": 743,\n", + " \"2016\": 797,\n", + " \"2017\": 787,\n", + " \"2018\": 446\n", + " },\n", + " \"refereed reads normalized\": {\n", + " \"1996\": 0.0,\n", + " \"1997\": 0.0,\n", + " \"1998\": 0.0,\n", + " \"1999\": 0.0,\n", + " \"2000\": 0.0,\n", + " \"2001\": 0.0,\n", + " \"2002\": 0.0,\n", + " \"2003\": 49.35294117647059,\n", + " \"2004\": 117.58823529411765,\n", + " \"2005\": 108.11764705882354,\n", + " \"2006\": 112.29411764705883,\n", + " \"2007\": 101.11764705882354,\n", + " \"2008\": 82.6470588235294,\n", + " \"2009\": 73.11764705882354,\n", + " \"2010\": 79.70588235294117,\n", + " \"2011\": 80.47058823529412,\n", + " \"2012\": 79.29411764705883,\n", + " \"2013\": 54.64705882352941,\n", + " \"2014\": 54.05882352941177,\n", + " \"2015\": 43.705882352941174,\n", + " \"2016\": 46.88235294117647,\n", + " \"2017\": 46.294117647058826,\n", + " \"2018\": 26.235294117647058\n", + " }\n", + " }\n", + " },\n", + " \"skipped bibcodes\": []\n", + "}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 6007 100 5970 100 37 5970 37 0:00:01 --:--:-- 0:00:01 24518\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# get all metrics for a single bibcode\n", + "curl -H \"Authorization: Bearer $token\" -H \"Content-Type: application/json\" \\\n", + " https://api.adsabs.harvard.edu/v1/metrics -X POST -d '{\"bibcodes\": [\"2003ApJS..148..175S\"]}' | python -m json.tool " + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\n", + " \"basic stats\": {\n", + " \"average number of downloads\": 7555.0,\n", + " \"average number of reads\": 18185.5,\n", + " \"median number of downloads\": 7555.0,\n", + " \"median number of reads\": 18185.5,\n", + " \"normalized paper count\": 0.10427807486631016,\n", + " \"number of papers\": 2,\n", + " \"recent number of downloads\": 240,\n", + " \"recent number of reads\": 759,\n", + " \"total number of downloads\": 15110,\n", + " \"total number of reads\": 36371\n", + " },\n", + " \"basic stats refereed\": {\n", + " \"average number of downloads\": 7555.0,\n", + " \"average number of reads\": 18185.5,\n", + " \"median number of downloads\": 7555.0,\n", + " \"median number of reads\": 18185.5,\n", + " \"normalized paper count\": 0.10427807486631016,\n", + " \"number of papers\": 2,\n", + " \"recent number of downloads\": 240,\n", + " \"recent number of reads\": 759,\n", + " \"total number of downloads\": 15110,\n", + " \"total number of reads\": 36371\n", + " },\n", + " \"skipped bibcodes\": []\n", + "}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 851 100 772 100 79 772 79 0:00:01 --:--:-- 0:00:01 3976\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# get basic metrics for multiple bibcodes\n", + "curl -H \"Authorization: Bearer $token\" -H \"Content-Type: application/json\" \\\n", + " https://api.adsabs.harvard.edu/v1/metrics \\\n", + " -X POST -d '{\"bibcodes\": [\"2003ApJS..148..175S\", \"2007ApJS..170..377S\"], \"types\":[\"basic\"]}' | python -m json.tool" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\n", + " \"histograms\": {\n", + " \"citations\": {\n", + " \"nonrefereed to nonrefereed\": {\n", + " \"2001\": 0,\n", + " \"2002\": 0,\n", + " \"2003\": 0,\n", + " \"2004\": 0,\n", + " \"2005\": 0,\n", + " \"2006\": 0,\n", + " \"2007\": 0,\n", + " \"2008\": 0,\n", + " \"2009\": 0,\n", + " \"2010\": 0,\n", + " \"2011\": 0,\n", + " \"2012\": 0,\n", + " \"2013\": 0,\n", + " \"2014\": 0,\n", + " \"2015\": 0,\n", + " \"2016\": 0,\n", + " \"2017\": 0,\n", + " \"2018\": 0\n", + " },\n", + " \"nonrefereed to nonrefereed normalized\": {\n", + " \"2001\": 0,\n", + " \"2002\": 0,\n", + " \"2003\": 0,\n", + " \"2004\": 0,\n", + " \"2005\": 0,\n", + " \"2006\": 0,\n", + " \"2007\": 0,\n", + " \"2008\": 0,\n", + " \"2009\": 0,\n", + " \"2010\": 0,\n", + " \"2011\": 0,\n", + " \"2012\": 0,\n", + " \"2013\": 0,\n", + " \"2014\": 0,\n", + " \"2015\": 0,\n", + " \"2016\": 0,\n", + " \"2017\": 0,\n", + " \"2018\": 0\n", + " },\n", + " \"nonrefereed to refereed\": {\n", + " \"2001\": 0,\n", + " \"2002\": 2,\n", + " \"2003\": 95,\n", + " \"2004\": 148,\n", + " \"2005\": 238,\n", + " \"2006\": 273,\n", + " \"2007\": 294,\n", + " \"2008\": 239,\n", + " \"2009\": 184,\n", + " \"2010\": 119,\n", + " \"2011\": 85,\n", + " \"2012\": 83,\n", + " \"2013\": 66,\n", + " \"2014\": 53,\n", + " \"2015\": 57,\n", + " \"2016\": 56,\n", + " \"2017\": 74,\n", + " \"2018\": 73\n", + " },\n", + " \"nonrefereed to refereed normalized\": {\n", + " \"2001\": 0,\n", + " \"2002\": 0.11764705882352941,\n", + " \"2003\": 5.588235294117634,\n", + " \"2004\": 8.705882352941153,\n", + " \"2005\": 13.986631016042738,\n", + " \"2006\": 14.441176470588152,\n", + " \"2007\": 14.713903743315441,\n", + " \"2008\": 11.866310160427789,\n", + " \"2009\": 9.299465240641725,\n", + " \"2010\": 6.104278074866325,\n", + " \"2011\": 4.331550802139039,\n", + " \"2012\": 4.401069518716577,\n", + " \"2013\": 3.4812834224598914,\n", + " \"2014\": 2.810160427807486,\n", + " \"2015\": 3.0454545454545445,\n", + " \"2016\": 3.013368983957218,\n", + " \"2017\": 3.885026737967912,\n", + " \"2018\": 3.8663101604278047\n", + " },\n", + " \"refereed to nonrefereed\": {\n", + " \"2001\": 0,\n", + " \"2002\": 0,\n", + " \"2003\": 0,\n", + " \"2004\": 0,\n", + " \"2005\": 0,\n", + " \"2006\": 0,\n", + " \"2007\": 0,\n", + " \"2008\": 0,\n", + " \"2009\": 0,\n", + " \"2010\": 0,\n", + " \"2011\": 0,\n", + " \"2012\": 0,\n", + " \"2013\": 0,\n", + " \"2014\": 0,\n", + " \"2015\": 0,\n", + " \"2016\": 0,\n", + " \"2017\": 0,\n", + " \"2018\": 0\n", + " },\n", + " \"refereed to nonrefereed normalized\": {\n", + " \"2001\": 0,\n", + " \"2002\": 0,\n", + " \"2003\": 0,\n", + " \"2004\": 0,\n", + " \"2005\": 0,\n", + " \"2006\": 0,\n", + " \"2007\": 0,\n", + " \"2008\": 0,\n", + " \"2009\": 0,\n", + " \"2010\": 0,\n", + " \"2011\": 0,\n", + " \"2012\": 0,\n", + " \"2013\": 0,\n", + " \"2014\": 0,\n", + " \"2015\": 0,\n", + " \"2016\": 0,\n", + " \"2017\": 0,\n", + " \"2018\": 0\n", + " },\n", + " \"refereed to refereed\": {\n", + " \"2001\": 1,\n", + " \"2002\": 0,\n", + " \"2003\": 379,\n", + " \"2004\": 992,\n", + " \"2005\": 1026,\n", + " \"2006\": 1438,\n", + " \"2007\": 1736,\n", + " \"2008\": 1674,\n", + " \"2009\": 1120,\n", + " \"2010\": 925,\n", + " \"2011\": 777,\n", + " \"2012\": 651,\n", + " \"2013\": 592,\n", + " \"2014\": 485,\n", + " \"2015\": 423,\n", + " \"2016\": 340,\n", + " \"2017\": 318,\n", + " \"2018\": 151\n", + " },\n", + " \"refereed to refereed normalized\": {\n", + " \"2001\": 0.058823529411764705,\n", + " \"2002\": 0,\n", + " \"2003\": 22.29411764705875,\n", + " \"2004\": 58.35294117647198,\n", + " \"2005\": 60.35294117647209,\n", + " \"2006\": 78.69251336898593,\n", + " \"2007\": 87.82620320855783,\n", + " \"2008\": 83.21657754010832,\n", + " \"2009\": 57.085561497326964,\n", + " \"2010\": 47.54010695187224,\n", + " \"2011\": 40.157754010695655,\n", + " \"2012\": 34.16310160427841,\n", + " \"2013\": 31.147058823529708,\n", + " \"2014\": 25.815508021390585,\n", + " \"2015\": 22.56951871657767,\n", + " \"2016\": 18.181818181818166,\n", + " \"2017\": 16.954545454545396,\n", + " \"2018\": 8.053475935828889\n", + " }\n", + " }\n", + " },\n", + " \"skipped bibcodes\": []\n", + "}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 112 0 0 100 112 0 112 0:00:01 --:--:-- 0:00:01 235\r", + "100 2650 100 2538 100 112 2538 112 0:00:01 --:--:-- 0:00:01 5567\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# get data for a single histogram for multiple bibcodes\n", + "curl -H \"Authorization: Bearer $token\" -H \"Content-Type: application/json\" \\\n", + " https://api.adsabs.harvard.edu/v1/metrics \\\n", + " -X POST \\\n", + " -d '{\"bibcodes\": [\"2003ApJS..148..175S\", \"2007ApJS..170..377S\"], \"types\":[\"histograms\"], \"histograms\":[\"citations\"]}' \\\n", + " | python -m json.tool" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 2", + "language": "python", + "name": "python2" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 2 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython2", + "version": "2.7.14" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/README.md b/README.md index d545643..b6a28a5 100644 --- a/README.md +++ b/README.md @@ -4,14 +4,19 @@ ADS Developer API description. For bugs, feature requests or even random questions feel free to use the [issues](https://github.com/adsabs/adsabs-dev-api/issues) section. -**Table of Contents** +**Table of Contents** + +The help pages and examples below are formatted as Jupyter notebooks. Browse them in Github or download them to your own computer to edit the code and run as desired. Note that the code in these notebooks was run using an API key or token. This has been edited out of the notebooks for security; [acquire your own API key](#access) and copy it into the notebook before running it locally. + +Note: sometimes Github's internal Jupyter notebook rendering agent fails. If that happens, copy the link to the notebook you wish to view into the form at [Jupyter's own notebook renderer](https://nbviewer.jupyter.org/). - [Access](#access) - [Access Settings](#access-settings) -- [Search API](search.md) -- [Metrics API](metrics.md) -- [Export API](export.md) -- [Libraries API](libraries.md) +- [Search API](Search_API.ipynb) +- [Metrics API](Metrics_API.ipynb) +- [Export API](Export_API.ipynb) +- [Libraries API](Libraries_API.ipynb) +- [Converting curl to Python - examples](Converting_curl_to_python.ipynb) ## Mailing List diff --git a/Search_API.ipynb b/Search_API.ipynb new file mode 100644 index 0000000..b3dd01a --- /dev/null +++ b/Search_API.ipynb @@ -0,0 +1,613 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Search API\n", + "This notebook explains how to use the API to search the database using either search terms or a set of bibcodes. Lists of query parameters and search fields are also given.\n", + "\n", + "The base URL (denoted as {base_url} in the examples below) for queries is `https://api.adsabs.harvard.edu/v1/search`. \n", + "\n", + "All queries to the API must include an HTTP header that specifies your API access token, e.g.\n", + "\n", + "`curl -H 'Authorization: Bearer ' 'https://api.adsabs.harvard.edu/v1/search/query?q=star'`\n", + "\n", + "In the examples below, {token_header} is used to denote `-H 'Authorization: Bearer '`, where `` is your personal API access token as described in the top-level [README](https://github.com/adsabs/adsabs-dev-api/blob/master/README.md).\n", + "\n", + "Notes:\n", + "* All examples use the default HTTP method GET, unless otherwise specified.\n", + "* the API examples in this notebook generally assume you're using `curl` statements from the shell command line--you'll see `%%bash` at the top of a cell if its contents should be run in the shell instead of in Python. Guidelines for translating bash `curl` statements into Python can be found in the [converting curl to python notebook](Converting_curl_to_python.ipynb). If you prefer, you can use the unofficial Python package instead; see the top-level [README](https://github.com/adsabs/adsabs-dev-api/blob/master/README.md) for more information." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Contents\n", + "- [Search API](#Search-API)\n", + "- [Query parameters](#Query-Parameters)\n", + "- [Fields](#Fields)\n", + "- [Examples](#Examples)\n", + "- [Advanced search syntax](#Advanced-Search-Syntax)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Get search results\n", + "`curl {token_header} {base_url}/query?q=search+term&fl=field1,field2`\n", + "\n", + "To search, construct a URL using the base URL given [above](#search-api), along with your search parameters as detailed in the [Query Parameters](#query-parameters) section below. All text must be UTF-8 and URL encoded. For example:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "author%3Amart%C3%ADnez+neutron+star\n" + ] + } + ], + "source": [ + "import sys\n", + "\n", + "v = sys.version\n", + "\n", + "# accented letters, special characters, and spaces need to be encoded\n", + "# Python has a package for this, though the functionality is slightly different between python 2 and python 3\n", + "if int(v[0]) == 2:\n", + " import urllib\n", + " query = \"author:martínez neutron star\"\n", + " encoded_query = urllib.quote_plus(query)\n", + "else:\n", + " from urllib.parse import urlencode, quote_plus\n", + " query = {\"author\":\"martínez neutron star\"}\n", + " encoded_query = urlencode(query,quote_via=quote_plus)\n", + "print(encoded_query)\n", + "\n", + "# note that the colon (:) may be encoded, depending on the algorithm you use. Your curl request \n", + "# should accept either the unencoded colon (:) or the encoded version (%3A)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The response from your `curl` statement will be JSON encoded. \n", + "\n", + "Also, note that the search API uses the same syntax as [Apache Solr](http://lucene.apache.org/solr/). For a full reference of query possibilities, please refer to the Solr documentation and [ADS Search Help](http://adsabs.github.io/help/search/). Later sections in this notebook present useful parameters and patterns for the vast majority of use cases, but are not meant to be exhaustive.\n", + "\n", + "Note: to print the JSON response in a more readable manner, add `| python -m json.tool` to the end of your `curl` statement." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\n", + " \"response\": {\n", + " \"docs\": [\n", + " {\n", + " \"author\": [\n", + " \"P\\u00e9rez Rojas, Hugo\",\n", + " \"Mart\\u00ednez, Aurora P\\u00e9rez\",\n", + " \"Mesquita, Alexandre\",\n", + " \"Razeira, Mois\\u00e9s\",\n", + " \"Gomes, Rosana O.\",\n", + " \"Vasconcellos, C\\u00e9sar A. Z.\"\n", + " ]\n", + " }\n", + " ],\n", + " \"numFound\": 168,\n", + " \"start\": 0\n", + " },\n", + " \"responseHeader\": {\n", + " \"QTime\": 743,\n", + " \"params\": {\n", + " \"fl\": \"author\",\n", + " \"q\": \"author:mart\\u00ednez neutron star\",\n", + " \"rows\": \"1\",\n", + " \"start\": \"0\",\n", + " \"wt\": \"json\",\n", + " \"x-amzn-trace-id\": \"Root=1-5b354e2a-a5b6ddca2051f3311a0beb8f\"\n", + " },\n", + " \"status\": 0\n", + " }\n", + "}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 398 100 398 0 0 398 0 0:00:01 0:00:01 --:--:-- 366\r", + "100 398 100 398 0 0 398 0 0:00:01 0:00:01 --:--:-- 366\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "curl -H \"Authorization: Bearer $token\" \\\n", + " \"https://api.adsabs.harvard.edu/v1/search/query?q=author%3Amart%C3%ADnez+neutron+star&fl=author&rows=1\" \\\n", + " | python -m json.tool" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Parse a query\n", + "`curl {token_header} {base_url}/qtree?q=termORother+term` \n", + "\n", + "Use the syntax above to return a query tree, also known as an Abstract Syntax Tree (AST), as understood by our query parser. This can be useful if you want to modify and/or enhance queries." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"responseHeader\":{\"status\":0,\"QTime\":3,\"params\":{\"q\":\"author:martínez neutron star\",\"x-amzn-trace-id\":\"Root=1-5b2d4f71-c818c6a8dfcab0d0fc622eba\",\"fl\":\"id\",\"start\":\"0\",\"rows\":\"10\",\"wt\":\"json\"}},\"qtree\":\"\\n{\\\"name\\\":\\\"OPERATOR\\\", \\\"label\\\":\\\"DEFOP\\\", \\\"children\\\": [\\n {\\\"name\\\":\\\"MODIFIER\\\", \\\"label\\\":\\\"MODIFIER\\\", \\\"children\\\": [\\n {\\\"name\\\":\\\"TMODIFIER\\\", \\\"label\\\":\\\"TMODIFIER\\\", \\\"children\\\": [\\n {\\\"name\\\":\\\"FIELD\\\", \\\"label\\\":\\\"FIELD\\\", \\\"children\\\": [\\n {\\\"name\\\":\\\"TERM_NORMAL\\\", \\\"input\\\":\\\"author\\\", \\\"start\\\":0, \\\"end\\\":5},\\n {\\\"name\\\":\\\"QNORMAL\\\", \\\"label\\\":\\\"QNORMAL\\\", \\\"children\\\": [\\n {\\\"name\\\":\\\"TERM_NORMAL\\\", \\\"input\\\":\\\"martínez\\\", \\\"start\\\":7, \\\"end\\\":14}]\\n }]\\n }]\\n }]\\n },\\n {\\\"name\\\":\\\"MODIFIER\\\", \\\"label\\\":\\\"MODIFIER\\\", \\\"children\\\": [\\n {\\\"name\\\":\\\"TMODIFIER\\\", \\\"label\\\":\\\"TMODIFIER\\\", \\\"children\\\": [\\n {\\\"name\\\":\\\"FIELD\\\", \\\"label\\\":\\\"FIELD\\\", \\\"children\\\": [\\n {\\\"name\\\":\\\"QNORMAL\\\", \\\"label\\\":\\\"QNORMAL\\\", \\\"children\\\": [\\n {\\\"name\\\":\\\"TERM_NORMAL\\\", \\\"input\\\":\\\"neutron\\\", \\\"start\\\":16, \\\"end\\\":22}]\\n }]\\n }]\\n }]\\n },\\n {\\\"name\\\":\\\"MODIFIER\\\", \\\"label\\\":\\\"MODIFIER\\\", \\\"children\\\": [\\n {\\\"name\\\":\\\"TMODIFIER\\\", \\\"label\\\":\\\"TMODIFIER\\\", \\\"children\\\": [\\n {\\\"name\\\":\\\"FIELD\\\", \\\"label\\\":\\\"FIELD\\\", \\\"children\\\": [\\n {\\\"name\\\":\\\"QNORMAL\\\", \\\"label\\\":\\\"QNORMAL\\\", \\\"children\\\": [\\n {\\\"name\\\":\\\"TERM_NORMAL\\\", \\\"input\\\":\\\"star\\\", \\\"start\\\":24, \\\"end\\\":27}]\\n }]\\n }]\\n }]\\n }]\\n}\"}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 1705 100 1705 0 0 1705 0 0:00:01 --:--:-- 0:00:01 7478\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "curl -H \"Authorization: Bearer $token\" \\\n", + " \"https://api.adsabs.harvard.edu/v1/search/qtree?q=author%3Amart%C3%ADnez+neutron+star\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Post a large identifier query\n", + "`method = POST`\n", + "\n", + "`curl -H \"Content-Type: big-query/csv\" {token_header} {base_url}/bigquery?{urlparams} -X POST -d {payload}`\n", + "\n", + "This type of query, typically called a \"big query\" for short, returns [standard search results](#get-search-results), but instead of the typical search terms, accepts as its input a list of bibcodes. There is a maximum of 2000 bibcodes allowed for each big query request. There are also limits on how often you can make a big query request; typically, only 100 requests per day per user are allowed.\n", + "\n", + "Name | Type | Description \n", + ":-------------|:------:|:--------------\n", + "`urlparams` |`string`| **Required**. Query parameters must be encoded as in a [standard query](#get-search-results). The query parameter `q=` must be included even if no additional searching is desired; to prevent it from restricting the search results, enter `q=*:*`. It's also recommended to use the `fl=` parameter to specify which fields to return.\n", + "`payload` |`string`| **Required**. Newline (`\\n`) separated list of bibcodes to return. The first line specifies the field that will be used for searching, i.e. `bibcode` (for example, `bibcode\\n1907AN....174...59.\\n1908PA.....16..445.\\n1989LNP...334..242S`)\n", + "\n", + "Notes:\n", + "- Currently, we only allow searching in the `bibcode` field. You can submit both canonical bibcodes (the ones that appear on a paper's abstract page, or in the URL pointing to the abstract page) as well as alternate bibcodes (e.g. bibcodes that pointed to a previous version of a paper, such as the arXiv version). The search automatically recognizes both types.\n", + "- The list of bibcodes supplied in the `payload` is *only applied after the main search*, specified with the `q=` parameter. To return the entire list of supplied bibcodes, use `q=*:*`.\n", + "- Currently, the default number of results returned by the big query is not controlled by the number of bibcodes requested, but rather is set to the default value of 10. To adjust this, set `&rows=` either to the number of bibcodes requested or to the maximum value of 2000.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"responseHeader\":{\"status\":0,\"QTime\":22,\"params\":{\"q\":\"*:*\",\"x-amzn-trace-id\":\"Root=1-5b2d4f73-0ac58ac0850c2bcc3104f85e\",\"fl\":\"bibcode,title\",\"start\":\"0\",\"fq\":\"{!bitset}\",\"rows\":\"2000\",\"wt\":\"json\"}},\"response\":{\"numFound\":3,\"start\":0,\"docs\":[{\"bibcode\":\"1908PA.....16..445.\",\"title\":[\"Variable Stars\"]},{\"bibcode\":\"1907AN....174...59.\",\"title\":[\"Kleine Mitteilungen\"]},{\"bibcode\":\"1989LNP...334..242S\",\"title\":[\"The Optical and Radio Properties of X-Ray Selected Bl-Lacertae Objects\"]}]}}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 557 100 490 100 67 490 67 0:00:01 --:--:-- 0:00:01 3396\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "curl -H \"Content-Type: big-query/csv\" -H \"Authorization: Bearer $token\" \\\n", + " \"https://api.adsabs.harvard.edu/v1/search/bigquery?q=*:*&fl=bibcode,title&rows=2000\" \\\n", + " -X POST -d $'bibcode\\n1907AN....174...59.\\n1908PA.....16..445.\\n1989LNP...334..242S'\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Query Parameters\n", + "All query parameters passed in the search URL must be UTF-8, URL-encoded strings. Due to the requirements of the authentication library used for validating requests, most non-ASCII characters that appear in the URL need to be encoded; for example, the double quote character (\") should be encoded as %22. In most programming languages, such as Python, the libraries used to retrieve content from web services will do this encoding for you; however, if you're using `curl` requests in the shell, such as shown in the examples in this notebook, you'll need to perform this encoding yourself.\n", + "\n", + "#### q\n", + "**Required** The search query. This should be a UTF-8, URL-encoded string of <=1000 characters. `q` accepts both fields (`title:exoplanets`) and unfielded (`exoplanets`) searches.\n", + "\n", + "#### rows \n", + "The number of results to return. The default is 10.\n", + "\n", + "#### start\n", + "The starting point for returned results, used for pagination. The default is 0. To return the next page of results, set `start` equal to the value of `start` from the previous request, plus the number of results returned in the previous request. For the default values, set `start=10` to return the second page of results.\n", + "\n", + "#### fl\n", + "The list of fields to return. The value should be a comma separated list of field names, e.g. `fl=bibcode,author,title`. The default is the document id (`fl=id`). A non-exhaustive list of available fields [is shown below](#fields).\n", + "\n", + "#### fq\n", + "Filters the list of search results. The syntax is the same as that for the `q` parameter. Adding search parameters via the `fq` parameter can speed up search results, as it searches only the results returned by the search entered via the `q` parameter, not the entire index. This parameter may be used more than once in a single search URL.\n", + "\n", + "#### sort\n", + "The sorting field and direction to be used when returning results. The format requires both the field to sort on (see [the list below](#fields) and the direction, either `asc` or `desc` (ascending or descending). For example, an appropriately formatted sort parameter is `sort=citation_count+desc`. The default sort method is the relevancy score as calculated by the search engine. Other useful fields to sort on may be `pubdate`, `read_count`, `first_author`, or `bibcode`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Fields\n", + "This is a non-exhaustive list of fields available for searching via the API. A more comprehensive list is available in our [help pages](https://adsabs.github.io/help/search/comprehensive-solr-term-list). These fields can be used with the `fl` and `sort` parameters.\n", + "\n", + "* `abstract` - the abstract of the record\n", + "* `ack` - the acknowledgements section of an article\n", + "* `aff` - an array of the authors' affiliations\n", + "* `alternate_bibcode` - list of alternate bibcodes for a single record; if a published record also has an arXiv version, the bibcode associated with the arXiv version is here\n", + "* `alternate_title` - list of alternate titles for a single record (usually if they are in multiple languages)\n", + "* `arxiv_class` - the arXiv class the pre-print was submitted to\n", + "* `author` - an array of the author names associated with the record\n", + "* `bibcode` - the canonical ADS bibcode identifier for this record\n", + "* `bibgroup` - the bibliographic groups that the bibcode has been associated with\n", + "* `bibstem` - the abbreviated name of the journal or publication, e.g., *ApJ*.\n", + "* `body`\\* - the full text content of the article\n", + "* `citation_count` - number of citations the item has received\n", + "* `copyright` - the copyright applied to the article\n", + "* `data` - the list of sources that have data related to this bibcode\n", + "* `database` - database the record is associated with (astronomy, physics, or general). By default, all three databases are searched; to limit to astronomy articles, add `fq=database:astronomy` to the URL\n", + "* `doi`- the digital object identifier of the article\n", + "* `doctype` - the type of document it is (see [here](https://adsabs.github.io/help/search/search-syntax) for a list of doctypes)\n", + "* `first_author` - the first author of the article\n", + "* `grant` - the list of grant IDs and agencies noted within an article\n", + "* `id` - a **non-persistent** unique integer for this record, used for fast look-up of a document\n", + "* `identifier` - an array of alternative identifiers for the record. May contain alternate bibcodes, DOIs and/or arxiv ids.\n", + "* `indexstamp` - time at which the document was (last) indexed\n", + "* `issue` - issue the record appeared in\n", + "* `keyword` - an array of normalized and un-normalized keyword values associated with the record\n", + "* `lang`\\* - the language of the article's title\n", + "* `orcid_pub` - ORCiD iDs supplied by publishers\n", + "* `orcid_user` - ORCiD iDs supplied by knonwn users in the ADS\n", + "* `orcid_other` - ORCiD iDs supplied by anonymous users in the ADS\n", + "* `page` - starting page\n", + "* `property` - an array of miscellaneous flags associated with the record (see [here](https://adsabs.github.io/help/search/search-syntax) for a list of properties\n", + "* `pub` - the canonical name of the publication the record appeared in\n", + "* `pubdate` - publication date in the form YYYY-MM-DD (DD value will always be \"00\")\n", + "* `read_count` - number of times the record has been viewed within in a 90-day windows (ads and arxiv)\n", + "* `title` - the title of the record\n", + "* `vizier` - the subject tags given to the article by VizieR\n", + "* `volume` - volume the record appeared in\n", + "* `year` - the year the article was published\n", + "\n", + "\\* *These fields are only indexed and so are only searchable - they are not stored or returned by the search end point, if requested*" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Examples\n", + "Example `curl` requests." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"responseHeader\":{\"status\":0,\"QTime\":6,\"params\":{\"q\":\"bibcode:2011ApJ...737..103S\",\"x-amzn-trace-id\":\"Root=1-5b2d4f77-eb5609cc57124c307500906c\",\"fl\":\"title\",\"start\":\"0\",\"rows\":\"10\",\"wt\":\"json\"}},\"response\":{\"numFound\":1,\"start\":0,\"docs\":[{\"title\":[\"Measuring Reddening with Sloan Digital Sky Survey Stellar Spectra and Recalibrating SFD\"]}]}}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 344 100 344 0 0 344 0 0:00:01 --:--:-- 0:00:01 1702\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# search by bibcode, return the title\n", + "curl -H \"Authorization: Bearer $token\" \\\n", + " \"https://api.adsabs.harvard.edu/v1/search/query?q=bibcode:2011ApJ...737..103S&fl=title\"" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"responseHeader\":{\"status\":0,\"QTime\":977,\"params\":{\"q\":\"black holes\",\"x-amzn-trace-id\":\"Root=1-5b2d4f78-87a695a096d4ece4e36e76c6\",\"fl\":\"bibcode\",\"start\":\"0\",\"fq\":\"database:astronomy\",\"rows\":\"10\",\"wt\":\"json\"}},\"response\":{\"numFound\":60233,\"start\":0,\"docs\":[{\"bibcode\":\"2004IAUS..222..115T\"},{\"bibcode\":\"2006kbls.confE..15H\"},{\"bibcode\":\"1997PKAS...12..145P\"},{\"bibcode\":\"1992ianp.symp..225S\"},{\"bibcode\":\"2017APS..APR.E5009F\"},{\"bibcode\":\"2017nova.pres.2298K\"},{\"bibcode\":\"1986Sci...234..882T\"},{\"bibcode\":\"2002PhRvD..66f3505B\"},{\"bibcode\":\"1996PhRvL..77.5164K\"},{\"bibcode\":\"2017ApJ...840L..24F\"}]}}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 600 100 600 0 0 600 0 0:00:01 0:00:01 --:--:-- 528\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# search in all metadata fields for \"black holes\", restricted to the astronomy database; return a list of bibcodes\n", + "curl -H \"Authorization: Bearer $token\" \\\n", + " \"https://api.adsabs.harvard.edu/v1/search/query?q=black+holes&fq=database:astronomy&fl=bibcode\"" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"responseHeader\":{\"status\":0,\"QTime\":223,\"params\":{\"q\":\"\\\"dark energy\\\"\",\"x-amzn-trace-id\":\"Root=1-5b2d4f7a-026b88865a0439add4cb3005\",\"fl\":\"bibcode,citation_count\",\"start\":\"0\",\"sort\":\"citation_count desc\",\"fq\":\"author:\\\"Spergel, D\\\"\",\"rows\":\"10\",\"wt\":\"json\"}},\"response\":{\"numFound\":62,\"start\":0,\"docs\":[{\"citation_count\":8728,\"bibcode\":\"2003ApJS..148..175S\"},{\"citation_count\":6436,\"bibcode\":\"2007ApJS..170..377S\"},{\"citation_count\":6397,\"bibcode\":\"2011ApJS..192...18K\"},{\"citation_count\":4484,\"bibcode\":\"2009ApJS..180..330K\"},{\"citation_count\":3976,\"bibcode\":\"2003ApJS..148....1B\"},{\"citation_count\":1581,\"bibcode\":\"2009ApJS..180..306D\"},{\"citation_count\":1383,\"bibcode\":\"2009ApJS..180..225H\"},{\"citation_count\":1209,\"bibcode\":\"2010MNRAS.401.2148P\"},{\"citation_count\":478,\"bibcode\":\"2010MNRAS.404...60R\"},{\"citation_count\":437,\"bibcode\":\"2011ApJS..192...17B\"}]}}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 866 100 866 0 0 866 0 0:00:01 --:--:-- 0:00:01 2086\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# search in all metadata fields for \"dark energy\", filter by author, return a list of bibcodes and citation counts, \n", + "# sorted by citation count (the double quotes are encoded as %22)\n", + "curl -H \"Authorization: Bearer $token\" \\\n", + " \"https://api.adsabs.harvard.edu/v1/search/query?q=%22dark+energy%22&fq=author:%22Spergel,+D%22&sort=citation_count+desc&fl=bibcode,citation_count\"\n", + " " + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"responseHeader\":{\"status\":0,\"QTime\":68,\"params\":{\"q\":\"author:\\\"Kurtz, M\\\"\",\"x-amzn-trace-id\":\"Root=1-5b2d4f7b-beaf5e3c449be466ef2fb630\",\"fl\":\"title\",\"start\":\"0\",\"fq\":\"property:refereed\",\"rows\":\"10\",\"wt\":\"json\"}},\"response\":{\"numFound\":116,\"start\":0,\"docs\":[{\"title\":[\"Correlated radon and CO2 variations near the San Andreas Fault\"]},{\"title\":[\"Methanol synthesis over ZnO: A structure-sensitive reaction?\"]},{\"title\":[\"Effects of Hyperbaric Hypocapnic Ventilation with One Hundred Per Cent Oxygen and Tham on Anoxia by Tracheal Occlusion *\"]},{\"title\":[\"Multiple pass laser interferometer\"]},{\"title\":[\"Electron impact ionization in the vicinity of comets\"]},{\"title\":[\"Measuring the Dark Matter Scale of Local Group Dwarf Spheroidals\"]},{\"title\":[\"Zaragozic acids: a family of fungal metabolites that are picomolar competitive inhibitors of squalene synthase.\"]},{\"title\":[\"A cognitive system for astronomical image interpretation\"]},{\"title\":[\"Sphingoid Base 1-phosphate Phosphatase: A Key Regulator of Sphingolipid Metabolism and Stress Response\"]},{\"title\":[\"Effect of Experience on Retention and Elimination of Misconceptions about Molecular Structure and Bonding\"]}]}}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 1189 100 1189 0 0 1189 0 0:00:01 --:--:-- 0:00:01 3898\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# limit a search to only refereed articles\n", + "curl -H \"Authorization: Bearer $token\" \\\n", + " \"https://api.adsabs.harvard.edu/v1/search/query?q=author:%22Kurtz,+M%22&fq=property:refereed&fl=title\"" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"responseHeader\":{\"status\":0,\"QTime\":137,\"params\":{\"q\":\"\\\"transiting exoplanets\\\" \",\"x-amzn-trace-id\":\"Root=1-5b2d4f7c-4920a2362a8bb94865f5483c\",\"fl\":\"bibcode\",\"start\":\"0\",\"sort\":\"bibcode desc\",\"rows\":\"5\",\"wt\":\"json\"}},\"response\":{\"numFound\":1631,\"start\":0,\"docs\":[{\"bibcode\":\"2018sptz.prop14131M\"},{\"bibcode\":\"2018sptz.prop14059B\"},{\"bibcode\":\"2018sptz.prop14047D\"},{\"bibcode\":\"2018Sci...359.1453C\"},{\"bibcode\":\"2018RNAAS...2b..40M\"}]}}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 439 100 439 0 0 439 0 0:00:01 --:--:-- 0:00:01 993\r", + "100 439 100 439 0 0 439 0 0:00:01 --:--:-- 0:00:01 993\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# search for the phrase \"transiting exoplanets\", fetch 5 rows\n", + "curl -H \"Authorization: Bearer $token\" \\\n", + " \"https://api.adsabs.harvard.edu/v1/search/query?q=%22transiting+exoplanets%22+&rows=5&fl=bibcode&sort=bibcode+desc\"" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"responseHeader\":{\"status\":0,\"QTime\":142,\"params\":{\"q\":\"\\\"transiting exoplanets\\\"\",\"x-amzn-trace-id\":\"Root=1-5b2d503c-893f6dcad5341f72c5f8fa5a\",\"fl\":\"bibcode\",\"start\":\"5\",\"sort\":\"bibcode desc\",\"rows\":\"5\",\"wt\":\"json\"}},\"response\":{\"numFound\":1631,\"start\":5,\"docs\":[{\"bibcode\":\"2018RNAAS...2b..20E\"},{\"bibcode\":\"2018RNAAS...2a..47J\"},{\"bibcode\":\"2018RNAAS...2a..28L\"},{\"bibcode\":\"2018PASP..130g4401B\"},{\"bibcode\":\"2018PASP..130f4502T\"}]}}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 438 100 438 0 0 438 0 0:00:01 --:--:-- 0:00:01 1575\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# same, but return the next 5 rows\n", + "curl -H \"Authorization: Bearer $token\" \\\n", + " \"https://api.adsabs.harvard.edu/v1/search/query?q=%22transiting+exoplanets%22&rows=5&start=5&fl=bibcode&sort=bibcode+desc\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Advanced Search Syntax\n", + "The `q` parameter supports both fielded and unfielded searches:\n", + "- `black holes`\n", + "- `title:exoplanets`\n", + "\n", + "Use quotation marks to indicate phrase searching (remember to encode quotes and spaces in `curl` statements; double quotes are encoded as %22, while spaces can be encoded as `+`)\n", + "- `\"black holes\"`\n", + "- `title:\"dark energy\"`\n", + "\n", + "Prepend terms with `+` or `-` to indicate inclusion or exclusion (note that after encoding spaces in the search string, the URL may include `++` or `+-`):\n", + "- `\"transiting exoplanets\" -Kepler`\n", + "- `\"transiting exoplanets\" +JWST`" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"responseHeader\":{\"status\":0,\"QTime\":120,\"params\":{\"q\":\"\\\"transiting exoplanets\\\" JWST\",\"x-amzn-trace-id\":\"Root=1-5b2d4f81-06c03436981012dfca638861\",\"fl\":\"title\",\"start\":\"0\",\"rows\":\"3\",\"wt\":\"json\"}},\"response\":{\"numFound\":150,\"start\":0,\"docs\":[{\"title\":[\"Transiting Exoplanets with JWST\"]},{\"title\":[\"Spectroscopy of transiting exoplanets: From Spitzer to JWST\"]},{\"title\":[\"Characterizing Transiting Exoplanet Atmospheres with JWST\"]}]}}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 441 100 441 0 0 441 0 0:00:01 --:--:-- 0:00:01 1373\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "curl -H \"Authorization: Bearer $token\" \\\n", + " \"https://api.adsabs.harvard.edu/v1/search/query?q=%22transiting+exoplanets%22++JWST&fl=title&rows=3\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To filter by a publication date range, you can use either the `year` or the `pubdate` fields:\n", + "- `pubdate:[2013-07-00 TO *]`\n", + "- `pubdate:[2005-01 TO 2007-01]`\n", + "- `pubdate:2013-02`\n", + "- `year:2013`\n", + "- `year:[2012 TO 2013]`\n", + "\n", + "The default search uses a boolean `AND` between terms, but you may use `OR` and `AND` in combination with `()` to create more complex queries.\n", + "\n", + "Prefix queries (wildcards, `*`) are supported for most fields." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 2", + "language": "python", + "name": "python2" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 2 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython2", + "version": "2.7.14" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/export.md b/export.md deleted file mode 100644 index 30af052..0000000 --- a/export.md +++ /dev/null @@ -1,22 +0,0 @@ -## Export service - -``` -https://api.adsabs.harvard.edu/v1/export/:format -``` -Currently, our API supports export in the following formats: AASTeX, BibTeX and EndNote. To get e.g. BibTeX for a set of records you do a POST request to the endpoint - - https://api.adsabs.harvard.edu/v1/export/bibtex - -with in the POST header - - {"bibcode":["","", ...]} - -Using `curl`, to get the BibTeX for e.g. the record with bibcode `2015ApJS..219...21Z`, you would do - - curl -H "Authorization: Bearer " -H "Content-Type: application/json" -X POST -d '{"bibcode":["2015ApJS..219...21Z"]}' https://api.adsabs.harvard.edu/v1/export/bibtex - -and the API then responds in JSON with - - {u'msg': u'Retrieved 1 abstracts, starting with number 1. Total number selected: 1.', u'export': u'@ARTICLE{2015ApJS..219...21Z,\n author = {{Zhang}, M. and {Fang}, M. and {Wang}, H. and {Sun}, J. and \n\t{Wang}, M. and {Jiang}, Z. and {Anathipindika}, S.},\n title = "{A Deep Near-infrared Survey toward the Aquila Molecular Cloud. I. Molecular Hydrogen Outflows}",\n journal = {\\apjs},\narchivePrefix = "arXiv",\n eprint = {1506.08372},\n primaryClass = "astro-ph.SR",\n keywords = {infrared: ISM, ISM: jets and outflows, shock waves, stars: formation, stars: winds, outflows},\n year = 2015,\n month = aug,\n volume = 219,\n eid = {21},\n pages = {21},\n doi = {10.1088/0067-0049/219/2/21},\n adsurl = {http://adsabs.harvard.edu/abs/2015ApJS..219...21Z},\n adsnote = {Provided by the SAO/NASA Astrophysics Data System}\n}\n\n'} - -The get AASTeX back, you use `aastex` as format parameter, and `endnote` for EndNote. diff --git a/libraries.md b/libraries.md deleted file mode 100644 index 910d633..0000000 --- a/libraries.md +++ /dev/null @@ -1,302 +0,0 @@ -# ADS Bibliographic Libraries API - -The source code of this microservice can be viewed [here](https://github.com/adsabs/biblib-service). - -## API end point -`https://api.adsabs.harvard.edu/v1/biblib` - -### End points and methods - -#### /libraries - -End point to create a library or view all libraries for a given user. - -##### GET -Obtains all the libraries that belong to the requesting user, along with some additional metadata. - -*Return data*: - * name: - * **string** - * Name of the library - * id: - * **string** - * ID of the library - * description: - * **string** - * Description of the library - * num_documents: - * **int** - * Number of documents in the library - * date_created: - * **string** - * ISO date library was created - * date_last_modified: - * **string** - * ISO date library was last modified - * permission: - * **string** - * Permission type, can be: 'read', 'write', 'admin', or 'owner' - * public: - * **boolean** - * True means it is a public library - * num_users: - * **int** - * Number of users with permissions to this library - * owner: - * **string** - * Identifier of the user who created the library - -*Permissions*: -Must be authenticated via the API. - -##### POST -Creates a library for the requesting user. Returns metadata on the library created. - -*Post body*: - * *name*: - * **string** - * name of the library (must be unique for that user) - * *description*: - * **string** - * description of the library - * *public*: - * **boolean** - * is the library public to view - * *bibcode (OPTIONAL)*: - * **list** - * list of bibcodes to add - -*Return data*: - * **name**: - * Name of the library - * **id**: - * - * ID of the library - * **description**: - * - * Description of the library - -*Permissions*: -Must be authenticated via the API. - -#### /libraries/<library_id> - -End point to interact with a specific library, only returns library content if the user has the correct privileges. See the document end point for deleting a library. - - * **library_id** [**string**] is the unique identifier of the library, as specified in the `/libraries` GET response. - -##### GET -Obtains all the documents of a given library. - -*Return data*: - - * documents: - * **list** - * Currently, a list containing the bibcodes. - * solr: - * **dict** - * The response from the solr bigquery end point - * metadata: - * **dict** - * contains the following: - * name: - * **string** - * Name of the library - * id: - * **string** - * ID of the library - * description: - * **string** - * Description of the library - * num_documents: - * **int** - * Number of documents in the library - * date_created: - * **string** - * ISO date library was created - * date_last_modified: - * **string** - * ISO date library was last modified - * permission: - * **sting** - * Permission type, can be: 'read', 'write', 'admin', or 'owner' - * public: - * **boolean** - * True means it is public - * num_users: - * **int** - * Number of users with permissions to this library - * owner: - * **string** - * Identifier of the user who created the library - * updates: - * **dict** - * contains the following - * num_updated: - * **int** - * Number of documents modified based on the response from solr - * duplicates_removed: - * **int** - * Number of files removed because they are duplications - * update_list: - * **list**[**dict**] - * List of dictionaries such that a single element described the original bibcode (key) and the updated bibcode now being stored (item) - -*Permissions*: -The following type of user can read a library: - - owner - - admin - - write - - read - -#### /documents/<library_id> - -End point to interact with a specific library, by adding documents and removing documents. You also use this endpoint to delete the entire library. - - * **library_id** [**string**] is the unique identifier of the library, as specified in the `/libraries` GET response. - -##### POST - -Request that adds a document to a library for a given user. - -*Post body*: - - * bibcode: - * **list** - * List of bibcodes to be added - * action: - * **string** - * 'add' or 'remove' - * add - adds a bibcode - * remove - removes a bibcode - -*Return data*: - * number_added: - * **int** - * number of documents added (if 'add' is used) - * number_removed: - * **int** - * number of documents removed (if 'remove' is used) - -*Permissions*: -The following type of user can add documents: - - owner - - admin - - write - -##### PUT - -Request that updates the meta-data of the library - -*Post-body*: - * name: - * **string** - * name of the library - * description: - * **string** - * description of the library - * public: - * **boolean** - * if the library should be publicly viewable - -*Note: The above are optional. Leave fields out if they do not need to be updated.* - -*Return data*: - * name: - * **string** - * name of the library - * description: - * **string** - * description of the library - * public: - * **boolean** - * if the library should be publicly viewable - -*Note: returns the key/value that was requested to be updated* - -*Permissions*: -The following type of user can update the 'name', 'library', and 'public' attributes: - - owner - - admin - -##### DELETE - -Request that deletes a library. - -*Permissions*: - -The following type of user can update a library: - - owner - -#### /permissions/<library_id> - -End point to manipulate the permissions between a user and a library. - - * **library_id** [**string**] is the unique identifier of the library, as specified in the `/libraries` GET response. - -##### GET -Request that returns the permissions for a given library. - -*Return data*: - * list of dictionaries, where each dictionary is for a specific user. The dictionary contains a list of all the permissions of the user. - * **list** - * user dictionary - * **dict** - * **key**: - * **string** - * user e-mail - * **item**: - * **list** - * list of permissions: 'read', 'write', 'admin', 'owner' - -*Permissions*: -The following type of user can access permissions of a library: - - owner - - admin - -##### POST -Request that modifies the permissions of a library - -*Post data*: - - * username: - * **string** - * specifies which user's permissions to be modified - * permission: - * **string** - * 'read', 'write', 'admin' or 'owner' - * value: - * **boolean** - * whether the user has this permission (True=yes, False=no) - - -*Permissions*: -The following type of user can update a permission for a user for a given library: - - owner - - admin - -#### /transfer/<library_id> - -End point to transfer a the ownership of a library to another ADS user. - - * **library_id** [**string**] is the unique identifier of the library, as specified in the `/libraries` GET response. - -##### POST -Request that transfers the ownership of a library from one user to another. - -*Post body*: - * transfer_user: - * **string** - * username of the user the account should be transfered to - -*Permissions*: -The following type of user can transfer libraries: - - owner - -## Permissions - * **owner**: Owns the library. Can read libraries, add/remove bibcodes, delete the library, add/remove permissions, transfer ownership of the library. - * **admin**: Can read libraries, add/remove bibcodes, add/remove permissions. - * **write**: Can read libraries, add/remove bibcdes. - * **read**: Can read libraries. - -Any other user that does not have a permission to the library they try to visit, can only see it if it is public. If the user has no permissions, they cannot view this library. diff --git a/metrics.md b/metrics.md deleted file mode 100644 index 5c15534..0000000 --- a/metrics.md +++ /dev/null @@ -1,257 +0,0 @@ -## Metrics Search Requests - -To retrieve metrics data for a particular search result set append `/metrics/` to the API search endpoint path. - -``` -https://api.adsabs.harvard.edu/v1/metrics -``` - -If you do a request from the command line like - - curl -H "Authorization: Bearer " -H "Content-Type: application/json" -X POST -d '{"bibcodes":["1980ApJS...44..137K","1980ApJS...44..489B"]}' https://api.adsabs.harvard.edu/v1/metrics - -and you should get back all metrics data. If you access the API using the Python `requests` module, you would do something like - - r = requests.post(queryURL, data=json.dumps(payload), headers=headers) - -where - - payload = {'bibcodes': [bibcode1, bibcode2, ...]} - headers = {'Content-type': 'application/json', 'Accept': 'text/plain', 'Authorization':'Bearer %s' % token} - -and `token` is your API token. - -You can also request a specific set of metrics by adding the `types` parameter in the JSON header. For example: - - curl -H "Authorization: Bearer " -H "Content-Type: application/json" -X POST -d '{"bibcodes":["1980ApJS...44..137K","1980ApJS...44..489B"],"types":["basic"]}' https://api.adsabs.harvard.edu/v1/metrics - -which will return just the basic statistics for the set of bibcodes. The following values are allowed for the `types` list: - - * 'basic': publication and usage stats (all papers, and just refereed papers) - * 'citations': citation stats - * 'indicators': indicators, like the h-index, g-index, m-index etc etc - * 'histograms': publication, citation, reads and downloads histograms - * 'timeseries': time series for a set of indicators - -If you just want a specific histogram, you can add a `histograms` parameter to the JSON header, specifying that histogram. For example: - - curl -H "Authorization: Bearer " -H "Content-Type: application/json" -X POST -d '{"bibcodes":["1980ApJS...44..137K","1980ApJS...44..489B"],'types':['histograms'], 'histograms':['publications']}' https://api.adsabs.harvard.edu/v1/metrics - -to get just the publications histogram. The service returns JSON with attributes determined by the contents of the `types` list (no such list means that everything gets returned). It always contains an attribute 'skipped bibcodes', which is a list of bibcodes for which no metrics data could be found. The following overview shows the JSON attributes returned by each type: - - * 'basic': 'basic stats' and 'basic stats refereed' - * 'citations': 'citation stats' and 'citation stats refereed' - * 'indicators': 'indicators' and 'indicators refereed' - * 'histograms': 'histograms' - * 'timeseries': 'time series' - -Each of these attributes contains specific data elements. The following overview shows what each section contains (the phrases listed are the attribute names). - -### 'basic stats' and 'basic stats refereed' - * number of papers - * normalized paper count - * median number of reads - * average number of reads - * total number of reads - * normalized number of reads - * median number of downloads - * average number of downloads - * total number of downloads - * normalized number of downloads - -### 'citation stats' and 'citation stats refereed' - * number of citing papers - * median number of citations - * average number of citations - * total number of citations - * normalized number of citations - * median number of refereed citations - * average number of refereed citations - * total number of refereed citations - * normalized number of refereed citations - * number of self-citations - -### 'indicators' - * i10 - * i100 - * h - * m - * g - * read10 - * tori - -### 'histograms' -For all histograms, the actual histogram values are stored as follows: - - { - "histogram name": {year: value, year: value, ...} - } - -where values can be either integers or floats, depending on the type of histograms. The histogram name is an attribute of the histogram type. For example, the `publications` type has 4 separate histograms: `all publications`, `refereed publications`, `all publications normalized` and `refereed publications normalized`, which are organized as follows in the JSON output: - - { - "histograms": { - "publications": { - "all publications": {}, - "refereed publications": {}, - "all publications normalized": {}, - "refereed publications normalized": {}, - - } - } - } - -The following overview lists for all histogram types the names of the actual histograms stored in them: - - * 'publications': 'all publications', 'refereed publications', 'all publications normalized', 'refereed publications normalized' - * 'reads': 'all reads', 'refereed reads', 'all reads normalized', 'refereed reads normalized' - * 'downloads': 'all downloads', 'refereed downloads', 'all downloads normalized', 'refereed downloads normalized' - * 'citations': 'refereed to refereed', 'nonrefereed to refereed', 'refereed to nonrefereed', 'nonrefereed to nonrefereed', 'refereed to refereed normalized', 'nonrefereed to refereed normalized', 'refereed to nonrefereed normalized', 'nonrefereed to nonrefereed normalized' - -Note that 'refereed reads' means: 'number of reads for the refereed publications in the set'. - -### 'timeseries' -The `timeseries` attribute contains time series data for: `h`, `g`, `i10`, `i100`, `read10` and `tori`. The output is structured as - - { - "time series": { - "h": { values }, - "g": { values }, - "i10": { values }, - "i100": { values }, - "read10": { values }, - "tori": { values }, - - } - } - -where the 'values' are organized in the same way as for the histograms: - - { - "indicator name": {year: value, year: value, ...} - } - -## Everything together -So, returning everything, the output looks like - - { - "skipped bibcodes": [ ... ], - "basic stats": - { - "number of papers": 1, - "normalized paper count": 0.1, - "median number of reads": 1.1, - "average number of reads": 1.1, - "total number of reads": 1, - "normalized number of reads": 0.1, - "median number of downloads": 1.1, - "average number of downloads": 1.1, - "total number of downloads": 1, - "normalized number of downloads": 0.1, - }, - "basic stats refereed": - { - "number of papers": 1, - "normalized paper count": 0.1, - "median number of reads": 1.1, - "average number of reads": 1.1, - "total number of reads": 1, - "normalized number of reads": 0.1, - "median number of downloads": 1.1, - "average number of downloads": 1.1, - "total number of downloads": 1, - "normalized number of downloads": 0.1, - }, - "citation stats": - { - "number of citing papers": 1, - "median number of citations": 0.1, - "average number of citations": 1.1, - "total number of citations": 1, - "normalized number of citations": 0.1, - "median number of refereed citations": 0.1, - "average number of refereed citations": 1.1, - "total number of refereed citations": 1, - "normalized number of refereed citations": 0.1, - "number of self-citations": 1, - }, - "citations stats refereed": - { - "number of citing papers": 1, - "median number of citations": 0.1, - "average number of citations": 1.1, - "total number of citations": 1, - "normalized number of citations": 0.1, - "median number of refereed citations": 0.1, - "average number of refereed citations": 1.1, - "total number of refereed citations": 1, - "normalized number of refereed citations": 0.1, - "number of self-citations": 1, - }, - "indicators": - { - "i10": 1, - "i100": 1, - "h": 1, - "m": 1.1, - "g": 1, - "read10": 0.1, - "tori": 1.1, - "riq": 1, - }, - "indicators refereed": - { - "i10": 1, - "i100": 1, - "h": 1, - "m": 1.1, - "g": 1, - "read10": 0.1, - "tori": 1.1, - "riq": 1, - }, - "histograms": - { - "publications": { - "all publications": { values }, - "refereed publications": { values }, - "all publications normalized": { values }, - "refereed publications normalized": { values }, - }, - "reads": { - "all reads": { values }, - "refereed reads": { values }, - "all reads normalized": { values }, - "refereed reads normalized": { values }, - }, - "downloads": { - "all downloads": { values }, - "refereed downloads": { values }, - "all downloads normalized": { values }, - "refereed downloads normalized": { values }, - }, - "citations": { - "refereed to refereed": { values }, - "nonrefereed to refereed": { values }, - "refereed to nonrefereed": { values }, - "nonrefereed to nonrefereed": { values }, - "refereed to refereed normalized": { values }, - "nonrefereed to refereed normalized": { values }, - "refereed to nonrefereed normalized": { values }, - "nonrefereed to nonrefereed normalized": { values }, - }, - }, - "time series": - { - "h": { values }, - "g": { values }, - "i10": { values }, - "i100": { values }, - "read10": { values }, - "tori": { values }, - } - } - -where, as before, the 'values' have the form - - {year: value, year: value, ...} diff --git a/search.md b/search.md deleted file mode 100644 index 015b2bf..0000000 --- a/search.md +++ /dev/null @@ -1,268 +0,0 @@ -## Contents - -* [Search API](#search-api) -* [Query parameters](#query-parameters) -* [Fields](#fields) -* [Examples](#example-search-requests) -* [Advanced search syntax](#advanced-search-syntax) - - -## Search API - -Base URL: `https://api.adsabs.harvard.edu/v1/search` - -Please note that all queries to the API need to include an HTTP header specifying an access token, e.g. - - curl -H 'Authorization: Bearer ' 'https://api.adsabs.harvard.edu/v1/search/query?q=star' - -For more information on this please see the top-level [README](README.md) file. - -## Get search results - - GET /query?q=value&fl=value2..... - -All text values shoud be UTF-8 and url-encoded. The response body will be json encoded. - -Note that the search API uses the same syntax as [Apache Solr](http://lucene.apache.org/solr/). For a full reference of query possibilities, please refer to the Solr documentation and [ADS Search Help](http://adsabs.github.io/help/search/). The sections below present useful parameters and patterns for the vast majority of use cases, but are not meant to be exhaustive. - - -## Parse a query - - GET /qtree?q=this+OR+that - -Returns a `query tree` (Abstract Syntax Tree - AST) as understood by our query parser. Useful if you want to modify and/or enhance -queries. - -Example response (value in the `qtree` is a string (JSON) serialized version of the AST): - -``` -{ - "qtree": "\n{\"name\":\"OPERATOR\", \"label\":\"DEFOP\", \"children\": [\n {\"name\":\"MODIFIER\", \"label\":\"MODIFIER\", \"children\": [\n {\"name\":\"TMODIFIER\", \"label\":\"TMODIFIER\", \"children\": [\n {\"name\":\"FIELD\", \"label\":\"FIELD\", \"children\": [\n {\"name\":\"QNORMAL\", \"label\":\"QNORMAL\", \"children\": [\n {\"name\":\"TERM_NORMAL\", \"input\":\"star\", \"start\":0, \"end\":3}]\n }]\n }]\n }]\n }]\n}", - "responseHeader": { - "status": 0, - "QTime": 6, - "params": { - "q": "star", - "wt": "json", - "fl": "id" - } - } - } -``` - -## Post a large identifier query - - POST /bigquery[?urlparams] - -Returns standard search results, but accepts as input a very large query (i.e. a query that can be expressed only as a list of search -criteria, typically IDs). There is currently no limit to the size of the submitted data (besides buffer/time limits imposed by our API -frontend), however there are severe limits on how often you can call this enpoint. Typically, only 100 requests per day are allowed. - -### Parameters - -Name | Type | Description ---------------|--------|-------------- -`urlparams` |`string`| **Required**. Url escaped/serialized query parameters if used in the URL (ie: `q=star&fl=bibcode,title`) or form encoded search paramaters. -`payload` |`string`| **Required**. Newline separated list of values, the first line specifies index that will be used for search (ie: `bibcode\n1907AN....174...59.\n1908PA.....16..445.\n1989LNP...334..242S`) - - -Currently, we allow to search in `bibcode` index only. You can submit `canonical` as well as `alternate` bibcodes; the search will automatically match both. In the future, the list of available indexes may be extended. (We do not plan to support search with other values than IDs). - -The bigquery filter is *applied only after* the main search (ie: it limits results of the main search). - -Example python session - get all papers from ADS and filter them using several IDs (thus returning only records for the list of input bibcodes): - -```python -import requests -bibcodes="bibcode\n1907AN....174...59.\n1908PA.....16..445.\n1989LNP...334..242S" -r = requests.post('https://api.adsabs.harvard.edu/v1/search/bigquery', - params={'q':'*:*', 'wt':'json', 'fq':'{!bitset}', 'fl':'bibcode'}, - headers={'Authorization': 'Bearer my_token'}, - data=bibcodes) -``` - -Output is exactly the same as from `/query` endpoint. - -An example of a bigquery using curl: - -``` -curl -H "Content-Type: big-query/csv" -H "Authorization: Bearer {token}" "https://api.adsabs.harvard.edu/v1/search/bigquery?q=*:*&wt=json&fl=bibcode" -X POST -d $'bibcode\n1907AN....174...59.\n1908PA.....16..445.\n1989LNP...334..242S\n' - -``` - - - -## Query parameters - -All query parameters appearing in the search URL must be UTF-8, url-encoded strings. Please note that due to the requirements of the authentication library used for validating requests, most non-ascii characters appearing in the URL need to be hex-encoded, e.g. a double quote character (") must be encoded as %22. In most programming languages the libraries used to retrieve content from web services will perform this encoding for you, but if you are using your own curl-based request you will need to take care of this. - -[Below](#fields) you can find the list of fields available to all API users. A more comprehensive list is [available in our help pages](https://adsabs.github.io/help/search/comprehensive-solr-term-list). - -#### q -*required:* a UTF-8, url-encoded string of <= 1000 characters representing the search query. `q` can be used for both fielded (`title:exoplanets`), and unfielded (`exoplanets`) search. -#### rows -number of results to return. Default is 10. - -#### start -starting point for returned results (for pagination). Default is 0. - -#### fl -Fields list: specify the fields contained in each returned document. Value should be a comma-separated list of field names. Default is just the document id (field `id`). - -#### fq -Filter query: filter your results using a particular `field:value` condition. This parameter is repeatable. - -* `field` can be any field listed in the field descriptions below -* `value` should be a UTF-8, url-encoded string - -#### sort -Indicate how you wish the result set sorted. The format is `field direction` where `direction` is one of `asc` or `desc`, and `field` is a field in the document that contains a numerical value. The default sorting is by relevance (computed by our search engine). - -Example of a properly formated & encoded sort param is `sort=read_count+desc` - -Some useful fields to sort by may be `pubdate`, `citation_count`, or `first_author`. - -## Fields - -Below are the fields available to all API users: - -* `abstract` - the abstract of the record -* `ack` - the acknowledgements section of an article -* `aff` - an array of the authors' affiliations -* `alternate_bibcode` - list of alternate bibcodes for a single record -* `alternate_title` - list of alternate titles for a single record (usually if they are in multiple languages) -* `arxiv_class` - the arXiv class the pre-print was submitted to -* `author` - an array of the author names associated with the record -* `bibcode` - the canonical ADS bibcode identifier for this record -* `bibgroup` - the bibliographic groups that the bibcode has been associated with -* `bibstem` - the abbreviated name of the journal or publication, e.g., *ApJ*. -* `body`\* - the full text content of the article -* `citation_count` - number of citations the item has received -* `copyright` - the copyright applied to the article -* `data` - the list of sources that have data related to this bibcode -* `database` - Which database the record is associated with. -* `doi`- the digital object identifier of the article -* `doctype` - the type of document it is (see [here](https://adsabs.github.io/help/search/search-syntax) for a list of doctypes) -* `first_author` - the first author of the article -* `grant` - the list of grant IDs and agencies noted within an article -* `id` - a (**non-persistent**) unique integer for this record, used for fast look-up of a document -* `identifier` - an array of alternative identifiers for the record. May contain alternative bibcodes, DOIs and/or arxiv ids. -* `indexstamp` - time at which the document was (last) indexed -* `issue` - issue the record appeared in -* `keyword` - an array of normalized and un-normalized keyword values associated with the record -* `lang`\* - the language of the article's title -* `orcid_pub` - ORCiD iDs supplied by publishers -* `orcid_user` - ORCiD iDs supplied by knonwn users in the ADS -* `orcid_other` - ORCiD iDs supplied by anonymous users in the ADS -* `page` - starting page -* `property` - an array of miscellaneous flags associated with the record (see [here](https://adsabs.github.io/help/search/search-syntax) for a list of properties -* `pub` - the canonical name of the publication the record appeared in -* `pubdate` - publication date in the form YYYY-MM-DD (DD value will always be "00") -* `read_count` - number of times the record has been viewed within in a 90-day windows (ads and arxiv) -* `title` - the title of the record -* `vizier` - the subject tags given to the article by VizieR -* `volume` - volume the record appeared in -* `year` - the year the article was published - -\* *These fields are only indexed and so are only searchable - they are not stored or returned by the search end point, if requested* - -#### databases - -The ADS content is divided into a few separate databases: "astronomy", "physics" & "general". By default, queries will return results from all three. To limit a search to "astronomy" articles, add `fq=database:astronomy` to the request parameters. - -## Example search requests - -Assuming your access token is `my_token`, all queries can be executed by any HTTP client. An example `curl` request is: - -`curl -H "Authorization: Bearer my_token" "https://api.adsabs.harvard.edu/v1/search/query?........"` - -...and by adding the following query params, the response will contain relevant documents: - - # Search by bibcode (make sure you enclose these arguments in quotes!) - ?q=bibcode:2011ApJ...737..103S - - # Search for "black holes", restricted to astronomy content - ?q=black+holes&fq=database:astronomy - - # Search for "dark energy", filter by author, sort by citation count - # (double quotes encoded as %22) - ?q=dark+energy&fq=author:%22Civano,+F%22&sort=citation_count+desc - - # Same search but only return *bibcode* and *property* values - ?q=dark+energy&fq=author:%22Civano,+F%22&sort=citation_count+desc&fl=bibcode,property - - # Limit a search to only refereed articles - ?q=author:%22Kurtz,+M%22&fq=property:refereed - - # Search for the phrase "transiting exoplanets", get 200 rows - ?q=%22transiting+exoplanets%22&rows=200 - - # Same search but get the next 200 rows - ?q=%22transiting+exoplanets%22&rows=200&start=201 - - -## Example record response - - { - "bibcode": "2012A&A...542A..16R", - "author": [ - "Ranalli, P.", - "Comastri, A.", - ... - ], - "pub": "Astronomy and Astrophysics", - "identifier": [ - "2012arXiv1204.4485R", - "arXiv:1204.4485", - "2012A&A...542A..16R" - ], - "title": "X-ray properties of radio-selected star forming galaxies in the Chandra-COSMOS survey", - "property": [ - "REFEREED", - "ARTICLE" - ], - "abstract": "X-ray surveys contain sizable numbers of star forming galaxies, ..." - "keyword": [ - "astronomy x rays", - "astronomy radio", - "galaxies fundamental parameters", - "galaxies star clusters", - "galaxies active", - ... - ], - "aff": [ - "Università di Bologna, Dipartimento di Astronomia, via Ranzani 1, 40127, Bologna, Italy ; Institute of Astronomy and Astrophysics, National Observatory of Athens, Palaia Penteli, 15236, Athens, Greece; INAF - Osservatorio Astronomico di Bologna, via Ranzani 1, 40127, Bologna, Italy", - "INAF - Osservatorio Astronomico di Bologna, via Ranzani 1, 40127, Bologna, Italy", - ... - ], - } - - -## Advanced Search Syntax - -The `q` parameter supports both fielded and unfielded searching: - -* `black holes` -* `title:black` `title:holes` - -Use quotation marks to indicate phrase searching: - -* `"black holes"` -* `title:"black holes"` - -Prepend terms with "+" or "-" to indicate inclusion or exclusion: - -* `+transiting exoplanets` -* `"dark energy" -"weak lensing"` - -To filter by a publication date range you can use either the `year` or `pubdate` fields: - -* `pubdate:[2013-07-00 TO *]` -* `pubdate:[2005-01 TO 2007-01]` -* `pubdate:2013-02` -* `year:2013` -* `year:[2012 TO 2013]` - -The default search uses a boolean `AND` between terms, but you may use `OR` and `AND` in combination with `()` to create more complex queries. - -Prefix queries (wildcards, `*`) are supported for most fields. From b996a62040ca980ff8fe54265f04d58702f559aa Mon Sep 17 00:00:00 2001 From: Kelly Lockhart <2926089+kelockhart@users.noreply.github.com> Date: Mon, 18 Mar 2019 14:34:22 -0400 Subject: [PATCH 30/39] Added set operations to libraries notebook --- Libraries_API.ipynb | 227 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 218 insertions(+), 9 deletions(-) diff --git a/Libraries_API.ipynb b/Libraries_API.ipynb index 5f41c40..aa53489 100644 --- a/Libraries_API.ipynb +++ b/Libraries_API.ipynb @@ -264,6 +264,215 @@ "curl -H \"Authorization: Bearer $token\" https://api.adsabs.harvard.edu/v1/biblib/libraries/hHGU1Ef-TpacAhicI3J8kQ | python -m json.tool" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### /libraries/operations/<library_id>\n", + "Perform set operations on one or more libraries. The `library_id` (string) is the unique identifier of the primary library, as specified in the `/libraries` GET response. Depending on the operation, a secondary library ID may need to be passed in the post payload.\n", + "\n", + "Function allowed: POST\n", + "\n", + "#### POST\n", + "The following operations are supported:\n", + "* union: take the union of the primary and the secondary libraries. The result is saved to a new library\n", + "* intersection: take the intersection of the primary and the secondary libraries. The result is saved to a new library\n", + "* difference: take the difference between the primary and the secondary libraries. The result is saved to a new library\n", + "* copy: copy the contents of the primary library into the secondary library. The secondary library is not emptied first; use the empty operation on the secondary library first in order to create a duplicate of the primary library in the secondary library\n", + "* empty: empty the primary library of its contents (no secondary library ID is needed)\n", + "\n", + "Specify that the request is a POST method by adding `-X POST` to your request. You must include a payload, passed via the `-d` flag, in JSON format. Add this header to your request to indicate your payload is in the appropriate format: `-H \"Content-Type: application/json\"`. The payload must include:\n", + "* action: (string, required) set operation to perform; allowed values are [`union`, `intersection`, `difference`, `copy`, `empty`]\n", + "* libraries: (list) list of secondary library IDs; multiple secondary libraries are allowed for [`union`, `intersection`, `difference`]; one secondary library is allowed for `copy`; no secondary libraries are allowed for `empty`\n", + "* name: (string, optional) name of the new library to be created for [`union`, `intersection`, `difference`]; the name must be unique for a given user; if no name is specified, the name used will be \"Untitled\" with a timestamp\n", + "* description: (string, optional) description of the new library to be created for [`union`, `intersection`, `difference`]; if no description is specified, the description used will include the set operation used and the primary and secondary library IDs\n", + "* public: (Boolean, optional) sets whether the new library created by [`union`, `intersection`, `difference`] is publicly viewable or not; the new library will be private unless otherwise specified\n", + "\n", + "*Return data*:\n", + "* name: (string) Name of the library\n", + "* id: (string) ID of the library\n", + "* description: (string) Description of the library\n", + "\n", + "*Permissions*:\n", + "The following type of user can perform set operations:\n", + "* owner\n", + "* admin\n", + "* write\n" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"id\": \"BeUbSWXtTPCaRhRV3nec_w\", \"bibcode\": [\"2009A&A...502..515G\", \"2011ApJ...734...87R\", \"2010ApJ...710..248C\", \"2003ApJ...595..665X\", \"2011ApJ...731L..12S\", \"2010ApJ...721..505R\", \"2010ApJ...710L.156R\", \"2010ApJ...723.1255R\", \"2006ApJ...639L..51A\", \"2001AJ....122.2993S\", \"2010ASPC..423..355R\", \"2010A&A...518A..59G\", \"2010ApJ...721L..48K\"], \"description\": \"Union of 891661ca-1bda-4967-b4b9-6c1c5cab7bef with [u'rdRs4g4AQ522k9TJ2umtVA', u's56fjycaRMKxw6tDEr31XQ']\", \"name\": \"New union library\"}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 613 100 498 100 115 498 115 0:00:01 --:--:-- 0:00:01 2043\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# take the union of 3 libraries, using the default description and public settings\n", + "curl -H \"Authorization: Bearer $token\" -H \"Content-Type: application/json\" \\\n", + " https://api.adsabs.harvard.edu/v1/biblib/libraries/operations/iRZhyhvaSWe0uWwcXKt77w \\\n", + " -X POST \\\n", + " -d '{\"action\": \"union\", \"libraries\": [\"rdRs4g4AQ522k9TJ2umtVA\", \"s56fjycaRMKxw6tDEr31XQ\"], \"name\": \"New union library\"}'" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"id\": \"8V8DzNqyQuOyTX6RF9x7qQ\", \"bibcode\": [\"2001AJ....122.2993S\", \"2009A&A...502..515G\", \"2006ApJ...639L..51A\", \"2010ApJ...710..248C\", \"2003ApJ...595..665X\", \"2010A&A...518A..59G\", \"2011ApJ...731L..12S\"], \"description\": \"Sample intersection library\", \"name\": \"New intersection library\"}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 438 100 289 100 149 289 149 0:00:01 --:--:-- 0:00:01 2147\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# take the intersection of 2 libraries, using the default public settings\n", + "curl -H \"Authorization: Bearer $token\" -H \"Content-Type: application/json\" \\\n", + " https://api.adsabs.harvard.edu/v1/biblib/libraries/operations/BeUbSWXtTPCaRhRV3nec_w \\\n", + " -X POST \\\n", + " -d '{\"action\": \"intersection\", \"libraries\": [\"rdRs4g4AQ522k9TJ2umtVA\"], \"name\": \"New intersection library\", \"description\": \"Sample intersection library\"}'" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"id\": \"9KBh8sVHQxaVkoPALtoXgg\", \"bibcode\": [\"2011ApJ...734...87R\", \"2010ApJ...723.1255R\", \"2010ApJ...710L.156R\", \"2010ASPC..423..355R\", \"2010ApJ...721L..48K\", \"2010ApJ...721..505R\"], \"description\": \"Sample difference library\", \"name\": \"New difference library\"}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 421 100 262 100 159 262 159 0:00:01 --:--:-- 0:00:01 2170\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# take the difference of 2 libraries\n", + "curl -H \"Authorization: Bearer $token\" -H \"Content-Type: application/json\" \\\n", + " https://api.adsabs.harvard.edu/v1/biblib/libraries/operations/BeUbSWXtTPCaRhRV3nec_w \\\n", + " -X POST \\\n", + " -d '{\"action\": \"difference\", \"libraries\": [\"rdRs4g4AQ522k9TJ2umtVA\"], \"name\": \"New difference library\", \"description\": \"Sample difference library\", \"public\": true}'" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"public\": true, \"bibcode\": [], \"name\": \"New difference library\", \"description\": \"Sample difference library\"}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 129 100 110 100 19 110 19 0:00:01 --:--:-- 0:00:01 689\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# empty a library of all of its bibcode contents\n", + "curl -H \"Authorization: Bearer $token\" -H \"Content-Type: application/json\" \\\n", + " https://api.adsabs.harvard.edu/v1/biblib/libraries/operations/9KBh8sVHQxaVkoPALtoXgg \\\n", + " -X POST \\\n", + " -d '{\"action\": \"empty\"}'" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\"public\": true, \"bibcode\": [\"2009A&A...502..515G\", \"2010ApJ...710L.156R\", \"2010ApJ...710..248C\", \"2003ApJ...595..665X\", \"2011ApJ...731L..12S\", \"2010ApJ...721..505R\", \"2011ApJ...734...87R\", \"2010ApJ...723.1255R\", \"2006ApJ...639L..51A\", \"2001AJ....122.2993S\", \"2010ASPC..423..355R\", \"2010A&A...518A..59G\", \"2010ApJ...721L..48K\"], \"name\": \"New difference library\", \"description\": \"Sample difference library\"}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 466 100 407 100 59 407 59 0:00:01 --:--:-- 0:00:01 2858\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# copy the bibcode contents from the primary library to the secondary library (this will not empty the library first)\n", + "curl -H \"Authorization: Bearer $token\" -H \"Content-Type: application/json\" \\\n", + " https://api.adsabs.harvard.edu/v1/biblib/libraries/operations/BeUbSWXtTPCaRhRV3nec_w \\\n", + " -X POST \\\n", + " -d '{\"action\": \"copy\", \"libraries\": [\"9KBh8sVHQxaVkoPALtoXgg\"]}'" + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -293,7 +502,7 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 9, "metadata": {}, "outputs": [ { @@ -327,7 +536,7 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 10, "metadata": {}, "outputs": [ { @@ -390,7 +599,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 11, "metadata": { "scrolled": true }, @@ -439,7 +648,7 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 12, "metadata": {}, "outputs": [ { @@ -473,7 +682,7 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 13, "metadata": {}, "outputs": [ { @@ -527,7 +736,7 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 14, "metadata": { "scrolled": false }, @@ -582,7 +791,7 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 15, "metadata": {}, "outputs": [ { @@ -636,7 +845,7 @@ }, { "cell_type": "code", - "execution_count": 11, + "execution_count": 16, "metadata": {}, "outputs": [ { @@ -685,7 +894,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", - "version": "2.7.14" + "version": "2.7.13" } }, "nbformat": 4, From 81f1189273d07c747b9bbb4f4fe6e0c15002215c Mon Sep 17 00:00:00 2001 From: Kelly Lockhart <2926089+kelockhart@users.noreply.github.com> Date: Tue, 19 Mar 2019 14:22:10 -0400 Subject: [PATCH 31/39] Added info on how to download PDFs via the API --- Search_API.ipynb | 83 +++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 82 insertions(+), 1 deletion(-) diff --git a/Search_API.ipynb b/Search_API.ipynb index b3dd01a..63e67d8 100644 --- a/Search_API.ipynb +++ b/Search_API.ipynb @@ -587,6 +587,87 @@ "\n", "Prefix queries (wildcards, `*`) are supported for most fields." ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Downloading PDFs\n", + "PDFs may be downloaded using a GET request to the `link_gateway` endpoint. The URL to query consists of the base URL `https://ui.adsabs.harvard.edu/link_gateway/`, the bibcode to retrieve the PDF for, and an endpoint to indicate whether to download the arXiv PDF (`/EPRINT_PDF`) or the publisher PDF (`PUB_PDF`). In addition to the usual header that contains your API token, the following flags will also need to be set:\n", + "* `-L` - this flag follows the redirect URL to the arXiv or to the publisher's site\n", + "* `-o /full/path/to/output.pdf` - this flag and the required path parameter that follows will download the PDF to the specified location" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + " 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0\r", + "100 281 100 281 0 0 281 0 0:00:01 0:00:01 --:--:-- 163\n", + "\r", + "100 305 100 305 0 0 305 0 0:00:01 0:00:01 --:--:-- 305\n", + "\r", + "100 286 100 286 0 0 143 0 0:00:02 0:00:02 --:--:-- 143\n", + "\r", + " 0 553k 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0\r", + " 37 553k 37 208k 0 0 70997 0 0:00:07 0:00:03 0:00:04 1024k\r", + "100 553k 100 553k 0 0 184k 0 0:00:03 0:00:03 --:--:-- 1641k\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# download the arXiv PDF\n", + "curl -H \"Authorization: Bearer $token\" \\\n", + " 'https://ui.adsabs.harvard.edu/link_gateway/1999ApJ...517..565P/EPRINT_PDF' \\\n", + " -L -o '/full/path/to/output.pdf'" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + " % Total % Received % Xferd Average Speed Time Time Time Current\n", + " Dload Upload Total Spent Left Speed\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 295 100 295 0 0 295 0 0:00:01 --:--:-- 0:00:01 1666\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", + "100 318 0 318 0 0 318 0 --:--:-- --:--:-- --:--:-- 521\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0\n", + "\r", + " 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0\r", + "100 184k 0 184k 0 0 94585 0 --:--:-- 0:00:02 --:--:-- 173k\r", + "100 2258k 0 2258k 0 0 752k 0 --:--:-- 0:00:03 --:--:-- 1295k\n" + ] + } + ], + "source": [ + "%%bash\n", + "token=\"your-token-here\"\n", + "# download the publisher PDF\n", + "curl -H \"Authorization: Bearer $token\" \\\n", + " 'https://ui.adsabs.harvard.edu/link_gateway/1998AJ....116.1009R/PUB_PDF' \\\n", + " -L -o '/full/path/to/output.pdf'" + ] } ], "metadata": { @@ -605,7 +686,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", - "version": "2.7.14" + "version": "2.7.13" } }, "nbformat": 4, From 9bcd908841beaf79f7c72acbf3e381aa7753ac76 Mon Sep 17 00:00:00 2001 From: Kelly Lockhart <2926089+kelockhart@users.noreply.github.com> Date: Mon, 8 Apr 2019 16:34:04 -0400 Subject: [PATCH 32/39] Added max number of rows to search API docs --- Search_API.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Search_API.ipynb b/Search_API.ipynb index 63e67d8..b12af0b 100644 --- a/Search_API.ipynb +++ b/Search_API.ipynb @@ -255,7 +255,7 @@ "**Required** The search query. This should be a UTF-8, URL-encoded string of <=1000 characters. `q` accepts both fields (`title:exoplanets`) and unfielded (`exoplanets`) searches.\n", "\n", "#### rows \n", - "The number of results to return. The default is 10.\n", + "The number of results to return. The default is 10 and the maximum is 2000.\n", "\n", "#### start\n", "The starting point for returned results, used for pagination. The default is 0. To return the next page of results, set `start` equal to the value of `start` from the previous request, plus the number of results returned in the previous request. For the default values, set `start=10` to return the second page of results.\n", From a689911d32dbd74298ff9134265d71fe463058d5 Mon Sep 17 00:00:00 2001 From: Kelly Lockhart <2926089+kelockhart@users.noreply.github.com> Date: Fri, 3 May 2019 14:15:45 -0400 Subject: [PATCH 33/39] Added content-type to big query example --- Converting_curl_to_python.ipynb | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/Converting_curl_to_python.ipynb b/Converting_curl_to_python.ipynb index 2a77099..6adb834 100644 --- a/Converting_curl_to_python.ipynb +++ b/Converting_curl_to_python.ipynb @@ -27,14 +27,14 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "{u'responseHeader': {u'status': 0, u'QTime': 283, u'params': {u'x-amzn-trace-id': u'Root=1-5b3552fa-ab284eb0f5fc9060ec6a6920', u'rows': u'1', u'q': u'author:mart\\xednez neutron star', u'start': u'0', u'wt': u'json', u'fl': u'author'}}, u'response': {u'start': 0, u'numFound': 168, u'docs': [{u'author': [u'P\\xe9rez Rojas, Hugo', u'Mart\\xednez, Aurora P\\xe9rez', u'Mesquita, Alexandre', u'Razeira, Mois\\xe9s', u'Gomes, Rosana O.', u'Vasconcellos, C\\xe9sar A. Z.']}]}}\n" + "{u'responseHeader': {u'status': 0, u'QTime': 1015, u'params': {u'x-amzn-trace-id': u'Root=1-5ccc81d3-3250564a86c43b181d6442e6', u'rows': u'1', u'q': u'author:mart\\xednez neutron star', u'start': u'0', u'wt': u'json', u'fl': u'author'}}, u'response': {u'start': 0, u'numFound': 179, u'docs': [{u'author': [u'P\\xe9rez Rojas, Hugo', u'Mart\\xednez, Aurora P\\xe9rez', u'Mesquita, Alexandre', u'Razeira, Mois\\xe9s', u'Gomes, Rosana O.', u'Vasconcellos, C\\xe9sar A. Z.']}]}}\n" ] } ], @@ -58,14 +58,14 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "{u'responseHeader': {u'status': 0, u'QTime': 42, u'params': {u'x-amzn-trace-id': u'Root=1-5b35547f-f290c88fb5e9c182bab8287f', u'rows': u'2000', u'fq': u'{!bitset}', u'q': u'*:*', u'start': u'0', u'wt': u'json', u'fl': u'bibcode,title'}}, u'response': {u'start': 0, u'numFound': 3, u'docs': [{u'bibcode': u'1908PA.....16..445.', u'title': [u'Variable Stars']}, {u'bibcode': u'1907AN....174...59.', u'title': [u'Kleine Mitteilungen']}, {u'bibcode': u'1989LNP...334..242S', u'title': [u'The Optical and Radio Properties of X-Ray Selected Bl-Lacertae Objects']}]}}\n" + "{u'responseHeader': {u'status': 0, u'QTime': 30, u'params': {u'x-amzn-trace-id': u'Root=1-5ccc81d8-a26813d03b9c5d6401eaab94', u'rows': u'2000', u'fq': u'{!bitset}', u'q': u'*:*', u'start': u'0', u'wt': u'json', u'fl': u'bibcode,title'}}, u'response': {u'start': 0, u'numFound': 3, u'docs': [{u'bibcode': u'1907AN....174...59.', u'title': [u'Kleine Mitteilungen']}, {u'bibcode': u'1908PA.....16..445.', u'title': [u'Variable Stars']}, {u'bibcode': u'1989LNP...334..242S', u'title': [u'The Optical and Radio Properties of X-Ray Selected Bl-Lacertae Objects']}]}}\n" ] } ], @@ -74,7 +74,7 @@ "bibcodes = \"bibcode\\n1907AN....174...59.\\n1908PA.....16..445.\\n1989LNP...334..242S\"\n", "r = requests.post(\"https://api.adsabs.harvard.edu/v1/search/bigquery\",\\\n", " params={\"q\":\"*:*\", \"fl\": \"bibcode,title\", \"rows\":2000},\n", - " headers={'Authorization': 'Bearer ' + token},\n", + " headers={'Authorization': 'Bearer ' + token, 'Content-Type': 'big-query/csv'},\n", " data=bibcodes)\n", "print(r.json())" ] @@ -91,7 +91,7 @@ }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 3, "metadata": {}, "outputs": [ { @@ -122,7 +122,7 @@ }, { "cell_type": "code", - "execution_count": 23, + "execution_count": 4, "metadata": {}, "outputs": [ { @@ -155,7 +155,7 @@ }, { "cell_type": "code", - "execution_count": 24, + "execution_count": 5, "metadata": {}, "outputs": [ { @@ -183,7 +183,7 @@ }, { "cell_type": "code", - "execution_count": 28, + "execution_count": 6, "metadata": { "scrolled": true }, @@ -215,7 +215,7 @@ }, { "cell_type": "code", - "execution_count": 29, + "execution_count": 7, "metadata": {}, "outputs": [ { @@ -248,7 +248,7 @@ }, { "cell_type": "code", - "execution_count": 33, + "execution_count": 8, "metadata": {}, "outputs": [ { @@ -275,7 +275,7 @@ }, { "cell_type": "code", - "execution_count": 34, + "execution_count": 9, "metadata": {}, "outputs": [ { @@ -319,7 +319,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", - "version": "2.7.14" + "version": "2.7.13" } }, "nbformat": 4, From 2a79e4aaba2f24834f4d8af2892624f8a286fc7e Mon Sep 17 00:00:00 2001 From: Kelly Lockhart <2926089+kelockhart@users.noreply.github.com> Date: Tue, 15 Oct 2019 13:35:18 -0400 Subject: [PATCH 34/39] Added Linux date syntax --- README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/README.md b/README.md index b6a28a5..4c8b829 100644 --- a/README.md +++ b/README.md @@ -69,7 +69,11 @@ And then noting the following values: The Limit variable indicates the amount of daily queries allowed to the user (in this case 5000). The Remaining variable indicates how many queries are still available. The Reset variable provides a UTC timestamp corresponding to the time the rate limits will be reset. To see its value in human-readable format, you can use the UNIX "date" command: + # syntax for UNIX on a Mac date -r 1435190400 + # Linux syntax + date -d @1435190400 + # output for either Wed Jun 24 20:00:00 EDT 2015 (the rate resetting happens at midnight UTC). From 4f4ac9e9cd16bc9f56d4b15aae48450c0cfb094b Mon Sep 17 00:00:00 2001 From: Kelly Lockhart <2926089+kelockhart@users.noreply.github.com> Date: Fri, 1 Nov 2019 15:03:08 -0400 Subject: [PATCH 35/39] Updated permissions schema to match microservice --- Libraries_API.ipynb | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/Libraries_API.ipynb b/Libraries_API.ipynb index aa53489..7df9966 100644 --- a/Libraries_API.ipynb +++ b/Libraries_API.ipynb @@ -778,11 +778,14 @@ "\n", "Specify that the request is a POST method by adding `-X POST` to your request. You must include a payload, passed via the `-d` flag, in JSON format. Add this header to your request to indicate your payload is in the appropriate format: `-H \"Content-Type: application/json\"`. The payload must include:\n", " * email: (string) specifies which user's permissions to be modified\n", - " * permission: (string) 'read', 'write', 'admin' or 'owner'\n", - " * value: (boolean) whether the user has this permission (true=yes, false=no)\n", + " * permission: (dict) specifies the permissions type and whether the user has this permission\n", + " - allowed keys: (string) 'read', 'write', or 'admin'\n", + " - allowed values: (boolean) whether the user has this permission (true=yes, false=no)\n", "\n", "There is no response to the request.\n", "\n", + "Note: An email will be sent to the affected user upon a successful permissions change.\n", + "\n", "*Permissions*:\n", "The following type of user can update a permission for a user for a given library:\n", " - owner\n", @@ -809,7 +812,7 @@ " Dload Upload Total Spent Left Speed\n", "\r", " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r", - "100 81 100 3 100 78 3 78 0:00:01 --:--:-- 0:00:01 465\n" + "100 67 100 3 100 64 3 64 0:00:01 --:--:-- 0:00:01 179\n" ] } ], @@ -820,7 +823,7 @@ "curl -H \"Authorization: Bearer $token\" -H \"Content-Type: application/json\" \\\n", " https://api.adsabs.harvard.edu/v1/biblib/permissions/hHGU1Ef-TpacAhicI3J8kQ \\\n", " -X POST \\\n", - " -d '{\"email\":\"test.user@example.email.com\", \"permission\":\"read\", \"value\": true}'" + " -d '{\"email\":\"test.user-api@example.email.com\", \"permission\": {\"read\": true}}'" ] }, { @@ -838,6 +841,8 @@ "\n", "There is no response to the request.\n", "\n", + "Note: An email will be sent to the affected user upon a successful permissions change.\n", + "\n", "*Permissions*:\n", "The following type of user can transfer libraries:\n", " - owner" From 390f0585c21147e31e2111b619df1f4629898698 Mon Sep 17 00:00:00 2001 From: Sergi Blanco-Cuaresma Date: Mon, 4 Nov 2019 09:42:43 -0500 Subject: [PATCH 36/39] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 4c8b829..a76aa0b 100644 --- a/README.md +++ b/README.md @@ -77,4 +77,4 @@ The Limit variable indicates the amount of daily queries allowed to the user (in Wed Jun 24 20:00:00 EDT 2015 (the rate resetting happens at midnight UTC). -To increase rate limits, please contact us directly at `adshelp@cfa.harvard.edu`. +For more information and tips about rate limits, please contact us directly at `adshelp@cfa.harvard.edu`. From e2033ce9d1dbccff6d3358bf167885205097a427 Mon Sep 17 00:00:00 2001 From: Kelly Lockhart <2926089+kelockhart@users.noreply.github.com> Date: Fri, 22 Nov 2019 11:20:07 -0500 Subject: [PATCH 37/39] Added info on returning search highlights --- Search_API.ipynb | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/Search_API.ipynb b/Search_API.ipynb index b12af0b..72b4049 100644 --- a/Search_API.ipynb +++ b/Search_API.ipynb @@ -588,6 +588,21 @@ "Prefix queries (wildcards, `*`) are supported for most fields." ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Highlighting search terms\n", + "Snippets of text, with search terms highlighted, may be returned by a search query. Limits are in place to prevent scraping of our full-text holdings, as noted below, but more highlights can be returned via these API calls than are displayed in the search results on the website.\n", + "\n", + "The following parameters control the highlights feature:\n", + "- `hl=true` - (required) turn on the highlighting feature\n", + "- `hl.fl` - (required) a comma-separated list of fields to return highlights for (e.g. title, abstract, or body)\n", + "- `hl.snippets` - number of highlighted snippets to return; the maximum is 4\n", + "- `hl.fragsize` - length of snippet to return; the maximum length is 100 characters. The snippet positioning is optimized to break on word and sentence boundaries, so the highlighted term will not always be centered in the snippet\n", + "- `hl.maxAnalyzedChars` - this parameter is only needed if the highlighted field is very long and the frequency of the search term is low. By default, the first 51,200 characters of a field are used for highlights. If the search is returning fewer snippets than expected, try increasing this value to capture more of the field. However, increasing this value unnecessarily will slow searching" + ] + }, { "cell_type": "markdown", "metadata": {}, From 872c50ef03b22ae3da632411318e919f19f07471 Mon Sep 17 00:00:00 2001 From: Kelly Lockhart <2926089+kelockhart@users.noreply.github.com> Date: Fri, 21 Feb 2020 12:40:13 -0500 Subject: [PATCH 38/39] Updated info on sorting by date --- Search_API.ipynb | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/Search_API.ipynb b/Search_API.ipynb index 72b4049..a94cd40 100644 --- a/Search_API.ipynb +++ b/Search_API.ipynb @@ -267,7 +267,7 @@ "Filters the list of search results. The syntax is the same as that for the `q` parameter. Adding search parameters via the `fq` parameter can speed up search results, as it searches only the results returned by the search entered via the `q` parameter, not the entire index. This parameter may be used more than once in a single search URL.\n", "\n", "#### sort\n", - "The sorting field and direction to be used when returning results. The format requires both the field to sort on (see [the list below](#fields) and the direction, either `asc` or `desc` (ascending or descending). For example, an appropriately formatted sort parameter is `sort=citation_count+desc`. The default sort method is the relevancy score as calculated by the search engine. Other useful fields to sort on may be `pubdate`, `read_count`, `first_author`, or `bibcode`." + "The sorting field and direction to be used when returning results. The format requires both the field to sort on (see [the list below](#fields) and the direction, either `asc` or `desc` (ascending or descending). For example, an appropriately formatted sort parameter is `sort=citation_count+desc`. The default sort method is the relevancy score as calculated by the search engine. Other useful fields to sort on may be `date`, `read_count`, `first_author`, or `bibcode`." ] }, { @@ -275,7 +275,9 @@ "metadata": {}, "source": [ "## Fields\n", - "This is a non-exhaustive list of fields available for searching via the API. A more comprehensive list is available in our [help pages](https://adsabs.github.io/help/search/comprehensive-solr-term-list). These fields can be used with the `fl` and `sort` parameters.\n", + "This is a non-exhaustive list of fields available for searching via the API. A more comprehensive list is available in our [help pages](https://adsabs.github.io/help/search/comprehensive-solr-term-list). These fields can be used with the `fl` and `sort` parameters. \n", + "\n", + "Note: some fields, such as body, can be searched but not returned via `fl` or sorted on. Also, multivalued fields, such as author, cannot be used for sorting.\n", "\n", "* `abstract` - the abstract of the record\n", "* `ack` - the acknowledgements section of an article\n", @@ -291,6 +293,7 @@ "* `citation_count` - number of citations the item has received\n", "* `copyright` - the copyright applied to the article\n", "* `data` - the list of sources that have data related to this bibcode\n", + "* `date` - the machine-readable version of `pubdate`, useful for sorting\n", "* `database` - database the record is associated with (astronomy, physics, or general). By default, all three databases are searched; to limit to astronomy articles, add `fq=database:astronomy` to the URL\n", "* `doi`- the digital object identifier of the article\n", "* `doctype` - the type of document it is (see [here](https://adsabs.github.io/help/search/search-syntax) for a list of doctypes)\n", @@ -308,7 +311,7 @@ "* `page` - starting page\n", "* `property` - an array of miscellaneous flags associated with the record (see [here](https://adsabs.github.io/help/search/search-syntax) for a list of properties\n", "* `pub` - the canonical name of the publication the record appeared in\n", - "* `pubdate` - publication date in the form YYYY-MM-DD (DD value will always be \"00\")\n", + "* `pubdate` - publication date in the form YYYY-MM-DD (DD value will always be \"00\"). This field is stored as a human-readable string, so is useful for being returned via `fl`, but not for sorting (see `date`)\n", "* `read_count` - number of times the record has been viewed within in a 90-day windows (ads and arxiv)\n", "* `title` - the title of the record\n", "* `vizier` - the subject tags given to the article by VizieR\n", From f49ab227a24460f0737000151c8f8d5b475092a6 Mon Sep 17 00:00:00 2001 From: Kelly Lockhart <2926089+kelockhart@users.noreply.github.com> Date: Mon, 2 Mar 2020 17:37:27 -0500 Subject: [PATCH 39/39] Minor clarification about export payload format --- Export_API.ipynb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Export_API.ipynb b/Export_API.ipynb index 0ceb304..4dbe408 100644 --- a/Export_API.ipynb +++ b/Export_API.ipynb @@ -33,7 +33,7 @@ "* Custom format (`/custom`)\n", "\n", "The POST payload (set via the `-d` flag) should be in JSON format. To indicate the payload format, you must add the following header to your `curl` request: `-H \"Content-Type: application/json\"`. For the export, the payload accepts the following keys: \n", - "* `bibcode`: **required**; set the value to a comma-separated list of the bibcodes to export\n", + "* `bibcode`: **required**; set the value to a list of the bibcodes to export\n", "* `sort`: optional; set the value to a combination of the field to sort on (see the Fields section in the Search API notebook), followed by a space plus `asc` or `desc` to indicate ascending or descending order. If the sort order is not specified, the default sort is by date, then by bibcode \n", "* `format`: **required when using custom format**; set the value to a string with the [desired formatting codes](http://adsabs.github.io/help/actions/export)\n", "\n", @@ -175,7 +175,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", - "version": "2.7.14" + "version": "2.7.13" } }, "nbformat": 4,