iiuc right now, the probes do not set either Accept header not ?format= query parameter, and blindly assume that the response will be a CAR.
Problem
Probing as it is now, makes implicit assumptions about content types. This is very brittle and will produce errors on SPs that correctly implement specification, and/or default to block instead of CAR.
This will get worse over time, as more and more clients shifts towards Block (application/vnd.ipld.raw) being the lowest common denominator, and a CAR (application/vnd.ipld.car) being an optional performance optimization.
Example 1: misbehaving SP
There are SP that are not implementing specs correctly, and ignore ?format URL query parameter, returning a CAR when a single block is requested.
For example, below request for a raw block (?format=raw) returns a CAR instead:
$ curl "https://f010479.twinquasar.io/ipfs/bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi?format=raw" -I -X GET -s | grep -i -e http -e vnd.ipld
HTTP/2 200
content-type: application/vnd.ipld.car;version=1;order=dfs;dups=y
Passing both Accept and ?format is a reliable way of dealing with all permutations:
$ curl "https://f010479.twinquasar.io/ipfs/bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi?format=raw" -I -X GET -s -H "Accept: application/vnd.ipld.raw" | grep -i -e http -e vnd.ipld
HTTP/2 200
content-type: application/vnd.ipld.raw
This class of errors may go in other direction: the default response may be something other than block or a CAR.
Example 2: misbehaving probe code
getRetrievalUrl produces URL without explicit ?format=car:
|
return `${baseUrl}/ipfs/${cid}?dag-scope=block` |
This depends on some old Boost code? Will break if default response type is something other than CAR.
Suggested fix
Probes should be made with explicit content type ask (both Accept header and ?format),
mirroring most popular client implementations (boxo in GO / helia in JS) and avoiding false negatives and false positives.
References
iiuc right now, the probes do not set either
Acceptheader not?format=query parameter, and blindly assume that the response will be a CAR.Problem
Probing as it is now, makes implicit assumptions about content types. This is very brittle and will produce errors on SPs that correctly implement specification, and/or default to block instead of CAR.
This will get worse over time, as more and more clients shifts towards Block (application/vnd.ipld.raw) being the lowest common denominator, and a CAR (application/vnd.ipld.car) being an optional performance optimization.
Example 1: misbehaving SP
There are SP that are not implementing specs correctly, and ignore
?formatURL query parameter, returning a CAR when a single block is requested.For example, below request for a raw block (
?format=raw) returns a CAR instead:Passing both
Acceptand?formatis a reliable way of dealing with all permutations:This class of errors may go in other direction: the default response may be something other than block or a CAR.
Example 2: misbehaving probe code
getRetrievalUrlproduces URL without explicit?format=car:spark-checker/lib/spark.js
Line 344 in 799cc5e
This depends on some old Boost code? Will break if default response type is something other than CAR.
Suggested fix
Probes should be made with explicit content type ask (both
Acceptheader and?format),mirroring most popular client implementations (boxo in GO / helia in JS) and avoiding false negatives and false positives.
References