Optionally use range to get file size

[ahobson]

It seems that GitHub pages now responds to HEAD requests with the compressed size of the sqlite file. This PR adds an option to use GET and a byte Range of 0-1 to determine the real file size. Without this change, the VFS layer thinks the database is corrupted.

[mmomtchev]

Isn't it better to not accept the gzip Content-Encoding? The problem is that ranges won't work with gzip encoded data anyway?

[mmomtchev]

This seems to (not be entirely) the fault of Github since the Content-Length header they return is correct.

The sync backend which uses XMLHttpRequest reports the correct size, while fetch in the multiplexed backend reports the compressed size.

[ahobson]

First of all, thank you for sqlite-wasm-http! I've been using sql.js-httpvfs for a while, but I believe GitHub has changed some of their implementation and it stopped working.

I've been using the sync backend with sqlite-wasm-http and it wasn't working because GitHub returns the Content-Length of the compressed content on a HEAD request. It's unclear to me if that's spec compliant or not.

My first thought was to change the Accept-Encoding header, but from what I have read, that's not an allowed header for clients to set for XmlHttpRequest.

I'm totally open to other approaches, but this hack/workaround unblocked me.

[mmomtchev]

Ok, there is no solution for fetch from what it seems: whatwg/fetch#986

[mmomtchev]

Can you simply replace the fetch HEAD with an asynchronous XMLHttpRequest? Just the HEAD, the rest can continue using fetch. The sync backend does not need any changes, it works with compressed content. It is this obscure option, work-around-a-bug that bothers me.