Support streaming bodies in the client

[evert]

Moved from here:

https://github.com/fruux/sabre-dav/issues/321

[Hywan]

Hello :-),

The Sabre\HTTP\Client::parseCurlResult method computes an array containing the response index. This index contains a Sabre\HTTP\Response object. This object extends Sabre\HTTP\Message. On this object, we have the getBody, getBodyAsStream and getBodyAsString methods.

So, because this response from Sabre\HTTP\Client::parseCurlResult is returned by the doRequest method, basically, this feature is already supported.

QED ￭.

[evert]

The point of this ticket is, that someone may be using the library to do downloads of very large files.

In those cases, we want to ensure that the entire file is accessed as a stream, and never placed into memory.

So while it's possible right now to convert the string into a stream, the goal is to change the client a bit so it uses a stream under the hood as well.

[Hywan]

@evert But the result is already in memory because cURL gives you all the “stuff”: https://github.com/fruux/sabre-http/blob/c55cbc1daa91293cda92ea4b90de79c743c4a149/lib/Client.php#L483. I will check if cURL can gives only few informations in order to create a stream.

[Hywan]

An interesting link from a friend of me @pmartin: http://stackoverflow.com/questions/1342583/manipulate-a-string-that-is-30-million-characters-long/1342760#1342760. However, I don't know how it works if we don't want to load the response yet we receive it but later: When reading the stream only.

[Hywan]

In case where the user has a stream ready and with the write permission, she can give this stream to the HTTP client and the response will be copied into this stream. This answer to one use-case.

[evert]

streams in request bodies already 100% work, this is about turning a HTTP response into a stream.

[Hywan]

@evert How does it work. I missed it in the source code?

[evert]

https://github.com/fruux/sabre-http/blob/master/lib/Client.php#L405

[Hywan]

@evert Yup, it's for sending a request. What you would like to do is for receiving a response, right?

[evert]

Indeed, yes!

[h44z]

Any news on this? I am very interrested in using streams as im going to down/upload large files >2GB with the webdav client.

[Hywan]

@h44z Not from me yet.

[evert]

This is still something that interesting for us, but we haven't had time implementing it yet.

[evert]

So the problem here is that curl actually doesn't have an easy way for us to just access the stream resource, as far as I can tell.

The only way we can progressively get access to the stream, is by using the CURLOPT_WRITEFUNCTION option, but that only gives us 'bits of the string' as opposed to a full-on stream.

With that function, we could send everything to a temporary stream (php://temp/) which would cache the result in memory, but that only solves part of the problem.

Ideally we'd want the response to return as soon as it starts coming in and not after all the bytes have arrived, and ideally we would want to not have to cache/buffer it anywhere.

I don't see an easy way to do that.