[🚀 Feature]: Add File Download Support to Bindings

[titusfortner]

Feature and motivation

In #11277 there is now support for moving files from a Grid standalone/node to the client machine.

A flag has to be set when the grid is started, but we should provide a Selenium specific endpoint and method for getting that from inside the bindings themselves.

@asolntsev I see you're tracking it for Selenide already, want to help get it in Selenium? selenide/selenide#1687

Usage example

String base64 = driver.download("my-file-name.pdf");

Or:

Zip remoteFile = driver.download("my-file-name.pdf");

Or Maybe:

driver.transferFromGrid("my-file-name.pdf", "/location/to/save/file");

[github-actions]

This issue is looking for contributors.

Please comment below or reach out to us through our IRC/Slack/Matrix channels if you are interested.

[asolntsev]

@titusfortner Sure, with a great pleasure. :)

But I have few suggestions:

1. We would need an endpoint for getting a list of all downloaded files. In many cases, we don't know the exact file name, and need to detect the downloaded file by comparing file list before and after user action.
2. Will it be new method only in RemoteWebDriver class?
3. I would not name it download. Rather getDownloadedFile() (and listDownloadedFiles()).

[titusfortner]

For the list I think we would need a separate implementation in the grid, right @krmahadevan?

[krmahadevan]

@titusfortner

So it looks like we have the below asks:

1. List the files that were downloaded till now.
2. Given a file name, retrieve its contents (This is present already)
3. Given a list of files, retrieve their contents.

The GET URL that currently responds to (2) is /session/:sessionId:/se/file?filename=

{
   "filename":"input.txt",
   "contents": "base64EncodedStringThatRepresentsAZippedFileContentsGoesHere"
}

To support (1), we could alter the behaviour such that if the user does a GET on /session/:sessionId:/se/file we can return back a payload that looks like

[
	{
		"filename":"input.txt",
		"contents": ""
	},
	{
		"filename":"anotherFile.txt",
		"contents": ""
	}
]

To support (3) we can have the user specify a comma separated list of file names /session/:sessionId:/se/file?filename=file1.txt,file2.txt

which would return back a payload of

[
	{
		"filename":"file1.txt",
		"contents": "base64EncodedStringThatRepresentsAZippedFileContentsGoesHere"
	},
	{
		"filename":"file2.txt",
		"contents": "base64EncodedStringThatRepresentsAZippedFileContentsGoesHere"
	}
]

So we wouldn't need to add additional URLs but can work within whatever we have already. Its just that for (1) wherein user asks for the list of files that can be downloaded it would be kind of awkward to send back a response but without the contents. But if we have a client API then we could perhaps abstract out this.

The client binding could either reside within the RemoteWebDriver (or) maybe we could do something like

DownloadManager dm = new DownloadManager(driver);
driver.transferFromGridAndUnzip("my-file-name.pdf", "/location/to/save/file"); //Since we give back only a zip file

Please let me know what do you think

[jsa34]

Just a quick comment about waits - I haven't checked the current implementation, but in order to avoid querying for a file that hasn't yet downloaded, should we create a wait or build the wait into fetch of the file (retry with polling intervals and timeout)?

[diemol]

List the files that were downloaded till now.

This makes sense. However, I would put this behind a flag because some users might be using a shared Grid and they just do not want to allow listing the directory contents. Also, it gets tricky because what if more than one session has downloaded a file? Should we list all files no matter the session that downloaded it or should we list only the files that were downloaded in that particular session? This all implies that we could potentially need to manage a state of downloaded files in the Node? I'd really like to avoid such overhead in the Node code (that already handles quite a bit).

Given a list of files, retrieve their contents.

I do not see the need for this. What is the use case that is not covered by a single file download? I would not expect users now using the Grid to download large amounts of file per session.

Just a quick comment about waits - I haven't checked the current implementation, but in order to avoid querying for a file that hasn't yet downloaded, should we create a wait or build the wait into fetch of the file (retry with polling intervals and timeout)?

This is the trickiest one because the download happens in a remote machine. AFAIK, there are only workarounds to know if a file has been completely downloaded or not, nothing really standard. That is why our suggestion is to alter the system under test to offer a way of knowing that the file has been downloaded.

Another thing I forgot to mention in the initial implementation is that we forgot to add some mechanism to clean downloaded files, otherwise the Node machine can run out of space at some point.

[krmahadevan]

@diemol

What is the use case that is not covered by a single file download? I would not expect users now using the Grid to download large amounts of file per session.

Lets say that a test basically downloads a file multiple times. Since a test cannot specify the name of the file to be saved as, depending on the OS, the file name gets chosen. For e.g., in OSX I noticed that, if there's a file name already available in the Downloads folder, then OSX appends ( 1 ), ( 2 ) and so on to the file name. So in this case, the user would perhaps not know what the actual file name is going to be. Another use case can be wherein a web application generates a file on the file and then saves it. In this case also the file name would not be available.
So maybe in such situations, it would be useful for a person to know the files that are available for download.

Maybe we can restrict the files on a session id basis (wherein inside the downloads path that a user specifies the respective files would be downloaded under a folder that represents a session id). That way, for a given session a user would be able to retrieve his/her downloaded files, but that list would perhaps never span across sessions and so the ask of My downloaded files should not be visible to the other user when I am sharing the same grid can be addressed.

we forgot to add some mechanism to clean downloaded files, otherwise the Node machine can run out of space at some point.

Should we drive it via a flag wherein we say if set to autoClean as true, then we basically clean up the folder every-time the node comes up ?

[krmahadevan]

Just a quick comment about waits - I haven't checked the current implementation, but in order to avoid querying for a file that hasn't yet downloaded, should we create a wait or build the wait into fetch of the file (retry with polling intervals and timeout)?

In the normal mode without the grid in the picture, today how do you determine if a file has been downloaded?

• Do you wait for some predetermined duration (or)
• Do you poll a directory wherein the file is going to be saved to check if its available ?