Commit 3fd99c8
authored
Add options to store files directly to the file system and to omit duplicating files in memory (#257)
* Add options to store files directly to the file system and to omit duplicating files in memory
* Delete accidentially added newline in http_resource.hpp
* Removed unused function, fixed some code style and typos in comments
* Removed asprintf in favor or string concatenation
As asprintf is a gnu extension it should not be used in a multi plattform code.
To generate the needed string, std::string is used
As the finally called function mkstemp changes its first argument of type char *
the concatenated string is still strdup-ed into a char * before handing it to mkstemp
* Made http_utils::generate_random_upload_filename plattform independant
Mkstemp is only available on windows. As there is no suitable plattform independant function
to generate a unique filename within a specified directory, use mkstemp on linux and a
combination of _mktemp_s and _sopen_s on windows.
Furthermore the path separator differs on linux and windows so use a static const char
which provides the appropriate path separator
Additionally within this patch the function http_utils::generate_random_upload_filename
uses an exception to reports errors rather than a empty string as return code
* Input and output of http_utils::generate_random_upload_filename are const
* Added options and functions for file upload to README.md
* Use correct include and functions in generate_random_upload_filename to run on windows
The current code won't compile on ApVeyor. So this is a an untested patch to include
the correct headers and not use functions, which are not supported (or deprecated)
on windows.
* Add missing includes for windows, fixed wrong argument for _mktemp_s
* Removed accidentially pushed debug code
* Use a static const char* for the random file system template
This is only used inside http_utils.cpp and therefor can be static
* Make output of get_or_create_file_info const and introduce new setters
The output of get_or_create_file_info should be const.
As previously the result was used to direclty alter the content of the struct
new setters to set the file_system_file_name and to grow the size
are introduced (and used)
* Use const variables for the webserver options
* Added an example for file upload
file_upload.cpp is a very basic example for a file upload server.
It features a simple HTML input form with file upload (which supports multiple files).
The uploaded files will be stored to the directory, which has to be given as
command line parameter to the example program.
All uploaded files will be printed in a table including their original name,
the generated random filepath and the size.
* Use a prefixed operator in for loop
* Added safety check for use of strlen and some missing free()s
Static code analysis complained about use of unprotected strlen.
This changes adds a safety check directly before the us of strlen.
Additionally some free()s (before some throws) where missing.
* Changed file map to be a map of keys with a nested map of files
There is the possibility to upload the same file with two different keys
or to upload multiple (different) files with the same key within the
same request.
The current solution can only handle the second use case but not the first.
This patch changes the file map to use a nested map of files within
a map of keys to support both use cases.
* Adjusted file_upload example to use new files map
* Updated description of files map in README.md
* Removed strlen from generate_random_upload_filename
As Codacy static code analysis warns about the safety issue of strlen
if a given string is not terminated, this is removed.
The solution uses the size of the original string, which is just duplicated
to a char* as the used function _mktemp_s requires a char* which will
be changed by the function and the changed result is needed afterwards.
* Use a pointer to std::ofstream in modded_request
One of the workflows did not run successfully, as it is obviously not allowed
to have a std::ofstream in a class if it is not a pointer or reference.
This patch uses a pointer to the ofstream in the modded requests and
adjusts all usage of the ofstream in webserver.cpp
* Moved struct file_info_s to class file_info
* Setters of class file_info are private and class webserver is a friend of this class
As the file_info has to be changed (especially the file_size) in multiple iterations
of the post iterator, this class cannot be fully immutable.
To realise a interface without setters to the outside of the library these
setters are private now. The only other classe, which uses these setters
is the class webserver and is therefor declared as friend to the file_info class.
* Use const reference as parameter for set_file_system_file_name
* Don't create zero length files if no file is uploaded
If an upload field is submitted without a file selected, the previous
implementation would still create a random file with zero length.
This is due to the fact, that the key would be present in the content,
but the filename is empty.
So a simple check, whether the filename is really present is introduced.
This is done by simply looking at the very first sign of the const char*
(which is given from libmicrohttpd). As strlen is considered unsafe and
there is already a check, whether the const char* is not a nullptr, it is
always safe to dereference and inspect the first char.
* Updated README.md as file_info is now a class
* Some code styling and small refactorings after review
* Replaced std::exception with custom generateFilenameException
Although there is no distinct error handling, if the filename generation
throws an exception, it should be usable in the future and destinguishable
from other common exceptions. Therefor the std::exception was replaced by
generateFilenameException
* Some more comments and code style fixes after review
* Some more code style changes after review
* Added error string to generateFilenameException
Though it is not used at the moment the generateFilenameException
features now a error string, which is set and can be used for debugging.
* Added comment for c functions in generate_random_upload_filename
Actually we should use standard C++ classes like ofstream for file operations.
But as these do not feature some wanted functions, this is done by C functions.
This patch adds some comments to explain the reasons for this decision.
* Removed const qualifier from file_info::get_file_size()
This is removed as the const qualifier is superfluous and generates
warning on pedantic checks.
* Added unit test for generate_random_upload_filename
This is a first show on a unit test for the function generate_random_upload_filename.
After calling the function, this unit test not only checks the returned filename,
but also, whether the file exists.
Maybe to be improved:
- the filename template and the path_separator are currently duplicated
- should there be a check, whether the removal of the temporary file worked?
* Removed filesystem include from unit tests again, as it would require C++17
* Close open upload file after post processor is done
If the file is not closed after the post processor is done, the
last info of the file might not be written to disk until the calling
application accesses the file. This could easily be reproduceable with
uploading very small files ( < 100 Bytes). In this case the content
of the file was missing for the calling process.
* Added content_type and transfer_encoding to file_info
The post iterator can set the content_type and the transfer_encoding
of the uploaded file into the file_info.
According to the documentation of libmicrohttpd this is sometimes unknown.
In this case the values will be nullptr and therefor the file_info
may be empty on this values.
* Fixed file_upload example
* Made filename_template and path_separator part of class http_utils
As these two constant values are used in the unit tests they could
no longer be static only in http_utils.cpp.
So they are members of the class http_utils and therefor accessible
in the unit tests
* Added integration test for file upload
Added an integration test for file upload which tests the upload
with different settings of the webserver:
- Upload to memory and disk
- Upload to disk only
- Upload to memory, duplicated in get_content and args
- Upload to memory, only stored in args
* Fixed integration test for file upload
* Removed empty AUTO_TEST from file_upload integration test
* Use internal values for http_ressource class in integration tests
Avoid using pointers in class print_file_upload_resource in favour of
internal values and according getter functions.
Additionally simplified the code by wrapping lines and using
the auto keyword instead of long map specifiers.
* Added further integration tests for file upload
Added the following integration tests
- Upload of a file via PUT (then there is no post_processor used
and the content of the file will only be available via get_content
- Upload of two files
- Upload of a file and an additional ordinary POST parameter
* Added test_content_2 to configure script
As the test_content_2 was missing in the configure script the
file was not copy to the build/test directory and therefor the
integration tests failed
* Fixed memory leak in post_iterator
If multiple files are uploaded, the ofstream to write the data to the file
was created again (with new()) for the next file without deleting the
previously created one.
This resulted in a memory leak.
* Some code style fixed on file upload integration test
* Use map definition instead of auto in integration test
Co-authored-by: Christian Tomahogh <ctomahogh@users.noreply.github.com>1 parent bd559af commit 3fd99c8
File tree
20 files changed
+978
-13
lines changed- examples
- src
- httpserver
- details
- test
- integ
- unit
20 files changed
+978
-13
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
190 | 190 | | |
191 | 191 | | |
192 | 192 | | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
193 | 200 | | |
194 | 201 | | |
195 | 202 | | |
| |||
553 | 560 | | |
554 | 561 | | |
555 | 562 | | |
| 563 | + | |
| 564 | + | |
556 | 565 | | |
557 | 566 | | |
558 | 567 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
273 | 273 | | |
274 | 274 | | |
275 | 275 | | |
| 276 | + | |
276 | 277 | | |
277 | 278 | | |
278 | 279 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
22 | | - | |
| 22 | + | |
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
| |||
41 | 41 | | |
42 | 42 | | |
43 | 43 | | |
| 44 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
22 | | - | |
| 22 | + | |
23 | 23 | | |
24 | | - | |
| 24 | + | |
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
127 | 127 | | |
128 | 128 | | |
129 | 129 | | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
130 | 134 | | |
131 | 135 | | |
132 | 136 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
26 | 30 | | |
27 | 31 | | |
28 | 32 | | |
| |||
33 | 37 | | |
34 | 38 | | |
35 | 39 | | |
| 40 | + | |
36 | 41 | | |
37 | 42 | | |
38 | 43 | | |
| |||
198 | 203 | | |
199 | 204 | | |
200 | 205 | | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
201 | 214 | | |
202 | 215 | | |
203 | 216 | | |
| |||
221 | 234 | | |
222 | 235 | | |
223 | 236 | | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
224 | 276 | | |
225 | 277 | | |
226 | 278 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
| 34 | + | |
34 | 35 | | |
35 | 36 | | |
36 | 37 | | |
| |||
0 commit comments