lib: fix reading GRASS vrts with multiple threads#7201
lib: fix reading GRASS vrts with multiple threads#7201metzm wants to merge 4 commits intoOSGeo:mainfrom
Conversation
There was a problem hiding this comment.
This is great. I think this is the direction we should go for these functions at this point. I'm assuming it works although I did not test this.
There is still a very slight risk of a race accessing the fileinfo to get vrt, so I suggest modifying the code to account for that. We already know that raster open/close is the problem, so I'm suggesting changes aligned with that.
The rule I'm using is that everything accessing fileinfo is in danger. For standard rasters, we avoid the issue by opening outside of parallel code and having per-thread file descriptors. This is my reason adding one more critical block.
What this does not fix is a combination of vrt rasters and standard rasters being read at the same time, but that's outside of scope of this PR. (My rule would apply to it, so I'm mentioning it for context.)
Additionally, I suggest naming the critical block (raster_vrt_read). This will allow other unrelated parallel code with a critical block to run in parallel (r.univar has one).
Co-authored-by: Vaclav Petras <wenzeslaus@gmail.com>
|
Thanks for your suggestion to also protect |
wenzeslaus
left a comment
There was a problem hiding this comment.
Thanks for the update. I think this is all we can do here.
I actually meant to replace the whole second comment with a implementation perspective comment (replacing the encountered bug perspective), but even this can be merged as is.
Description
Parallelised reading of the real raster maps constituting a GRASS virtual raster causes IO read errors and segmentation faults. This PR enforces reading of the different rasters in only one thread.
Someone with better knowledge of OpenMP directives than me should have a look at this PR. At least for me this PR fixes the read errors.
How to reproduce
r.tiler.buildvrtr.univaron this vrt with nprocs > 1This also fixes #6616