Update soap docs

[robjmcgibbon]

Some updates to the SOAP documentation. Current version can be found at https://home.strw.leidenuniv.nl/~mcgibbon/flamingo-docs/html/soap/property_table.html

The main change is related to SWIFTSIM/SOAP#205. I felt the property table we have in the SOAP pdf is too large to copy across directly, so I have tried to condense it by hiding some of the less important fields. I still need to work on the text, and clean up some of the footnotes, but just wondering if you think hiding these fields is ok @jchelly ?

[jchelly]

I agree that the original table is too large. When you say fields, do you mean dropping halo properties or columns from the table?

I like the dataset specifications drop down. Would it work if the name in the first column was the drop down, to avoid repeating "dataset specifications" on every row? I'm not sure if that would be confusing but it would save space.

[robjmcgibbon]

When you say fields, do you mean dropping halo properties or columns from the table?

Sorry I wasn't clear, by fields I mean columns in the table (e.g. the ones I've hidden).

Would` it work if the name in the first column was the drop down, to avoid repeating "dataset specifications" on every row? I'm not sure if that would be confusing but it would save space.

I've moved the dropdown, I agree it was a lot of unnecessary repetition. I don't think it will be confusing, I can mention it in the table preamble.

[robjmcgibbon]

I still need to work on the text, but I'm going to post the current version on the data release channel to see if anyone has comments on the layout.

[robjmcgibbon]

Joey and Joop pointed out that they can't view the entire table since it's hidden under the contents bar (although it is possible to just hide the contents bar). I don't think this is specific to the soap table, it's just that the snapshot properties table is narrow enough that this doesn't become an issue.

e1ad44a moves the page contents to the LHS. My understanding is that the piccolo theme does not allow you to disable the RHS page contents bar, so I have just hidden it with some custom css. This seems slightly hacky, but I'm not familiar enough with sphinx docs to know if there's a better way.

[jchelly]

I'm not very happy with the handling of page width in any theme I've tried. I think it makes sense to limit the width of body text but for tables and figures it would be nice to be able to use all of the available width. I haven't found any way to do that and even just adjusting the width limit takes theme-specific hacks. Switching themes probably isn't a huge job if there's a better one.

[robjmcgibbon]

After a brief look at other themes I've not come up with anything better. I'd vote we stay with this, I like how it looks.

[robjmcgibbon]

Update on the status of this

• The main description and the property table should be complete
• Are we having a separate section for all the different example scripts, or should we put the soap examples within the soap directory? I think I prefer a separate section for examples, as some will span multiple data products
• I have removed the description of the membership files from the soap section, in anticipation of listing them in the particle property table instead. I still need to create a virtual file that links the snapshots, memberships, and recalculated xrays together, but I can't do that until I have rerun the xrays.
• I need to add some information on the matching files, and a basic script for that

A couple of notes about the data (not all related to this PR)

• It doesn't look like the service is currently configured to serve the HBT directory for each run.
• I also just remembered we would like to share snapshots_downsampled and snapshots_reduced_HBT
• The matching files match each hydro run to its dmo counterpart. I propose placing the matching files in the SOAP-HBT subdirectory of the hydro run

[jchelly]

So far I've put examples in each section, but we could move them to a separate examples section and add links from the descriptions of the data products.

For the virtual snapshots, we have two versions on /cosma8: one in the snapshots directory that just joins all of the sub-files and another in SOAP-HBT that adds in group info. For the data release I think we just want one virtual snapshot that includes everything. We might need to be careful where we put it because any relative paths will need to work with the real paths on Cosma and with the paths presented by the web service.

I can configure the service to serve the extra snapshots. I guess those just need a page explaining what subset of the particles they contain. Then for dataset descriptions and how to read them we can link to the full snapshot documentation.

Would we want to serve the full HBT outputs, or just (maybe sorted) catalogues? If we serve the full outputs we're committing to keeping them on disk.

[robjmcgibbon]

We might need to be careful where we put it because any relative paths will need to work with the real paths on Cosma and with the paths presented by the web service.

Good point, I think we need to remember to highlight this somewhere in the documentation

I can configure the service to serve the extra snapshots. I guess those just need a page explaining what subset of the particles they contain. Then for dataset descriptions and how to read them we can link to the full snapshot documentation.

Thanks, I can write an short page for both of them

Would we want to serve the full HBT outputs, or just (maybe sorted) catalogues? If we serve the full outputs we're committing to keeping them on disk.

I think it's better to go with the sorted catalogues, the feedback I have got from some students here is that they find them a lot easier to work with. One downside would be that the HaloCatalogueIndex in SOAP will no longer match up, but people can match on TrackId instead. I can start some jobs to generate the sorted catalogues.