adds JSON config file generation examples#30
adds JSON config file generation examples#30alvaradoo wants to merge 1 commit intosstsimulator:masterfrom
Conversation
Signed-off-by: Oliver Alvarado Rodriguez <oliver.alvarado-rodriguez@hpe.com>
|
The above relies on the following PR to sst-core to be approved & merged: sstsimulator/sst-core#1584. |
gvoskuilen
left a comment
There was a problem hiding this comment.
Edit the description for parallel-loading so it's clear that parallel-loading is independent of running with MPI and that you can generate JSON files for both serial/parallel loading under MPI without having a python file that itself is parallel-loaded.
|
|
||
| The generated `test_StatisticsComponent_basic.json` corresponds to [JSON Configuration Example 2](#json-configuration-example-2) below. | ||
|
|
||
| ### Multi-Rank Examples |
There was a problem hiding this comment.
I think this should be edited to be clear that there are two distinct capabilities being described:
- multi-rank sst simulations can load JSON files (
mpirun -np 2 sst input.json). The file is loaded on rank 0 and the graph partitioned/distributed to the rest of the ranks. - JSON files can be parallel loaded
(mpirun -np 2 sst --parallel-load=MULTI input.json). Each rank loads only the file for its rank.
When generating JSON files (serial or parallel), the --parallel-load flag isn't needed unless the input file you are generating from needs to be loaded in parallel.
When running generated JSON files, "serial" files can be run in serial or parallel simulations while "parallel" files can only be run in parallel simulations whose parallelism matches the parallelism that the files were generated with.
I put some ideas below for clarifying but open to suggestions.
|
|
||
| #### test_ParallelLoad.py | ||
|
|
||
| For simulations that use multiple MPI ranks, JSON files can be generated using `mpirun` with the `--parallel-load=SINGLE` option. This ensures all ranks process the same Python configuration file and the resulting JSON captures the full partitioned configuration graph. |
There was a problem hiding this comment.
"For simulations that use multiple MPI ranks, inputs can either be loaded on rank 0 and distributed by SST (default) or, if correctly configured, each rank can directly load the relevant part of the input (parallel loading). To generate files for the first case, use the method above for generating a single JSON input file and run in parallel. To generate multiple JSON files for parallel loading, do the following..."
Then examples for
- MPI/serial load
# Generate
sst --run-mode=init --json-output=test_MessageMesh.json test_MessageMesh.py -- 6 6
# Run in parallel
mpirun -np 2 sst test_MessageMesh.json
- MPI/parallel load w/ multiple input files
# Generate
mpirun -np 2 sst --run-mode=init --json-output=test_MessageMesh.json --parallel-output test_MessageMesh.py -- 6 6
# Run in parallel
mpirun -np 2 sst --parallel-load test_MessageMesh.json
- I need to double check but I believe that MPI/parallel load w/ single input file only works for Python inputs since the input has to programmatically excludes parts of the config depending which rank it is loaded on.
--parallel-load=SINGLE is only used with test_ParallelLoad.py because it is set up to be able to be loaded by each rank in parallel. In general, that flag is not needed for generating either serial or parallel-load JSON outputs.
Adds examples to https://sst-simulator.org/sst-docs/docs/guides/configuration/jsonConfigGuide that shows how users can create JSON configuration files. Uses scripts at
sst-docs/testsfor the examples, since the original documentation showed the JSON configuration files fortest_Links.pyandtest_StatisticsComponent_basic.py.I added an extra example with
test_ParallelLoad.pyto show how to output a configuration file per rank. However, I ran into the following error with this:I created a workaround to this by changing all the instances of
"port"in theaddLinkcalls to"port0". This is probably not the right fix, but wanted to double check to see what the best way to add a json-per-rank output example would be.Before:
No examples.
After:
