Implement in new study area

[sinkan851014]

Hi, I want to implement Silo intergrated MATSim in new area, Taoyuan City, Taiwan for my research rigion.

I have prepared data for input, such as network, geodata, populations.

So, should I replace java class file from Muc( imformation form https://wiki.tum.de/display/msmmodels/Implementation+for+a+New+Study+Area)or Maryland would be better?

Also want to know if where is any input data like maryland or other example area and could be replaced step by step to implement in my study area?

Thanks

Cooper

[moeckel]

@cllorca1 : In the past, I would have argued that the Maryland implementation is easier to start with. However, we have not worked with Maryland for a while, and I wonder if this is still fully operational with all recent changes in code. Could you advise whether it will be easier to start with a Maryland or a Munich implementation? I also would appreciate if you could share the input data accordingly with Cooper.

[cllorca1]

@sinkan851014, you can start running SILO using the test implementations. To do so, you just need to clone the repository from Github and run, for instance, the class https://github.com/msmobility/silo/blob/6b6b159ca34822960f8d3bd48a4dea4e35934df7/useCases/maryland/src/test/java/de/tum/bgu/msm/transportModel/matsim/SiloMatsimMultiYearTest.java

This class runs a small version of SILO-MATSim. All the input files are located in the repository. I think this can be the best option to start, as the runtime is very short. From there, you can start creating your input data and finally, your own implementation following the steps of https://wiki.tum.de/display/msmmodels/Implementation+for+a+New+Study+Area. Please come back to us if there is any issue.

[sinkan851014]

@cllorca1 @moeckel
Hi, I run class SiloMatsimMultiYearTest.java sucessfully!
Now I write a class and put the same input data.(.properties, config.xml, and others)

public class Run {
    private final static Logger logger = Logger.getLogger(SiloMatsim.class);
    public static void main(String[] args) {
        String arg = "/silo/useCases/maryland/test/scenarios/annapolis/javaFiles/siloMatsim.properties";
        Config config = ConfigUtils.loadConfig("/silo/useCases/maryland/test/scenarios/annapolis/matsim_input/config.xml");
        SiloMatsim siloMatsim = new SiloMatsim(arg, config);
        siloMatsim.run();
        System.out.println("DONE!");
    }
}

There are four issue now

1. It seems work well but I am not sure if this is a good way to implement in new study area.

2. In addition, I found the output of MATSim was weird.
   For example, the "2001.output_events.xml.gz", as I know the event.xml file would record all agent's event but the file was emtpy. As I guess, I should write down my own siloMatsim.properties referred to “https://wiki.tum.de/display/msmmodels/Properties”

3. However, some key value in the defult siloMatsim.properties (https://github.com/msmobility/silo/blob/master/useCases/maryland/test/scenarios/annapolis/javaFiles/siloMatsim.properties) couldn't be found in the web, so I should find another technical documents to know this value or just download "full settings"(https://wiki.tum.de/download/attachments/214206490/fullProperties.properties?version=1&modificationDate=1559285519093&api=v2) and write file as I need?

4. I had prepared Data for Synthetic population, since code of maryland need PUMS for SP, so Munich(SyntheticPopulationGeneratorDe) may be a better option?
   For SP input.properties, if there are any document describe setting?

Thanks!
Cooper

[cllorca1]

Hi, what you wrote should replicate the SiloMatsim test... Not sure why your MATSim output is empty... Could you store and share the console output (log) of your run? That would be required to asses questions 1, 2, and 3... Esentially the code inside SiloMatsim.run() is something like this:

logger.info("Starting SILO program for MATSim");
        DataContainer dataContainer = DataBuilder.buildDataContainer(properties, matsimConfig);
        DataBuilder.readInput(properties, dataContainer);
        ModelContainer modelContainer = ModelBuilderMstm.getModelContainerForMstm(dataContainer, properties, matsimConfig);
        ResultsMonitor resultsMonitor = new DefaultResultsMonitor(dataContainer, properties);
        SiloModel model = new SiloModel(properties, dataContainer, modelContainer);
        model.addResultMonitor(resultsMonitor);
        model.runModel();
        logger.info("Finished SILO.");

which you can also find in https://github.com/msmobility/silo/blob/master/useCases/munich/src/main/java/de/tum/bgu/msm/SiloMuc.java

Regarding the synthetic population generator (point 4), you can use any method of your preference (perhaps outside of silo). The population files have to be consistent with the specifications in https://wiki.tum.de/display/msmmodels/Input+files+SILO under microdata. The Maryland metod uses PUMS which probably are open and you can download (but from US), the Munich mehtod is based on census data which is not open at all (some more details at https://wiki.tum.de/display/msmmodels/Synthetic+population). You'll need to figure out what is your microdata from Taiwan and how to generate the population from it. It is very likely that most of the code of this module is not useful, since it is strongly adapted to each data source...

You can fork our repository and make edits there and push them to your forked repo, if you want us to have some look at the code...

Let us know any question.

[cllorca1]

I see you forked silo already. Make sure to update the code regularly from msmobility/silo (master) so things keep up to date!

[sinkan851014]

Hi, I solve problem 2 from finding change "matsim.scale.factor " to "1 and "2001.output_events.xml.gz" became normal!

I found that the data from my research area may not be available for the Silo model, so now I wanted to use only the synthetic population and simulate a single year using MATSim as an alternative.

I would like to replace my data based on the synthetic population code for Munich, since only some data such as place of work and place of residence are needed if only the traffic scenario is simulated using MATSim without simulating years of real estate development.

However, when I tried to run the synthetic population with the existing data used for testing in Silo, I was confused by some error messages, and here are my test results and input data.
In the case, since the original tdTest.omx file would have an error when reading, I used HwyPK_iter6.omax as a replacement, I wonder if this also caused the problem?

In addition, according to the website, Silo needs certain control properties when synthesizing the population, can these contents be customized according to the input properties, or do I need to change the code myself?

Finally, synthesizing the population requires distance by car OMX files as input data, which makes me confused because how can I run MATSim to generate OMX files without synthesizing the population?

Thanks!

Cooper

[atmoreno]

@sinkan851014 The synthetic population generator in SILO is tailored to each input data format. Each implementation has its own control properties with specific format, spatial resolution and requirements.

You would need to define, for your study area, which are your control totals (aggregate totals that you would like to match, usually from census data), the micro data (individual households and persons that the code would replicate according to weights), the zone system (zones with target population density), and distance between zones (e.g. car OMX files or beeline distance between centroids). All this data must be prepared before running the synthetic population generation in SILO.

In the input properties, you specify, among others, the path to input files and which attributes you would like to match. However, the code needs to be changed yourself to read the appropriate variables in control totals and micro data, recode them internally to the categories that you define in the control totals and to generate the "frequency matrix". After that point, all implementations are homogenized and use the same methods internally.

But if you don't need to execute SILO, the SILO synthetic population generator may be too cumbersome to generate a simpler synthetic population for MATSim that only requires persons and trips, and not all objects generated by SILO (persons, households, dwellings, jobs and schools). Further, SILO does not generate travel demand and you would need to use another model afterwards.

[sinkan851014]

@atmoreno
Sorry for the misunderstanding, I still need to run Silo, just for one year only, e.g. 2022-2023 to get the data from MATSim.
Now I understand that the synthetic population needs to control the attributes, can you please provide the SP data for testing purposes? The Munich_new data and .properties in the test folder seems to be incorrect and there are many errors when using it for testing.

Thanks,
Cooper