Skip to content

Documentation: Clarify multi-processing setup; provide MP run guidance without editing script #205

@ghost

Description

Setting up both the multi-processing configuration settings file and running populationsim such that it uses the multi-processing settings is difficult. I think there would be benefit to a readme on how to manipulate the run_populationsim.py file.

  • I was ultimately able to get multi-processing to work for NCTCOG by launching the scenario via CLI.
  • My initial understanding of the run_populationsim.py script was that, if you had your scenario folder structure and settings files set up correctly, this script would run populationsim using all of the information provided. The way run_populationsim.py currently works is by initializing all of the arguments added to the parser as ‘None’ except for the fast and multiprocess arguments which have default values of False. Then, the working directory is used to inject the required arguments based on expected folder naming convention (configs, data, output). It is also set up to read the configs_mp folder by default but only if the multiprocess argument is True. The point I’m trying to make is this:
    When you set up a scenario for single processing, it just works. So, it sets the expectation that if you set up a scenario for multi-processing, it will also just work. In reality, you need to manually update the run_populationsim.py with a line that sets multiprocess to True in the args variable.

I think the easiest way address this is to include a run_populationsim_mp.py script in the example_calm folder that has this additional line setting the multiprocess argument to True (and maybe the output to ‘output_mp’ as well). I think the current way example_calm is set up is unintentionally misleading regarding running populationsim with multi-processing. Alternatively, the argument/input handling could be modified to flip multiprocess to True when the configs_mp folder exists in the folder structure. However, this option makes it harder for a user to set up single and multiprocessing to both work within a single scenario folder.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions