compass.Step

class compass.Step(test_case, name, subdir=None, cpus_per_task=1, min_cpus_per_task=1, ntasks=1, min_tasks=1, openmp_threads=1, max_memory=None, cached=False, run_as_subprocess=False)

The base class for a step of a test cases, such as setting up a mesh, creating an initial condition, or running the MPAS core forward in time. The step is the smallest unit of work in compass that can be run on its own by a user, though users will typically run full test cases or test suites.

Below, the terms “input” and “output” refer to inputs and outputs to the step itself, not necessarily the MPAS model. In fact, the MPAS model itself is often an input to the step.

Variables:

• name (str) – the name of the test case

• test_case (compass.TestCase) – The test case this step belongs to

• test_group (compass.TestGroup) – The test group the test case belongs to

• mpas_core (compass.MpasCore) – The MPAS core the test group belongs to

• subdir (str) – the subdirectory for the step

• path (str) – the path within the base work directory of the step, made up of mpas_core, test_group, the test case’s subdir and the step’s subdir

• cpus_per_task (int) – the number of cores per task the step would ideally use. If fewer cores per node are available on the system, the step will run on all available cores as long as this is not below min_cpus_per_task

• min_cpus_per_task (int) – the number of cores per task the step requires. If the system has fewer than this number of cores per node, the step will fail

• ntasks (int) – the number of tasks the step would ideally use. If too few cores are available on the system to accommodate the number of tasks and the number of cores per task, the step will run on fewer tasks as long as as this is not below min_tasks

• min_tasks (int) – the number of tasks the step requires. If the system has too few cores to accommodate the number of tasks and cores per task, the step will fail

• openmp_threads (int) – the number of OpenMP threads to use

• max_memory (int) – the amount of memory that the step is allowed to use in MB. This is currently just a placeholder for later use with task parallelism

• input_data (list of dict) – a list of dict used to define input files typically to be downloaded to a database and/or symlinked in the work directory

• inputs (list of str) – a list of absolute paths of input files produced from input_data as part of setting up the step. These input files must all exist at run time or the step will raise an exception

• outputs (list of str) – a list of absolute paths of output files produced by this step (or cached) and available as inputs to other test cases and steps. These files must exist after the test has run or an exception will be raised

• namelist_data (dict) – a dictionary used internally to keep track of updates to the default namelist options from calls to compass.Step.add_namelist_file() and compass.Step.add_namelist_options()

• streams_data (dict) – a dictionary used internally to keep track of updates to the default streams from calls to compass.Step.add_streams_file()

• config (compass.config.CompassConfigParser) – Configuration options for this test case, a combination of the defaults for the machine, core and configuration

• machine_info (mache.MachineInfo) – Information about E3SM supported machines

• config_filename (str) – The local name of the config file that config has been written to during setup and read from during run

• work_dir (str) – The step’s work directory, defined during setup as the combination of base_work_dir and path

• base_work_dir (str) – The base work directory

• logger (logging.Logger) – A logger for output from the step

• log_filename (str) – At run time, the name of a log file where output/errors from the step are being logged, or None if output is to stdout/stderr

• cached (bool) – Whether to get all of the outputs for the step from the database of cached outputs for this MPAS core

• run_as_subprocess (bool) – Whether to run this step as a subprocess, rather than just running it directly from the test case. It is useful to run a step as a subprocess if there is not a good way to redirect output to a log file (e.g. if the step calls external code that, in turn, calls additional subprocesses).

__init__(test_case, name, subdir=None, cpus_per_task=1, min_cpus_per_task=1, ntasks=1, min_tasks=1, openmp_threads=1, max_memory=None, cached=False, run_as_subprocess=False)

Create a new test case

Parameters:

• test_case (compass.TestCase) – The test case this step belongs to

• name (str) – the name of the test case

• subdir (str, optional) – the subdirectory for the step. The default is name

• cpus_per_task (int, optional) – the number of cores per task the step would ideally use. If fewer cores per node are available on the system, the step will run on all available cores as long as this is not below min_cpus_per_task

• min_cpus_per_task (int, optional) – the number of cores per task the step requires. If the system has fewer than this number of cores per node, the step will fail

• ntasks (int, optional) – the number of tasks the step would ideally use. If too few cores are available on the system to accommodate the number of tasks and the number of cores per task, the step will run on fewer tasks as long as as this is not below min_tasks

• min_tasks (int, optional) – the number of tasks the step requires. If the system has too few cores to accommodate the number of tasks and cores per task, the step will fail

• openmp_threads (int) – the number of OpenMP threads to use

• max_memory (int, optional) – the amount of memory that the step is allowed to use in MB. This is currently just a placeholder for later use with task parallelism

• cached (bool, optional) – Whether to get all of the outputs for the step from the database of cached outputs for this MPAS core

• run_as_subprocess (bool) – Whether to run this step as a subprocess, rather than just running it directly from the test case. It is useful to run a step as a subprocess if there is not a good way to redirect output to a log file (e.g. if the step calls external code that, in turn, calls additional subprocesses).

Methods

__init__(test_case, name[, subdir, ...])	Create a new test case
add_input_file([filename, target, database, ...])	Add an input file to the step (but not necessarily to the MPAS model).
add_model_as_input()	make a link to the model executable and add it to the inputs
add_namelist_file(package, namelist[, ...])	Add a file with updates to namelist options to the step to be parsed when generating a complete namelist file if and when the step gets set up.
add_namelist_options(options[, out_name, mode])	Add the namelist replacements to be parsed when generating a namelist file if and when the step gets set up.
add_output_file(filename)	Add the output file that must be produced by this step and may be made available as an input to steps, perhaps in other test cases.
add_streams_file(package, streams[, ...])	Add a streams file to the step to be parsed when generating a complete streams file if and when the step gets set up.
constrain_resources(available_resources)	Constrain cpus_per_task and ntasks based on the number of cores available to this step
process_inputs_and_outputs()	Process the inputs to and outputs from a step added with compass.Step.add_input_file() and compass.Step.add_output_file().
run()	Run the step.
runtime_setup()	Update attributes of the step at runtime before calling the run() method.
set_resources([cpus_per_task, ...])	Update the resources for the subtask.
setup()	Set up the test case in the work directory, including downloading any dependencies.
update_namelist_at_runtime(options[, out_name])	Update an existing namelist file with additional options.
update_namelist_pio([out_name])	Modify the namelist so the number of PIO tasks and the stride between them consistent with the number of nodes and cores (one PIO task per node).
update_streams_at_runtime(package, streams, ...)	Update the streams files during the run phase of this step using the given template and replacements.