Running the IMI on a local cluster
The IMI is setup to run on AWS by default. However, if you have a local cluster available to you, you may choose to run the IMI there. This option requires some manual changes and is therefore only recommended for advanced users.
You must first ensure you have the proper hardware and software requirements for running GEOS-Chem.
When logged onto your local cluster, navigate to the path where you want to download the IMI repository and type the following command:
$ git clone https://github.com/geoschem/integrated_methane_inversion.git
This will clone the IMI code into a local folder named integrated methane_inversion
.
Tip
If you wish, you can clone the IMI repository into a different local folder by supplying the name of the folder at the end of the git clone command. For example:
git clone https://github.com/geoschem/integrated_methane_inversion.git imi-v1.0
Next, download the GEOS-Chem source code and its submodules within the IMI folder using these commands:
$ cd integrated_methane_inversion
$ git clone https://github.com/geoschem/GCClassic.git
$ cd GCClassic
$ git submodule update --init --recursive
See Downloading rhe GEOS-Chem source code for more details.
Navigate back to the top-level IMI folder and view the contents:
$ cd ..
$ ls
config.yml envs/ LICENSE.md resources/ setup_imi.sh*
docs/ GCClassic/ README.md run_imi.sh* src/
Within the IMI is a subfolder called envs
that constains files for
running the IMI on different systems. By default, files are provided
for AWS and Harvard’s Cannon cluster.
$ ls envs/*
envs/aws:
conda_env.yml slurm/ spack_env.env
envs/Harvard-Cannon:
ch4_inv.yml gcc.gfortran10.2_cannon.env* gcc.ifort19_cannon.env*
config.harvard-cannon.yml gcc.ifort17_cannon.env* README
We recommend you add a subfolder within envs
for your own system
to easily access your customized files needed for the IMI. In this
directory, we recommend storing any environment files needed to load
the libraries for GEOS-Chem (e.g. fortran compiler, netcdf, openmpi,
cmake), a conda environment file, and a copy of the IMI configuration file
modified for your system. See the files in envs/Harvard-Cannon
for examples.
Within the copied IMI configuration file, you will need to modify the
settings in the section labeled “Settings for running on your local
cluster.” If you already have the GEOS-Chem input data on your system,
you may set the *DryRun
options to false
.
It is recommended that you set up and run the IMI in stages when
running on a local cluster to ensure that each stage works
properly. You can do this by modifying the settings under “Setup
modules” and “Run modules” and turning them on one or a few at a
time. You may find that you need to manually edit some files. For
example, after creating the template run directory, but before
creating your spinup, Jacobian, and posterior run directories, you should open
ch4_run.template
in a text editor and modify as needed for your
system (by default this script is set up to submit to a SLURM
scheduler).
Once your have finished customizing the IMI settings for your cluster,
you can run the IMI by executing run_imi.sh
and passing an
argument for the location of your IMI configuration file. For example:
$ ./run_imi.sh config.harvard-cannon.yml
If you do not pass a configuration file, config.yml
in
the top-level IMI directory will be used. That file is set up for
running the IMI on AWS by default.