Output files and directories

This page describes the contents of various files and directories generated by the IMI in the course of an inversion. These files are created in {OutputPath}/{YourRunName}/.

When the IMI has completed successfully, you should see the following files and directories in your IMI output path:

$ cd {OutputPath}/{YourRunName}/  # Replace with your path and run name
$ ls
archive_perturbation_sfs/  inversion/                  satellite_data@
cleanup_script.sh          jacobian_lowbg_ics_bcs/     spinup_run/
config_YourRunName.yml     jacobian_runs/              StateVector.nc
GEOSChem_build/            make_state_vector_file.py*  template_run/
hemco_prior_emis/          posterior_run/
imi_output.log             preview/

These directories and files are summarized below:

`archive_perturbation_sfs/`	Directory containing gridded perturbation scale factor files. If using the Kalman filter option, the period number will be included in the file name.
`cleanup_script.sh`	Bash script to cleanup the directories and files created by the IMI workflow.
`config_YourRunName.yml`	A copy of the configuration file used to run the IMI.
`GEOSChem_build/`	Contains the GEOS-Chem executable file and a summary of settings used to build GEOS-Chem.
`hemco_prior_emissions/`	HEMCO standalone run directory used to generate gridded prior emissions files for use in the IMI preview and in the Jacobian runs.
`imi_output.log`	Text file containing the output of print statements to a file. You can check the log files to see how your run has progressed and to check for any error messages.
`inversion/`	Directory containing inversion scripts, processed model output, and inversion results. The contents of this directory are described further in the next section.
`jacobian_lowbg_ics_bcs/`	Directory containing restart and boundary condition files with the species concentration to 1 mixing ratio unit for use in the Jacobian runs.
`jacobian_runs/`	Directory containing GEOS-Chem run directories for each state vector element. State vector elements may also be grouped into fewer GEOS-Chem simulations using the `NumJacobianTracers` setting in `config.yml`.
`make_state_vector_file.py`	Python script used to generate the StateVector.nc` file.
`posterior_run/`	GEOS-Chem run directory for running a simulation applying the scale factors generated by the inversion.
`preview/`	Directory containing images and textual output from the IMI preview.
`satellite_data`	Symbolic link to the satellite data used in the inversion.
`spinup_run/`	GEOS-Chem run directory for generating spun-up initial conditions for the Jacobian runs.
`StateVector.nc`	Gridded netCDF file containing the 2D elements to be optimized in the inversion.
`template_run/`	GEOS-Chem template run directory used to generate `spinup_run`, `jacobian_run`. and `posterior_run`.

Inversion directory

The inversion/ directory is where the IMI computes the Jacobian, obtains the optimal estimate of emissions, and saves the results.

In addition to a shell script and several Python scripts used in the inversion, you will find the following items in the inversion directory after completing an inversion:

`data_converted/`	Directory of Python pickle (`.pkl`) files containing TROPOMI observations virtual TROPOMI observations of the GEOS-Chem reference simulation elements of the Jacobian matrix for each TROPOMI orbit relevant to the inversion. All quantities have been converted to 1D fields indexed by latitude and longitude.
`data_converted_posterior/`	Directory of Python pickle (`.pkl`) files containing TROPOMI observations virtual TROPOMI observations of the GEOS-Chem posterior simulation for each TROPOMI orbit relevant to the inversion. All quantities have been converted to 1D fields indexed by latitude and longitude.
`data_geoschem/`	Directory of netCDF (`.nc`) files containing daily GEOS-Chem `SpeciesConc` output from the reference simulation. These files are used to generate virtual TROPOMI observations for comparison with the true observations.
`data_geoschem_posterior/`	Directory of netCDF (`.nc`) files containing daily GEOS-Chem `SpeciesConc` output from the posterior simulation. These files are used to generate virtual TROPOMI observations for comparison with the true observations.
`data_sensitivities/`	Directory of netCDF (`.nc`) files containing daily 4-D GEOS-Chem sensitivities to perturbations in the state variables of the inversion (i.e., in the emission elements being optimized). The data have dimensions `(element, lev, lat, lon)`, where `element` is the emission element id (state variable id) and `lev` is the vertical dimension. These files are used to compute the Jacobian matrix by application of the TROPOMI operator.
`inversion_result.nc`	NetCDF file containing the raw output of the inversion (`invert.py`) as vectors (posterior emission estimate) and matrices (posterior error covariance matrix, averaging kernel matrix `A`).
`gridded_posterior.nc`	NetCDF file containing the posterior emission scale factors (`ScaleFactor`), diagonal elements of the posterior error covariance matrix (`S_post`), and diagonal elements of the averaging kernel matrix (`A`) projected onto the 2-D inversion grid.
`visualization_notebook.ipynb`	Jupyter notebook for quickly visualizing key results of the inversion. Plots generated by this notebook are also saved out to the output/ directory as `.png` files.
`visualization_notebook.html`	For convenience, we run the Jupyter notebook as part of the IMI and output the resultant notebook to an html file, which can be viewed and rendered by your browser.