Include ILAMB output in notebook and provide link to full diagnostic package

[TeaganKing]

Description of changes:

• Wrote script to generate ILAMB model_setup.txt file based on config.yml
• Included link to ILAMB notebook which includes a link to ILAMB full output as well as a few 'key metric' plots from ILAMB directly in the jupyterbook
• Included config files for BGC & SP cases (without hardcoded glade paths) in a parallel location to ADF config file templates.

• Have you followed the guidelines in our Contributor's Guide?
• Have you checked to ensure there aren't other open Pull Requests for the same update/change?
• Have you made sure that the pre-commit checks passed (#8 in Adding Notebooks Guide)?
• Have you hidden the code cells (#8 in Adding Notebooks Guide) in your notebook?
• Have you added an explanation of what your changes do and why you'd like us to include them?
• Have you successfully tested your changes locally?

[TeaganKing]

Note the linked issue also requires workflow updates to include ILAMB and choose BGC or SP.

[TeaganKing]

• Note to self: need to remove some printed statements in link_to_ILAMB.ipynb that are currently there for development purposes once we get ILAMB working with CLM6

[TeaganKing]

Made some changes per discussion in CUPiD meeting today.

• Added active layer thickness plot
• A few other minor changes (eg, remove word 'link')

Still need to do the following:

• Add overarching "score chart"
• Remove glade filepaths, specify them in BGC cfg file, and add to helper script to generate these
• Add into key metrics?

[TeaganKing]

The overarching score chart is not a static image-- rather an html page. Thus, I think it'd be difficult to reproduce as a static image within the CUPiD jupyterbook output. This will also be the first thing that appears when you click on the link to the full ILAMB output, so I'm going to suggest we don't implement this as one of the plots that shows up automatically within the jupyterbook after all.

Also, for single runs, this is not the most interesting score chart...

It's really only useful for comparisons against other runs:

[mnlevy1981]

For key_metrics, can we run ILAMB on both the new case and the baseline? And is that table useful if there are two columns? If so, I think you can imbed one HTML page into another with something like this:

display(
    HTML(
         '<iframe src="/path/to/table.html"></iframe>'
    )
)

But I agree that it's not very useful for the single run case. If the above code works, maybe we could include it in an if print_table: block where print_table defaults to False, just so it is available in the future. If the above code doesn't work, I don't think it's worth the time and agree with

I'm going to suggest we don't implement this as one of the plots that shows up automatically within the jupyterbook after all

[mnlevy1981]

(my first version of the above comment relied on javascript, but I think the HTML <iframe> option would work better)

[mnlevy1981]

I didn't try to run this, but read through the diffs and have a few comments. Overall, this looks really good and I'm excited about adding ILAMB to external_diag_packages!

Should we address #160 in this PR, since we'll have the same problem with ILAMB and users only running CUPiD on non-land components? Seeing your change to cupid/cupid_webpage.py brought that issue to mind, but I'd be happy to wait for a later PR to try to fix it.

[TeaganKing]

This is running smoothly now. I added a note to remove the ILAMB_output directory if it exists, as that potentially causes errors.

After talking with Keith, we also decided that using a BGC/SP flag specified by users is better than relying on an assumption that f cases are usually SP and b cases are usually BGC. If an SP case is run with a BGC flag, we may just have some additional variables that don't produce graphs, etc.
Also, ILAMB is best used with historical cases. If 1850 cases are used, one could technically specify an additional argument in the model_setup.txt file that includes a model year that corresponds with a real year. However, this is potentially misleading, so we're going to assume that people will be comparing reasonable cases if using ILAMB.

This is now ready for review.

[mnlevy1981]

I had one comment about cleaning up some of the code in cupid_webpage.py, but I also have a general question about the functionality of the new generate_ilamb_config_files.py.

I was trying to see how this would fit in with the CESM workflow, so I went into a test case directory and used generate_cupid_config_for_cesm_case.py to create config.yml. Then I used the new script to generate the config files for ILAMB:

$ ../../../tools/CUPiD/helper_scripts/generate_ilamb_config_files.py --cesm-root /glade/work/mlevy/codes/CESM/cesm3_0_beta04_cupid --cupid-config-loc ${PWD} --run-type SP
{'cesm_root': '/glade/work/mlevy/codes/CESM/cesm3_0_beta04_cupid', 'cupid_config_loc': '/glade/work/mlevy/codes/CESM/cesm3_0_beta04_cupid/cases/test_cupid_30b04.005/cupid', 'run_type': 'SP'}
wrote /glade/work/mlevy/codes/CESM/cesm3_0_beta04_cupid/tools/CUPiD/ilamb_aux/ilamb_nohoff_final_CLM_SP.cfg
wrote /glade/work/mlevy/codes/CESM/cesm3_0_beta04_cupid/cases/test_cupid_30b04.005/cupid/model_setup.txt
WARNING: ILAMB requires regridded output to be in /glade/derecho/scratch/mlevy/archive/b.e23_alpha17f.BLT1850.ne30_t232.092/lnd/hist/regrid/ directory.
You can now run ILAMB with the following commands:
(Users on a super computer should make sure they are on a compute node rather than a login node)
---------
conda activate cupid-analysis
export ILAMB_ROOT=../ilamb_aux
ilamb-run --config /glade/work/mlevy/codes/CESM/cesm3_0_beta04_cupid/tools/CUPiD/ilamb_aux/ilamb_nohoff_final_CLM_SP.cfg --build_dir /glade/work/mlevy/codes/CESM/cesm3_0_beta04_cupid/cases/test_cupid_30b04.005/cupid/ILAMB_output/ --df_errs /glade/work/mlevy/codes/CESM/cesm3_0_beta04_cupid/tools/CUPiD/ilamb_aux/quantiles_Whittaker_cmip5v6.parquet --define_regions /glade/work/mlevy/codes/CESM/cesm3_0_beta04_cupid/tools/CUPiD/ilamb_aux/DATA/regions/LandRegions.nc /glade/work/mlevy/codes/CESM/cesm3_0_beta04_cupid/tools/CUPiD/ilamb_aux/DATA/regions/Whittaker.nc --regions global --model_setup /glade/work/mlevy/codes/CESM/cesm3_0_beta04_cupid/cases/test_cupid_30b04.005/cupid/model_setup.txt --filter .clm2.h0.
---------

The first thing that caught my eye is the location of the output files:

wrote /glade/work/mlevy/codes/CESM/cesm3_0_beta04_cupid/tools/CUPiD/ilamb_aux/ilamb_nohoff_final_CLM_SP.cfg
wrote /glade/work/mlevy/codes/CESM/cesm3_0_beta04_cupid/cases/test_cupid_30b04.005/cupid/model_setup.txt

model_setup.txt showed up in the directory where I ran the generate script, which is good. Shouldn't ilamb_nohoff_final_CLM_SP.cfg also be written locally? We may need to also copy the quantiles file locally, then we can change the recommendation from export ILAMB_ROOT=../ilamb_aux to set the current working directory to ILAMB_ROOT.

-print("export ILAMB_ROOT=../ilamb_aux")
+print(f"export ILAMB_ROOT={os.getcwd()}")

(which was actually the second thing I noticed -- as the script is written, we want print(f"export ILAMB_ROOT=os.path.join(cesm_root,'tools','CUPiD','ilamb_aux')")).

I think some of these suggestions for getting better output in the CESM workflow may break things for the CUPiD examples; it'll probably take a little back-and-forth to figure out how we want things to work in both use-cases.

[TeaganKing]

Hi @mnlevy1981 , Thanks for reviewing this-- I have addressed your comments!

RE "Shouldn't ilamb_nohoff_final_CLM_SP.cfg also be written locally?": I did update this per your suggestion. For context, my original thought was that these won't actually change with every configuration for running CUPiD; there's really just a general specification of ilamb_config_data_loc that gets added, and that's the only change, so I figured it could be used from the same place for any ILAMB runs using the same CESM root. That said, it's not a large file, so I think that it may be clearer to write it locally and keep everything together anyways.

[mnlevy1981]

I'm still having trouble running this in a CESM case, though it's getting closer. I'm playing with this in /glade/work/mlevy/codes/CESM/cesm3_0_beta04_cupid/cases/test_cupid_30b04.005/cupid and, with the changes mentioned in the individual comments

${CESM_ROOT}/tools/CUPiD/helper_scripts/generate_ilamb_config_files.py --cesm-root /glade/work/mlevy/codes/CESM/cesm3_0_beta04_cupid --cupid-config-loc . --run-type SP

works as expected, but

export ILAMB_ROOT=ilamb_aux
ilamb-run --config ilamb_nohoff_final_CLM_SP.cfg --build_dir bld/ --df_errs ${ILAMB_ROOT}/quantiles_Whittaker_cmip5v6.parquet --define_regions ${ILAMB_ROOT}/DATA/regions/LandRegions.nc ${ILAMB_ROOT}/DATA/regions/Whittaker.nc --regions global --model_setup model_setup.txt --filter .clm2.h0.

Results in lots of VarNotInModel messages.

The good news is that I tried to generate a CUPiD config file based on external_diag_packages/config.yml, and realized one of the issues I was having was that b.e23_alpha17f.BLT1850.ne30_t232.092 doesn't have regridded output while f.e23_alpha17f.FLTHIST_ne30.roughtopo.099 does. So I can sort of run this with CESM output, except my new run doesn't have regridded output either :)

Can we try to chat on Monday to iron out these last issues before merging in?

[mnlevy1981]

LGTM!