Reduced output in NorESM simulations

[adagj]

As part of an INES2 deliverable, we need namelists for all components with a reduced set of variables and output frequencies. This will optimize storage use for long coupled simulations and test experiments. It will also support users running long simulations (e.g., up to 2300) who may not be familiar with all components and key variables. A first set is needed as soon as possible for testing, with updates possible later.

But what is the best way to include the reduced output namelists? Some options:

1. implement these options during namelist generation, i.e. it would be an option in env_run.xml taken up by the namelist generation script
2. Create a user-mods directory, similar to what was done for CMIP6 simulations, so these namelists can be added when creating a new case.

I think it make sense that all components use the same method, but I'm not sure if that is absolutely necessary.

@JensBDebernard @gold2718 @matsbn @rosiealice @hgoelzer

[jmaerz]

Currently started for iHAMOCC: NorESMhub/BLOM#478 - but still in discussion, how to best proceed/up to change/to be aligned to the discussion here.

[hgoelzer]

Even if CISM may not run in many of the experiments, we would really like relevant coupler output to always be written in all development experiments, so we can use it for forcing and testing in data mode. The output we need is typically regional (polar) and with annual time resolution, so this will likely be negligible in the the larger context.

[JorgSchwinger]

With "relevant coupler output" do you mean the output written by CLM that we have activated before in NorESM2 simulations?

[hgoelzer]

Exactly! And we will have something similar from BLOM for ocean forcing. We are working on getting this into the next tag, so it is activated automatically.

[JorgSchwinger]

I din not think about the possibility of having use-cases before, so maybe that is the easiest option? Apart from the fact that this needs to be maintained outside of the respective components, but that is probably fine?

[gold2718]

One advantage of using user-mods at the NorESM level is it can include output configuration information for all components in one place. I believe each component still needs to deliver namelist options relevant for various levels of reduced output for long simulations.

[TomasTorsvik]

I'm not entirely convinced that user-mods is the best option. As long as we don't have a code freeze, I suppose the component output may still change. With user_mods, I guess we would need to change both the component output and the user_mods namelists at the NorESM level, at the same time.

[jmaerz]

Since we had the discussion BLOM internally recently, I wondered, if one could enhance/expand cime to provide/populate the key-word given to the user-mods somehow to the components (as in that it enables to provide a flag to the namelist creation) - which would enable to e.g. handle all those things at the component level, if possible, and those components which don't have this possibility can still rely on user-mods? If user-mods would be provided for the component user-mods nml would override the namelist-flagged component level.

[JorgSchwinger]

Ok then let's go for user-mods. I can provide something for BLOM/HAMOCC

For now we would just go for a "reduced output" case that should be good enough for spin-up, or are we going for something more fine-grained (e.g. small/medium/large/xxlarge)?

[adagj]

I suggest we go for good-enough for the spin up. Then we can add more fine-grained options later

[oyvindseland]

I see I have misunderstood the discussion point so deleted the specific CAM issues.
With respect to the discussion: I think user-mods is fine, but at least for CAM I suggest the much of the output should be grouped together
as namelist variables, but used within the user-mods, i.e. user_nl_cam as it is today.
I do suggest however that the the core group should agree on a specific set of variables for the spin-up so that if someone from outside the core group runs the model they can expect a decent selection of variables needed for analysis without needing read through a long namelist and check with the code to understand what variables are excluded.

On the other hand if we end up using user-modes we can have a relatively large set of user-modes for different simulation types.

[kjetilaas]

For the land output, we should use the following functionality to reduce FATES output in NorESM development simulations. This is in addition to using a reduced set of variables, but related to the discussion here.

Suggested namelist setting:
fates_history_dimlevel = 1,1

Description:
doc/ChangeLog:One-line Summary: New FATES namelist variable, fates_history_dimlevel
doc/ChangeLog:This set of changes introduces a new namelist setting that allows more control over fates history diagnostics. This setting, fates_history_dimlevel accepts two integers, comma-delimited, from 0-2. The first specifies the history output dimension level for high-frequency output (ie model timestep) and the second is for output at the dynamics timestep. A value of 0 indicates no history variables should be processed. A value of 1 indicates that only site-level mean values should be processed. A value of 2 indicates that all variables, including those that use an extra dimension should be processed. This is different from adding and excluding history variable names from the namelist, in that these settings not only omit variables from the output file, but they prevent their allocations and calculations all together. Processing history diagnostics in FATES takes a non-trivial amount of time.
doc/ChangeLog:Users should be aware of the new namelist setting fates_history_dimlevel. Omitting this setting will default to a level of "2" which enables all output and should maintain existing model behavior.

@rosiealice @adagj @mvdebolskiy

[jmaerz]

Hi @mvdebolskiy , could you share, how you set up the usermods folder and script that it just sets the usermods via a xmlchange command at the component level?

[TomasTorsvik]

There are some exaples in the CAM repository, see for instance
https://github.com/NorESMhub/CAM/tree/noresm_develop/cime_config/usermods_dirs/scam_camfrc

[jmaerz]

Thanks, that was the missing bit of information. So we can keep everything on output related entries in namelist_definition_blom.xml and just put a shell_commands file with an xmlchange command in the respective usermods directory.

[TomasTorsvik]

I suppose there should be only one shell commands file for each usermods, so for fully coupled setup this file should probably be placed in the NorESM repository(?)

[jmaerz]

I thought it would be picked up by cime in the BLOM component folder, but it might be a misperception. In any case, I feel it as a cleaner solution than distribution several user_nl_blom files ...

[mvdebolskiy]

Were do you put the file shell_commands and how do you pick up the the output volume there (e.g. by using the keyword provided to user_mods)? - is there somewhere an example, how this is done?

In the directory which you are pointing to with --user-mods-dir or through include_user_mods file.