Skip to content

Protocols on updating FATES within CTSM

Jump to bottom
Erik Kluzek edited this page Mar 29, 2018 · 39 revisions

Introduction:

The FATES code is an external to CTSM and an optional configuration. The version of FATES that CTSM points to is a version under the NCAR/fates-release repository. Only major tags are brought to this repository from the main NGEET/fates repository. The version of FATES brought into CTSM will also undergo normal CTSM testing and as such may contain small bug fixes as needed to the FATES NGEET versions. Those bug fixes and changes will then make their way to the NGEET fates repository. FATES users and developers however, may have need to update the default version of FATES pointed to in CTSM tags and will need to update their FATES external pointed to in CTSM to a tag or branch in the NGEET FATES fork, as well as their own fork of FATES. Some of the core CTSM developers are also responsible to update the version of FATES pointed to by default CTSM as well as do the needed testing with the new version of FATES for CTSM. Core FATES developers then will also take any changes that go into NCAR/fates-release and bring it into the FATES NGEET repository. So this guide is divided into those three functions. Updating the FATES version for one's own use, update FATES for CTSM tags, and bringing FATES changes in NCAR/fates-release to the NGEET/fates repository.

NOTE REGARDING MANAGE_EXTERNALS: The philosophy of the manage_externals tool is to provide an easy way to checkout all of the externals needed that go along with a CTSM tag, and to checkout specific versions that a given tag was tested with. It's expected that your real version control work will be done using git. With that philosophy manage_externals tries to stay out of your way and not do too much. It does exactly what you ask it to do, and then stops, and that means pointing to a known frozen version. So it might not do what you want it to do, and in that case you should do what you are actually trying to do in git rather than with manage_externals. In general when you run manage_externals, you want to have "clean" directories in your clone, where any changes you have made have been committed. And you want the externals that are pointed to in Externals_CLM.cfg for manage_externals to be the repository and tag or branch to be in agreement with what the FATES git clone is pointing to. You don't want manage_externals to needlessly clobber over your work.

What we will show here are steps that keep the FATES version that manage_externals is pointing to in sync with what is in your fates clone under "src/fates". You don't HAVE to do this, but it makes it easier when it's time to actually run manage_externals. Note, that typically you don't really need to run manage_externals, except the first time you create a clone, and if you are updating to a newer CTSM tag that contains externals updates. If you aren't updating tags or externals, for a given clone, you don't really need to keep manage_externals up to date, with what you are doing in your FATES clone.

FATES user

Updating FATES for your own use

When a FATES user wants to point to a different version of FATES than CTSM is pointing to. The following steps are taken. The first list are the steps you do to update your CTSM clone to point to a different NGEET FATES version.

  1. $EDITOR Externals_CLM.cfg
  2. Change repo_url to the NGEET fork (repo_url = https://github.com/NGEET/fates)
  3. Change the tag pointed to for FATES (or replace tag = <tagname> with branch = <branchname>).
  4. manage_externals/checkout_externals (will add a remote for fates called NGEET_fates)
  5. cd src/fates
  6. git remote rename origin NCAR_fates # Rename origin
  7. git remote add $GITUSER https://github.com/$GITUSER/fates.git # Add a remote for your fork

Creating a branch on your fork

These next steps are what you do to create a new FATES branch from your fork, to checkin changes that you are working on. It does assume you did all of the previous steps. This includes the step of creating a new branch, if you already have a branch on your fork, you can skip that step. To create a branch to work on...

  1. cd src/fates
  2. Create branch from NGEET_fates branch or tag: i.e. git checkout -b <newbranch> NGEET_fates/master
  3. Send the new branch to your fork i.e. git push -u $GITUSER <newbranch> # Note, assumes step 7 above
  4. cd ../..
  5. $EDITOR Externals_CLM.cfg
  6. Change repo_url to your fork (repo_url = https://github.com/$GITUSER/fates)
  7. Remove tag/branch and add branch keyword for the new branch you made (branch = <newbranch>)
  8. ./manage_externals/checkout_externals # Fates will now point to your branch as a DETACHED HEAD
  9. $EDITOR Externals_CLM.cfg
  10. Now change the repo_url to be your local clone (repo_url = .)
  11. ./manage_externals/checkout_externals
  12. cd src/fates
  13. git checkout <newbranch> # Do a git checkout of the branch, so you can checkin and no longer in DETACHED HEAD state

Then make changes and commit to your clone as normal.

To push commits on your clone to your fork.

  1. cd src/fates
  2. git push -u $GITUSER <newbranch>

CTSM Developer

Updating to a newer FATES version (without an API change)

When new FATES tags have gone into NGEET and we want to bring those changes into CTSM, the following steps need to happen.

  1. Changes in a NGEET tag are brought in as a Pull Request to the NCAR fates-release fork, and a tag made for the update. Any FATES developer could do the PR.
  2. The PR is approved by the FATES/CTSM gatekeeper. And a new tag is created in the NCAR fates-release fork,
  3. A CTSM developer runs the standard testing for the FATES update on cheyenne and hobart using both the "aux_clm" and "fates" test lists. Any changes needed for testing go into a new tag in the NCAR fates-release fork.
  4. The changes are presented as Pull Request to CTSM, and approved by a CTSM developer who updates the CTSM ChangeLog and creates a CTSM tag for the update.
  5. If there are changes in the NCAR fates-release fork, those changes should be integrated into NGEET/fates as below.

Updating to a newer FATES version (with an API change)

The same steps as above are followed, but the Pull Request is created from the fates_next_api branch according to the steps under the last section below [on updating a newer FATES version WITH an API change] (#ctsmfates-developer).

FATES Developer

Bringing changes from NCAR/fates-release to NGEET/fates

A FATES developer creates a Pull Request for any changes that went into the NCAR fates-release fork, into the NGEET fates fork. Those changes then get integrated into the NGEET master as normal for NGEET/fates. If there aren't any such changes they won't have to be brought into NGEET.

CTSM/FATES Developer

Updating to a newer FATES version WITH an API change

In order to faster synchronize FATES API development we have a permanent branch in CTSM called fates_next_api that allows a development path forward for a change in FATES that also requires a change in CTSM. Eventually and periodically this branch will come to the trunk of CTSM by the normal process outlined above. But, in order for FATES development to happen faster, we have this fates branch on ESCOMP/CTSM for both FATES developers and users.

  1. FATES or CTSM developer creates a Pull Request on ESCOMP/ctsm to the fates_next_api branch. The changes for the PR should have run the fates test-list on any one of the machines for fates testing: cheyenne, hobart, or lawrencium-lr3
  2. The PR in 1 is approved by a different developer than the one that created it.
  3. The changes to fates_next_api then are brought in as a Pull Request for CTSM by the normal process above.
  4. Periodically the fates_next_api will need to updated to tags brought in CTSM master. This should be done as a Pull Request in the process as at the start of this process. It should only be done for CTSM tags and only as needed.
Clone this wiki locally