Feature/issue 7 - Remove wait for GITC response #30
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
frankinspace
requested changes
Jun 5, 2024
frankinspace
approved these changes
Jun 17, 2024
frankinspace
added a commit
that referenced
this pull request
Feb 4, 2025
* /version 0.2.0a0 * feature: convertToPNG --> sendToHarmony (#21) * update state_machine, sendToHarmony * --changelog Issue-15 * moved to unreleased, changelog * /version 0.2.0a1 * Update README.md (#22) * /version 0.2.0a2 * Use -j to remove top-level directory in zip artifact (#25) * /version 0.2.0a3 * Issue#16: Change apply opera treatment choice and lambda to be specific to HLS (#24) * update for HLS -- lambda and state machine * py lint error * update file name * /version 0.2.0a4 * /version 0.2.0a5 * Feature/issue 7 - Remove wait for GITC response (#30) * fix type in readme * change task token to uuid in send to gitc * remove wait for task token & gitc response handler from state machine * change gitc response handler to invoke save cma message * fix typo * fix context parameter * convert uuid to string * fix json typo * remove uuid lib to use stdlib uuid * change identifier to image set name instead of uuid * remove uuid import * fix json formatting of cma invoke * add granule concept ID to identifier * update image set name with granule conceptid * remove whitespace * move get umm json to utils and save gitc outgoing cnm * remove unused import * try setting cmr query env based on stage * remove task token from tests * fix granule index * change how image set name defined * add region to ssm client in utils * add ssm parameters to gitc lambdas * change region reference * update how granule name referenced in send to gitc * remove unused cmr var * fix case * fix cnm parsing * reformat save cnm to separate step * update tf vars for save cnm * fix type in module definition * fix cnm in cma * fix collection reference in save cnm * add debugging log statements * fix input * update save cnm input * debugging * debugging * linting * debugging * debugging * linting * fix state machine * update state machine * reorg state machine * missing comma * move save cnm into map * fix state machine transitions * fix sm * fix boolean * debugging cnm input * fix input * change input * debug input * add cnm as config parameter * lint * fix cnm payload * remove debugging statements * add prefix to cnm path and fix gitc response * increase handle gitc response timeout * update changelog * add original shortname as cnm prefix * revert shortname and change collection ref to save cnm & cnm-r in same location * remove save cma lambda no longer used * remove EDL env params from sendtogitc * change parsing of granule concept id * /version 0.2.0a6 * Update README.md * /version 0.2.0a7 * Support datetimes without microseconds (#37) * Support datetimes without microseconds * changelog * Update generate_image_metadata.py (#39) add docstring * /version 0.2.0a8 * /version 0.2.0a9 * Issues/23: Enable Harmony write output to user bucket (#46) * add s3 destination to harmony job * syntax * job success * harmony block update * --variable * temp test in uat * manual to uat * test s1-v1 * lint * deploy --sit * Update to place harmony results in our bucket * Update to place harmony results in our bucket * Update to place harmony results in our bucket * Update to place harmony results in our bucket * Update to place harmony results in our bucket * Update to place harmony results in our bucket --------- Co-authored-by: Josh <joshhaile7@gmail.com> * /version 0.2.0a10 * fix deploy failure * /version 0.2.0a11 * Add new message attribute response_topic_arn to all outgoing messages to GIBS (#47) * /version 0.2.0a12 * prep for 0.2.0 * /version 0.2.0a13 * Fix build error * /version 0.2.0a14 * /release 0.2.0 * Update graph * /version 0.2.0rc1 * Turn prevent_destroy back on for gibs response topic * /version 0.2.0rc2 * Improve error message when missing config * /version 0.2.0rc3 * Improve error message when missing config * /version 0.2.0rc4 * UAT is in east-1 region * /version 0.2.0rc5 * fix gibs queue name * /version 0.2.0rc6 * calculate md5sum for files from harmony * calculate md5sum for files from harmony * /version 0.2.0rc7 * bump chunk size to 100 MB * /version 0.2.0rc8 * Update test message for UAT, add checksum to generated tif, add logging, fix type error * Lint * Lint * /version 0.2.0rc9 * Change named variables for audit path from pobit to bignbit * /version 0.2.0rc10 * Update docs --------- Co-authored-by: frankinspace <frankinspace@users.noreply.github.com> Co-authored-by: Josh Haile <35245966+voxparcxls@users.noreply.github.com> Co-authored-by: Victoria McDonald <49625194+torimcd@users.noreply.github.com> Co-authored-by: Jake Herrmann <jtherrmann1@gmail.com> Co-authored-by: James Wood <James.F.Wood@jpl.nasa.gov> Co-authored-by: jamesfwood <jamesfwood@users.noreply.github.com> Co-authored-by: Josh <joshhaile7@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Github Issue: closes #7
Description
Waiting for the GITC response currently throttles the number of image sets we can send. Change to remove the wait period and complete successfully when we complete the send. Handle the GITC response outside of the state machine.
Overview of work done
Overview of verification done
No new unit tests, existing tests pass
Overview of integration done
Deployed to SIT. Workflow succeeds when invoked manually and both CNM & CNM-R messages are saved in the audit bucket under collection_var prefix
PR checklist:
See Pull Request Review Checklist for pointers on reviewing this pull request