Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Local DynamoDB streams Integration #2918

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

dakota002
Copy link
Contributor

Resolves #2804

This PR adds @nasa-gcn/architect-plugin-dynamodb-local-streams as a dependency, updates the app.arc, and adds a few other changes to configure the testing environment to work both with a local docker instance for dynamodb and the table-streams functionality.

There is a known issue with the docker dynamodb local image that I have described in the readme of this package. https://github.com/nasa-gcn/architect-plugin-dynamodb-local-streams
But it does not impede our development. In short, the first invocation for each stream will throw a TrimmedDataAccessException. This is expected and is intentionally suppressed and logged in the console. Subsequent invocations run as expected.

In order to fully set this up, you must add the following to your .env:

ARC_TABLES_PORT=8000
ARC_DB_EXTERNAL=true

To Test:

  • Make sure docker is running.
  • Install new packages:
npm i
  • Run dev
npm run dev

Current Streams that can be invoked:

  • Circulars
    • New circulars, edit both will trigger the stream
  • Synonyms
    • Need to enable the SYNONYM feature flag.
    • Create new synonyms through the Synonym Moderation feature

Copy link

codecov bot commented Feb 24, 2025

Codecov Report

Attention: Patch coverage is 0% with 7 lines in your changes missing coverage. Please review.

Project coverage is 6.29%. Comparing base (016d123) to head (e9ca2ea).

Files with missing lines Patch % Lines
app/lib/email.server.ts 0.00% 7 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##            main   #2918      +/-   ##
========================================
- Coverage   6.30%   6.29%   -0.01%     
========================================
  Files        175     175              
  Lines       4348    4353       +5     
  Branches     474     474              
========================================
  Hits         274     274              
- Misses      4072    4077       +5     
  Partials       2       2              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Member

@lpsinger lpsinger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dakota002
Copy link
Contributor Author

See nasa-gcn/architect-plugin-dynamodb-local#6 (comment).

I left a response there, but I am going to track this as a further enhancement to the plugin. It requires more work, but doesn't hold up reaching this goal

@lpsinger
Copy link
Member

See nasa-gcn/architect-plugin-dynamodb-local#6 (comment).

I left a response there, but I am going to track this as a further enhancement to the plugin. It requires more work, but doesn't hold up reaching this goal

We need to add some more items for this to our development instructions, don't we?

@dakota002
Copy link
Contributor Author

See nasa-gcn/architect-plugin-dynamodb-local#6 (comment).

I left a response there, but I am going to track this as a further enhancement to the plugin. It requires more work, but doesn't hold up reaching this goal

We need to add some more items for this to our development instructions, don't we?

Yea, I can update the docs for that

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do the changes in this file have to do with the DynamoDB simulator?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you trigger the lambda, it tries to send an email, which can fail if either the template is not defined (which it isnt on local, thats a sandbox thing I believe) or if the user doesn't have valid credentials and pointing to a matching env (which would be the case for running playwright tests)

Also I am working on resolving the currently failing tests locally, should have that out later

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like you're still working on fixing the playwright tests?

@lpsinger
Copy link
Member

lpsinger commented Mar 5, 2025

Also, please resolve merge conflicts.

Install actual package

cleanup

remove comment

Update docs for config updates

Trying a fix for tests and move the dependency
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrate table streams into locally testable environment
3 participants