Skip to content

Fix clear cache to clear valkey instead of filesystem cache #1234

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 12, 2025

Conversation

kalilsn
Copy link
Member

@kalilsn kalilsn commented May 8, 2025

Issue(s) Resolved

Resolves #1233

High-level Explanation of PR

After #1131 we no longer cache data on the filesystem, and instead store it in redis. In both local development and in the sandbox deployments, we regularly run the reset command, which also attempts to clear the cache. Since that cache-clearing function wasn't working, we could end up with stale data in the sandbox cache.

Test Plan

It's easiest to see this clear the cache if you install redis-cli (or some kind of gui redis app). redis-cli monitor will let you see commands as they come in, and redis cli scan 100 will print 100 keys so you can see if there's data in the cache.

  • With the valkey cache running (pnpm -w dev:cache:start) and after using the app a bit, run redis cli scan 100 to show there's data in the cache
  • Optionally, run redis-cli monitor in a separate terminal
  • Run pnpm clear-cache
  • Scan again and see that the cache is clear + observe the flushall command sent in the monitor window

To actually recreate the initial sandbox bug requires a bit more effort, since you need to check out a previous commit (1c21713d4321f728ee4b6362f2b63675542868c1) and run reset.

I've also confirmed that this reset succeeds on lightsail by sshing into the preview instance for this branch and confirming that the migrations container exits with status 0 and that you can see the cache clearing command run in the logs for that container.

Notes

I don't think this is an issue in prod. My guess is the reason we're seeing strange behavior is that the seed scripts have a mixture of hardcoded and generated data, so we end up with some cache hits to stale data after reset. But that doesn't entirely make sense, so lmk if you have other ideas.

@kalilsn kalilsn added the preview Auto-deploys a preview application label May 8, 2025
@3mcd 3mcd temporarily deployed to gh-654103159-pr-1234 May 8, 2025 19:32 Inactive
@kalilsn kalilsn requested review from 3mcd and tefkah May 8, 2025 19:33
@3mcd 3mcd temporarily deployed to gh-654103159-pr-1234 May 8, 2025 21:25 Inactive
@kalilsn kalilsn force-pushed the kalilsn/fix-clear-cache branch from f8b4801 to 67df831 Compare May 8, 2025 21:30
@3mcd 3mcd requested a deployment to gh-654103159-pr-1234 May 8, 2025 21:35 Pending
Copy link
Member

@tefkah tefkah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to work! No clue what's happening in the preview env tho

@kalilsn
Copy link
Member Author

kalilsn commented May 12, 2025

No clue what's happening in the preview env tho

Failing to load env vars :(. dotenv doesn't put env vars in the environment so I was using source. but source throws an error if the passed file doesn't exist (.env.local), so I would have needed some pretty ugly conditionals to make that version work everywhere.

@3mcd 3mcd had a problem deploying to gh-654103159-pr-1234 May 12, 2025 13:32 Error
@tefkah
Copy link
Member

tefkah commented May 12, 2025

my guess is that that will also not work, could you not do echo 'FLUSHALL' | dotenv -e .... -- nc ...?

@tefkah
Copy link
Member

tefkah commented May 12, 2025

if that also doesn't work i'd just write a separate script 😅

@kalilsn
Copy link
Member Author

kalilsn commented May 12, 2025

I think what I have now should work. I'm using dotenv ... -p VALKEY_HOST which just prints out the specific var I'm trying to use.

echo 'FLUSHALL' | dotenv -e .... -- nc ... doesn't actually set environment variables, it populates process.env. So only a node script will actually be able to access those (not nc).

@3mcd 3mcd had a problem deploying to gh-654103159-pr-1234 May 12, 2025 13:38 Error
@kalilsn kalilsn merged commit 5299366 into main May 12, 2025
17 checks passed
@kalilsn kalilsn deleted the kalilsn/fix-clear-cache branch May 12, 2025 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
preview Auto-deploys a preview application
Projects
None yet
Development

Successfully merging this pull request may close these issues.

sandbox users stuck on Settings page
3 participants