fix: infinite cache growth

[regisb]

See the discussion here: #984
And the upstream PR here: openedx/edx-platform#34210

The tl;dr is that the Redis course structure cache was growing without bounds. While the upstream fix should resolve that issue, we decided that Tutor should have a maxmemory limit and an eviction policy set for operational safety.

tutor local run cms ./manage.py cms shell -c "from django.core.cache import caches; c = caches['course_structure_cache']; [c.expire(key, 604800) for key in c.keys('*')]"

FYI @Ian2012 @ormsbee

[regisb]

Let's wait until the upstream PR is merged, then update the commit sha1 in the dockerfile.

[VirtualDiegox]

@regisb just to be sure, this fix should be set also in the k8s templates? specifically in volumes.yml? I faced the same issue and i had to resize the vpc manually before running the command to expire the existing keys

[regisb]

@VirtualDiegox As far as I know, this fix does not require any change made to volumes.yml. Well, we could increase the volume size to 4GB, but that would not be required by this change. Did you have something else in mind?

[VirtualDiegox]

@regisb With this fix the maxmemory parameter for redis is 4gb, but i don't know if the policy (LRU) apply if the volume size is smaller than the maxmemory parameter, in a k8s environment if the volume storage for redis is full, the platform just collapses with 500 errors because it is not able to create new session keys, so i think it should be consistent in the volume declaration or it could be a configuration parameter (4gb by default for example) that overrides the size required for redis in both files (redis.conf and volumes.yml)

[regisb]

@VirtualDiegox I'm wondering if you are facing the issue that we actually solved here. I.e: the infinite growth of the course structure cache. How much space is your Redis instance using after you clear the course structure cache?

[VirtualDiegox]

@regisb Yes I think i was facing that issue too, but our environment it's based on the k8s deployment, that's why I asked if the volume size in the k8s template should be increased too, right now our Redis volume is 4gb. The command to clear the course structure cache doesn't work if the space of the volume is full, so I solved it temporarily with a plugin that addionally to the changes that you made, create a new volume with 4gb instead of 1gb and override the redis deployment in order to use it. If you are wondering why create new volume instead of modifying the existing one it is due permissions issues in the cluster.

[regisb]

Right, I understand. But after you clear the course structure cache, how much memory is your Redis volume using, out of the 4GB maximum space?