Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution][Exception Lists] Prevent loading of >10k value list items into memory #212460

Open
rylnd opened this issue Feb 25, 2025 · 2 comments
Labels
Feature:Rule Exceptions Security Solution Detection Rule Exceptions area Feature:Rule Value Lists Security Solution Detection Rule Value Lists area Team:Detection Engine Security Solution Detection Engine Area Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. triage_needed

Comments

@rylnd
Copy link
Contributor

rylnd commented Feb 25, 2025

Summary

As a result of the investigation in #211637, it was discovered that we do in fact load all of the items from a particular value list into memory during the deletion of said list. As per #203017, this should be mitigated as much as possible.

This might involve performing these deletions in batches, or we might be able to stretch beyond the soft 10k limit by minimizing the data that we're retrieving (as suggested in #203017).

However, it appears as though the current logic is a consequence of our current APIs, so we may need to develop some additional functionality (perhaps as public HTTP endpoints) to make this work.

@rylnd rylnd added Feature:Rule Exceptions Security Solution Detection Rule Exceptions area Feature:Rule Value Lists Security Solution Detection Rule Value Lists area Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detection Engine Security Solution Detection Engine Area triage_needed labels Feb 25, 2025
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detection-engine (Team:Detection Engine)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Rule Exceptions Security Solution Detection Rule Exceptions area Feature:Rule Value Lists Security Solution Detection Rule Value Lists area Team:Detection Engine Security Solution Detection Engine Area Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. triage_needed
Projects
None yet
Development

No branches or pull requests

2 participants