Skip to content

Commit

Permalink
[8.17] [Security Solution] Added concurrency limits and request throt…
Browse files Browse the repository at this point in the history
…tling to prebuilt rule routes (#209551) (#210773)

# Backport

This will backport the following commits from `main` to `8.17`:
- [[Security Solution] Added concurrency limits and request throttling
to prebuilt rule routes
(#209551)](#209551)

<!--- Backport version: 9.6.4 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Dmitrii
Shevchenko","email":"dmitrii.shevchenko@elastic.co"},"sourceCommit":{"committedDate":"2025-02-11T17:12:03Z","message":"[Security
Solution] Added concurrency limits and request throttling to prebuilt
rule routes (#209551)\n\n**Resolves:
https://github.com/elastic/kibana/issues/208357**\n**Resolves:
https://github.com/elastic/kibana/issues/208355**\n\n## Summary \n\nTo
prevent possible OOM errors, we need to limit concurrent requests
to\nprebuilt rule routes (see attached tickets for more details).\n\n-
`installation/_perform` and `upgrade/_perform` endpoints\n- Concurrency
is limited to one parallel call. If another call is
made\nsimultaneously, the server responds with 429 Too Many Requests.\n-
On the front end, all rule install and upgrade operations are
retried\nin case of a 429 response. This ensures proper handling when a
user\nclicks multiple times an update or install rule buttons\n\n-
`prebuilt_rules/_bootstrap` endpoint\n- Install prebuilt rules and
endpoint packages sequentially instead of\nin parallel to prevent from
having them both downloaded into memory\nsimultaneously.\n- Added a
30-minute socket timeout to prevent the proxy from closing
the\nconnection while rule installation is in progress.\n- Introduced a
`throttleRequests` wrapper, ensuring the endpoint handler\nis called
only once when multiple concurrent requests are received.\n- The first
request triggers the handler, while subsequent requests wait\nfor the
first one to complete and reuse its result.\n- This prevents costly
prebuilt rule package installation from running\nin parallel.\n- Reusing
the response ensures the frontend correctly invalidates cached\nprebuilt
rule queries. Since concurrent frontend requests should receive\nthe
same installed package information, responding with 421 and using\nthe
retry logic as in cases above is not an option here because the\nsecond
request would receive a package installation skipped response\nleading
to no cache invalidation.\n\n- `installation/_review` and
`upgrade/_review` endpoints\n- Concurrency is limited to one parallel
call. If another call is made\nsimultaneously, the server responds with
429 Too Many Requests.\n- On the front end, all rule install and upgrade
operations are retried\nin case of a 429 response. This ensures proper
handling when a user\nclicks multiple times an update or install rule
buttons","sha":"c5557f33213f699acd9bb656af9166b1449d18f9","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","performance","v9.0.0","Team:Detections
and Resp","Team: SecuritySolution","Team:Detection Rule
Management","Feature:Prebuilt Detection
Rules","backport:version","v8.18.0","v9.1.0","v8.19.0","v8.17.3"],"title":"[Security
Solution] Added concurrency limits and request throttling to prebuilt
rule
routes","number":209551,"url":"https://github.com/elastic/kibana/pull/209551","mergeCommit":{"message":"[Security
Solution] Added concurrency limits and request throttling to prebuilt
rule routes (#209551)\n\n**Resolves:
https://github.com/elastic/kibana/issues/208357**\n**Resolves:
https://github.com/elastic/kibana/issues/208355**\n\n## Summary \n\nTo
prevent possible OOM errors, we need to limit concurrent requests
to\nprebuilt rule routes (see attached tickets for more details).\n\n-
`installation/_perform` and `upgrade/_perform` endpoints\n- Concurrency
is limited to one parallel call. If another call is
made\nsimultaneously, the server responds with 429 Too Many Requests.\n-
On the front end, all rule install and upgrade operations are
retried\nin case of a 429 response. This ensures proper handling when a
user\nclicks multiple times an update or install rule buttons\n\n-
`prebuilt_rules/_bootstrap` endpoint\n- Install prebuilt rules and
endpoint packages sequentially instead of\nin parallel to prevent from
having them both downloaded into memory\nsimultaneously.\n- Added a
30-minute socket timeout to prevent the proxy from closing
the\nconnection while rule installation is in progress.\n- Introduced a
`throttleRequests` wrapper, ensuring the endpoint handler\nis called
only once when multiple concurrent requests are received.\n- The first
request triggers the handler, while subsequent requests wait\nfor the
first one to complete and reuse its result.\n- This prevents costly
prebuilt rule package installation from running\nin parallel.\n- Reusing
the response ensures the frontend correctly invalidates cached\nprebuilt
rule queries. Since concurrent frontend requests should receive\nthe
same installed package information, responding with 421 and using\nthe
retry logic as in cases above is not an option here because the\nsecond
request would receive a package installation skipped response\nleading
to no cache invalidation.\n\n- `installation/_review` and
`upgrade/_review` endpoints\n- Concurrency is limited to one parallel
call. If another call is made\nsimultaneously, the server responds with
429 Too Many Requests.\n- On the front end, all rule install and upgrade
operations are retried\nin case of a 429 response. This ensures proper
handling when a user\nclicks multiple times an update or install rule
buttons","sha":"c5557f33213f699acd9bb656af9166b1449d18f9"}},"sourceBranch":"main","suggestedTargetBranches":["8.17"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/210642","number":210642,"state":"MERGED","mergeCommit":{"sha":"df87081d8a574c6b0bb2d9fc026776502622dc11","message":"[9.0]
[Security Solution] Added concurrency limits and request throttling to
prebuilt rule routes (#209551) (#210642)\n\n# Backport\n\nThis will
backport the following commits from `main` to `9.0`:\n- [[Security
Solution] Added concurrency limits and request throttling\nto prebuilt
rule
routes\n(#209551)](https://github.com/elastic/kibana/pull/209551)\n\n<!---
Backport version: 9.4.3 -->\n\n### Questions ?\nPlease refer to the
[Backport
tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT
[{\"author\":{\"name\":\"Dmitrii\nShevchenko\",\"email\":\"dmitrii.shevchenko@elastic.co\"},\"sourceCommit\":{\"committedDate\":\"2025-02-11T17:12:03Z\",\"message\":\"[Security\nSolution]
Added concurrency limits and request throttling to prebuilt\nrule routes
(#209551)\\n\\n**Resolves:\nhttps://github.com//issues/208357**\\n**Resolves:\nhttps://github.com//issues/208355**\\n\\n##
Summary \\n\\nTo\nprevent possible OOM errors, we need to limit
concurrent requests\nto\\nprebuilt rule routes (see attached tickets for
more details).\\n\\n-\n`installation/_perform` and `upgrade/_perform`
endpoints\\n- Concurrency\nis limited to one parallel call. If another
call is\nmade\\nsimultaneously, the server responds with 429 Too Many
Requests.\\n-\nOn the front end, all rule install and upgrade operations
are\nretried\\nin case of a 429 response. This ensures proper handling
when a\nuser\\nclicks multiple times an update or install rule
buttons\\n\\n-\n`prebuilt_rules/_bootstrap` endpoint\\n- Install
prebuilt rules and\nendpoint packages sequentially instead of\\nin
parallel to prevent from\nhaving them both downloaded into
memory\\nsimultaneously.\\n- Added a\n30-minute socket timeout to
prevent the proxy from closing\nthe\\nconnection while rule installation
is in progress.\\n- Introduced a\n`throttleRequests` wrapper, ensuring
the endpoint handler\\nis called\nonly once when multiple concurrent
requests are received.\\n- The first\nrequest triggers the handler,
while subsequent requests wait\\nfor the\nfirst one to complete and
reuse its result.\\n- This prevents costly\nprebuilt rule package
installation from running\\nin parallel.\\n- Reusing\nthe response
ensures the frontend correctly invalidates cached\\nprebuilt\nrule
queries. Since concurrent frontend requests should receive\\nthe\nsame
installed package information, responding with 421 and
using\\nthe\nretry logic as in cases above is not an option here because
the\\nsecond\nrequest would receive a package installation skipped
response\\nleading\nto no cache invalidation.\\n\\n-
`installation/_review` and\n`upgrade/_review` endpoints\\n- Concurrency
is limited to one parallel\ncall. If another call is
made\\nsimultaneously, the server responds with\n429 Too Many
Requests.\\n- On the front end, all rule install and upgrade\noperations
are retried\\nin case of a 429 response. This ensures proper\nhandling
when a user\\nclicks multiple times an update or install
rule\nbuttons\",\"sha\":\"c5557f33213f699acd9bb656af9166b1449d18f9\",\"branchLabelMapping\":{\"^v9.1.0$\":\"main\",\"^v8.19.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"release_note:fix\",\"performance\",\"v9.0.0\",\"Team:Detections\nand
Resp\",\"Team: SecuritySolution\",\"Team:Detection
Rule\nManagement\",\"Feature:Prebuilt
Detection\nRules\",\"backport:version\",\"v8.18.0\",\"v9.1.0\",\"v8.19.0\",\"v8.17.3\"],\"title\":\"[Security\nSolution]
Added concurrency limits and request throttling to
prebuilt\nrule\nroutes\",\"number\":209551,\"url\":\"https://github.com/elastic/kibana/pull/209551\",\"mergeCommit\":{\"message\":\"[Security\nSolution]
Added concurrency limits and request throttling to prebuilt\nrule routes
(#209551)\\n\\n**Resolves:\nhttps://github.com//issues/208357**\\n**Resolves:\nhttps://github.com//issues/208355**\\n\\n##
Summary \\n\\nTo\nprevent possible OOM errors, we need to limit
concurrent requests\nto\\nprebuilt rule routes (see attached tickets for
more details).\\n\\n-\n`installation/_perform` and `upgrade/_perform`
endpoints\\n- Concurrency\nis limited to one parallel call. If another
call is\nmade\\nsimultaneously, the server responds with 429 Too Many
Requests.\\n-\nOn the front end, all rule install and upgrade operations
are\nretried\\nin case of a 429 response. This ensures proper handling
when a\nuser\\nclicks multiple times an update or install rule
buttons\\n\\n-\n`prebuilt_rules/_bootstrap` endpoint\\n- Install
prebuilt rules and\nendpoint packages sequentially instead of\\nin
parallel to prevent from\nhaving them both downloaded into
memory\\nsimultaneously.\\n- Added a\n30-minute socket timeout to
prevent the proxy from closing\nthe\\nconnection while rule installation
is in progress.\\n- Introduced a\n`throttleRequests` wrapper, ensuring
the endpoint handler\\nis called\nonly once when multiple concurrent
requests are received.\\n- The first\nrequest triggers the handler,
while subsequent requests wait\\nfor the\nfirst one to complete and
reuse its result.\\n- This prevents costly\nprebuilt rule package
installation from running\\nin parallel.\\n- Reusing\nthe response
ensures the frontend correctly invalidates cached\\nprebuilt\nrule
queries. Since concurrent frontend requests should receive\\nthe\nsame
installed package information, responding with 421 and
using\\nthe\nretry logic as in cases above is not an option here because
the\\nsecond\nrequest would receive a package installation skipped
response\\nleading\nto no cache invalidation.\\n\\n-
`installation/_review` and\n`upgrade/_review` endpoints\\n- Concurrency
is limited to one parallel\ncall. If another call is
made\\nsimultaneously, the server responds with\n429 Too Many
Requests.\\n- On the front end, all rule install and upgrade\noperations
are retried\\nin case of a 429 response. This ensures proper\nhandling
when a user\\nclicks multiple times an update or install
rule\nbuttons\",\"sha\":\"c5557f33213f699acd9bb656af9166b1449d18f9\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[\"9.0\",\"8.18\",\"8.x\",\"8.17\"],\"targetPullRequestStates\":[{\"branch\":\"9.0\",\"label\":\"v9.0.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"8.18\",\"label\":\"v8.18.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"main\",\"label\":\"v9.1.0\",\"branchLabelMappingKey\":\"^v9.1.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/209551\",\"number\":209551,\"mergeCommit\":{\"message\":\"[Security\nSolution]
Added concurrency limits and request throttling to prebuilt\nrule routes
(#209551)\\n\\n**Resolves:\nhttps://github.com//issues/208357**\\n**Resolves:\nhttps://github.com//issues/208355**\\n\\n##
Summary \\n\\nTo\nprevent possible OOM errors, we need to limit
concurrent requests\nto\\nprebuilt rule routes (see attached tickets for
more details).\\n\\n-\n`installation/_perform` and `upgrade/_perform`
endpoints\\n- Concurrency\nis limited to one parallel call. If another
call is\nmade\\nsimultaneously, the server responds with 429 Too Many
Requests.\\n-\nOn the front end, all rule install and upgrade operations
are\nretried\\nin case of a 429 response. This ensures proper handling
when a\nuser\\nclicks multiple times an update or install rule
buttons\\n\\n-\n`prebuilt_rules/_bootstrap` endpoint\\n- Install
prebuilt rules and\nendpoint packages sequentially instead of\\nin
parallel to prevent from\nhaving them both downloaded into
memory\\nsimultaneously.\\n- Added a\n30-minute socket timeout to
prevent the proxy from closing\nthe\\nconnection while rule installation
is in progress.\\n- Introduced a\n`throttleRequests` wrapper, ensuring
the endpoint handler\\nis called\nonly once when multiple concurrent
requests are received.\\n- The first\nrequest triggers the handler,
while subsequent requests wait\\nfor the\nfirst one to complete and
reuse its result.\\n- This prevents costly\nprebuilt rule package
installation from running\\nin parallel.\\n- Reusing\nthe response
ensures the frontend correctly invalidates cached\\nprebuilt\nrule
queries. Since concurrent frontend requests should receive\\nthe\nsame
installed package information, responding with 421 and
using\\nthe\nretry logic as in cases above is not an option here because
the\\nsecond\nrequest would receive a package installation skipped
response\\nleading\nto no cache invalidation.\\n\\n-
`installation/_review` and\n`upgrade/_review` endpoints\\n- Concurrency
is limited to one parallel\ncall. If another call is
made\\nsimultaneously, the server responds with\n429 Too Many
Requests.\\n- On the front end, all rule install and upgrade\noperations
are retried\\nin case of a 429 response. This ensures proper\nhandling
when a user\\nclicks multiple times an update or install
rule\nbuttons\",\"sha\":\"c5557f33213f699acd9bb656af9166b1449d18f9\"}},{\"branch\":\"8.x\",\"label\":\"v8.19.0\",\"branchLabelMappingKey\":\"^v8.19.0$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"8.17\",\"label\":\"v8.17.3\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"}]}]\nBACKPORT-->\n\nCo-authored-by:
Dmitrii Shevchenko
<dmitrii.shevchenko@elastic.co>"}},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/210640","number":210640,"state":"MERGED","mergeCommit":{"sha":"2bd85b19aa84575ec0745ba2ad24d4c7718d824f","message":"[8.18]
[Security Solution] Added concurrency limits and request throttling to
prebuilt rule routes (#209551) (#210640)\n\n# Backport\n\nThis will
backport the following commits from `main` to `8.18`:\n- [[Security
Solution] Added concurrency limits and request throttling\nto prebuilt
rule
routes\n(#209551)](https://github.com/elastic/kibana/pull/209551)\n\n<!---
Backport version: 9.4.3 -->\n\n### Questions ?\nPlease refer to the
[Backport
tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT
[{\"author\":{\"name\":\"Dmitrii\nShevchenko\",\"email\":\"dmitrii.shevchenko@elastic.co\"},\"sourceCommit\":{\"committedDate\":\"2025-02-11T17:12:03Z\",\"message\":\"[Security\nSolution]
Added concurrency limits and request throttling to prebuilt\nrule routes
(#209551)\\n\\n**Resolves:\nhttps://github.com//issues/208357**\\n**Resolves:\nhttps://github.com//issues/208355**\\n\\n##
Summary \\n\\nTo\nprevent possible OOM errors, we need to limit
concurrent requests\nto\\nprebuilt rule routes (see attached tickets for
more details).\\n\\n-\n`installation/_perform` and `upgrade/_perform`
endpoints\\n- Concurrency\nis limited to one parallel call. If another
call is\nmade\\nsimultaneously, the server responds with 429 Too Many
Requests.\\n-\nOn the front end, all rule install and upgrade operations
are\nretried\\nin case of a 429 response. This ensures proper handling
when a\nuser\\nclicks multiple times an update or install rule
buttons\\n\\n-\n`prebuilt_rules/_bootstrap` endpoint\\n- Install
prebuilt rules and\nendpoint packages sequentially instead of\\nin
parallel to prevent from\nhaving them both downloaded into
memory\\nsimultaneously.\\n- Added a\n30-minute socket timeout to
prevent the proxy from closing\nthe\\nconnection while rule installation
is in progress.\\n- Introduced a\n`throttleRequests` wrapper, ensuring
the endpoint handler\\nis called\nonly once when multiple concurrent
requests are received.\\n- The first\nrequest triggers the handler,
while subsequent requests wait\\nfor the\nfirst one to complete and
reuse its result.\\n- This prevents costly\nprebuilt rule package
installation from running\\nin parallel.\\n- Reusing\nthe response
ensures the frontend correctly invalidates cached\\nprebuilt\nrule
queries. Since concurrent frontend requests should receive\\nthe\nsame
installed package information, responding with 421 and
using\\nthe\nretry logic as in cases above is not an option here because
the\\nsecond\nrequest would receive a package installation skipped
response\\nleading\nto no cache invalidation.\\n\\n-
`installation/_review` and\n`upgrade/_review` endpoints\\n- Concurrency
is limited to one parallel\ncall. If another call is
made\\nsimultaneously, the server responds with\n429 Too Many
Requests.\\n- On the front end, all rule install and upgrade\noperations
are retried\\nin case of a 429 response. This ensures proper\nhandling
when a user\\nclicks multiple times an update or install
rule\nbuttons\",\"sha\":\"c5557f33213f699acd9bb656af9166b1449d18f9\",\"branchLabelMapping\":{\"^v9.1.0$\":\"main\",\"^v8.19.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"release_note:fix\",\"performance\",\"v9.0.0\",\"Team:Detections\nand
Resp\",\"Team: SecuritySolution\",\"Team:Detection
Rule\nManagement\",\"Feature:Prebuilt
Detection\nRules\",\"backport:version\",\"v8.18.0\",\"v9.1.0\",\"v8.19.0\",\"v8.17.3\"],\"title\":\"[Security\nSolution]
Added concurrency limits and request throttling to
prebuilt\nrule\nroutes\",\"number\":209551,\"url\":\"https://github.com/elastic/kibana/pull/209551\",\"mergeCommit\":{\"message\":\"[Security\nSolution]
Added concurrency limits and request throttling to prebuilt\nrule routes
(#209551)\\n\\n**Resolves:\nhttps://github.com//issues/208357**\\n**Resolves:\nhttps://github.com//issues/208355**\\n\\n##
Summary \\n\\nTo\nprevent possible OOM errors, we need to limit
concurrent requests\nto\\nprebuilt rule routes (see attached tickets for
more details).\\n\\n-\n`installation/_perform` and `upgrade/_perform`
endpoints\\n- Concurrency\nis limited to one parallel call. If another
call is\nmade\\nsimultaneously, the server responds with 429 Too Many
Requests.\\n-\nOn the front end, all rule install and upgrade operations
are\nretried\\nin case of a 429 response. This ensures proper handling
when a\nuser\\nclicks multiple times an update or install rule
buttons\\n\\n-\n`prebuilt_rules/_bootstrap` endpoint\\n- Install
prebuilt rules and\nendpoint packages sequentially instead of\\nin
parallel to prevent from\nhaving them both downloaded into
memory\\nsimultaneously.\\n- Added a\n30-minute socket timeout to
prevent the proxy from closing\nthe\\nconnection while rule installation
is in progress.\\n- Introduced a\n`throttleRequests` wrapper, ensuring
the endpoint handler\\nis called\nonly once when multiple concurrent
requests are received.\\n- The first\nrequest triggers the handler,
while subsequent requests wait\\nfor the\nfirst one to complete and
reuse its result.\\n- This prevents costly\nprebuilt rule package
installation from running\\nin parallel.\\n- Reusing\nthe response
ensures the frontend correctly invalidates cached\\nprebuilt\nrule
queries. Since concurrent frontend requests should receive\\nthe\nsame
installed package information, responding with 421 and
using\\nthe\nretry logic as in cases above is not an option here because
the\\nsecond\nrequest would receive a package installation skipped
response\\nleading\nto no cache invalidation.\\n\\n-
`installation/_review` and\n`upgrade/_review` endpoints\\n- Concurrency
is limited to one parallel\ncall. If another call is
made\\nsimultaneously, the server responds with\n429 Too Many
Requests.\\n- On the front end, all rule install and upgrade\noperations
are retried\\nin case of a 429 response. This ensures proper\nhandling
when a user\\nclicks multiple times an update or install
rule\nbuttons\",\"sha\":\"c5557f33213f699acd9bb656af9166b1449d18f9\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[\"9.0\",\"8.18\",\"8.x\",\"8.17\"],\"targetPullRequestStates\":[{\"branch\":\"9.0\",\"label\":\"v9.0.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"8.18\",\"label\":\"v8.18.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"main\",\"label\":\"v9.1.0\",\"branchLabelMappingKey\":\"^v9.1.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/209551\",\"number\":209551,\"mergeCommit\":{\"message\":\"[Security\nSolution]
Added concurrency limits and request throttling to prebuilt\nrule routes
(#209551)\\n\\n**Resolves:\nhttps://github.com//issues/208357**\\n**Resolves:\nhttps://github.com//issues/208355**\\n\\n##
Summary \\n\\nTo\nprevent possible OOM errors, we need to limit
concurrent requests\nto\\nprebuilt rule routes (see attached tickets for
more details).\\n\\n-\n`installation/_perform` and `upgrade/_perform`
endpoints\\n- Concurrency\nis limited to one parallel call. If another
call is\nmade\\nsimultaneously, the server responds with 429 Too Many
Requests.\\n-\nOn the front end, all rule install and upgrade operations
are\nretried\\nin case of a 429 response. This ensures proper handling
when a\nuser\\nclicks multiple times an update or install rule
buttons\\n\\n-\n`prebuilt_rules/_bootstrap` endpoint\\n- Install
prebuilt rules and\nendpoint packages sequentially instead of\\nin
parallel to prevent from\nhaving them both downloaded into
memory\\nsimultaneously.\\n- Added a\n30-minute socket timeout to
prevent the proxy from closing\nthe\\nconnection while rule installation
is in progress.\\n- Introduced a\n`throttleRequests` wrapper, ensuring
the endpoint handler\\nis called\nonly once when multiple concurrent
requests are received.\\n- The first\nrequest triggers the handler,
while subsequent requests wait\\nfor the\nfirst one to complete and
reuse its result.\\n- This prevents costly\nprebuilt rule package
installation from running\\nin parallel.\\n- Reusing\nthe response
ensures the frontend correctly invalidates cached\\nprebuilt\nrule
queries. Since concurrent frontend requests should receive\\nthe\nsame
installed package information, responding with 421 and
using\\nthe\nretry logic as in cases above is not an option here because
the\\nsecond\nrequest would receive a package installation skipped
response\\nleading\nto no cache invalidation.\\n\\n-
`installation/_review` and\n`upgrade/_review` endpoints\\n- Concurrency
is limited to one parallel\ncall. If another call is
made\\nsimultaneously, the server responds with\n429 Too Many
Requests.\\n- On the front end, all rule install and upgrade\noperations
are retried\\nin case of a 429 response. This ensures proper\nhandling
when a user\\nclicks multiple times an update or install
rule\nbuttons\",\"sha\":\"c5557f33213f699acd9bb656af9166b1449d18f9\"}},{\"branch\":\"8.x\",\"label\":\"v8.19.0\",\"branchLabelMappingKey\":\"^v8.19.0$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"8.17\",\"label\":\"v8.17.3\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"}]}]\nBACKPORT-->\n\nCo-authored-by:
Dmitrii Shevchenko
<dmitrii.shevchenko@elastic.co>"}},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/209551","number":209551,"mergeCommit":{"message":"[Security
Solution] Added concurrency limits and request throttling to prebuilt
rule routes (#209551)\n\n**Resolves:
https://github.com/elastic/kibana/issues/208357**\n**Resolves:
https://github.com/elastic/kibana/issues/208355**\n\n## Summary \n\nTo
prevent possible OOM errors, we need to limit concurrent requests
to\nprebuilt rule routes (see attached tickets for more details).\n\n-
`installation/_perform` and `upgrade/_perform` endpoints\n- Concurrency
is limited to one parallel call. If another call is
made\nsimultaneously, the server responds with 429 Too Many Requests.\n-
On the front end, all rule install and upgrade operations are
retried\nin case of a 429 response. This ensures proper handling when a
user\nclicks multiple times an update or install rule buttons\n\n-
`prebuilt_rules/_bootstrap` endpoint\n- Install prebuilt rules and
endpoint packages sequentially instead of\nin parallel to prevent from
having them both downloaded into memory\nsimultaneously.\n- Added a
30-minute socket timeout to prevent the proxy from closing
the\nconnection while rule installation is in progress.\n- Introduced a
`throttleRequests` wrapper, ensuring the endpoint handler\nis called
only once when multiple concurrent requests are received.\n- The first
request triggers the handler, while subsequent requests wait\nfor the
first one to complete and reuse its result.\n- This prevents costly
prebuilt rule package installation from running\nin parallel.\n- Reusing
the response ensures the frontend correctly invalidates cached\nprebuilt
rule queries. Since concurrent frontend requests should receive\nthe
same installed package information, responding with 421 and using\nthe
retry logic as in cases above is not an option here because the\nsecond
request would receive a package installation skipped response\nleading
to no cache invalidation.\n\n- `installation/_review` and
`upgrade/_review` endpoints\n- Concurrency is limited to one parallel
call. If another call is made\nsimultaneously, the server responds with
429 Too Many Requests.\n- On the front end, all rule install and upgrade
operations are retried\nin case of a 429 response. This ensures proper
handling when a user\nclicks multiple times an update or install rule
buttons","sha":"c5557f33213f699acd9bb656af9166b1449d18f9"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/210641","number":210641,"state":"MERGED","mergeCommit":{"sha":"8018c82f7d92b1d4798dea3c0a677bae740d4f8a","message":"[8.x]
[Security Solution] Added concurrency limits and request throttling to
prebuilt rule routes (#209551) (#210641)\n\n# Backport\n\nThis will
backport the following commits from `main` to `8.x`:\n- [[Security
Solution] Added concurrency limits and request throttling\nto prebuilt
rule
routes\n(#209551)](https://github.com/elastic/kibana/pull/209551)\n\n<!---
Backport version: 9.4.3 -->\n\n### Questions ?\nPlease refer to the
[Backport
tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT
[{\"author\":{\"name\":\"Dmitrii\nShevchenko\",\"email\":\"dmitrii.shevchenko@elastic.co\"},\"sourceCommit\":{\"committedDate\":\"2025-02-11T17:12:03Z\",\"message\":\"[Security\nSolution]
Added concurrency limits and request throttling to prebuilt\nrule routes
(#209551)\\n\\n**Resolves:\nhttps://github.com//issues/208357**\\n**Resolves:\nhttps://github.com//issues/208355**\\n\\n##
Summary \\n\\nTo\nprevent possible OOM errors, we need to limit
concurrent requests\nto\\nprebuilt rule routes (see attached tickets for
more details).\\n\\n-\n`installation/_perform` and `upgrade/_perform`
endpoints\\n- Concurrency\nis limited to one parallel call. If another
call is\nmade\\nsimultaneously, the server responds with 429 Too Many
Requests.\\n-\nOn the front end, all rule install and upgrade operations
are\nretried\\nin case of a 429 response. This ensures proper handling
when a\nuser\\nclicks multiple times an update or install rule
buttons\\n\\n-\n`prebuilt_rules/_bootstrap` endpoint\\n- Install
prebuilt rules and\nendpoint packages sequentially instead of\\nin
parallel to prevent from\nhaving them both downloaded into
memory\\nsimultaneously.\\n- Added a\n30-minute socket timeout to
prevent the proxy from closing\nthe\\nconnection while rule installation
is in progress.\\n- Introduced a\n`throttleRequests` wrapper, ensuring
the endpoint handler\\nis called\nonly once when multiple concurrent
requests are received.\\n- The first\nrequest triggers the handler,
while subsequent requests wait\\nfor the\nfirst one to complete and
reuse its result.\\n- This prevents costly\nprebuilt rule package
installation from running\\nin parallel.\\n- Reusing\nthe response
ensures the frontend correctly invalidates cached\\nprebuilt\nrule
queries. Since concurrent frontend requests should receive\\nthe\nsame
installed package information, responding with 421 and
using\\nthe\nretry logic as in cases above is not an option here because
the\\nsecond\nrequest would receive a package installation skipped
response\\nleading\nto no cache invalidation.\\n\\n-
`installation/_review` and\n`upgrade/_review` endpoints\\n- Concurrency
is limited to one parallel\ncall. If another call is
made\\nsimultaneously, the server responds with\n429 Too Many
Requests.\\n- On the front end, all rule install and upgrade\noperations
are retried\\nin case of a 429 response. This ensures proper\nhandling
when a user\\nclicks multiple times an update or install
rule\nbuttons\",\"sha\":\"c5557f33213f699acd9bb656af9166b1449d18f9\",\"branchLabelMapping\":{\"^v9.1.0$\":\"main\",\"^v8.19.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"release_note:fix\",\"performance\",\"v9.0.0\",\"Team:Detections\nand
Resp\",\"Team: SecuritySolution\",\"Team:Detection
Rule\nManagement\",\"Feature:Prebuilt
Detection\nRules\",\"backport:version\",\"v8.18.0\",\"v9.1.0\",\"v8.19.0\",\"v8.17.3\"],\"title\":\"[Security\nSolution]
Added concurrency limits and request throttling to
prebuilt\nrule\nroutes\",\"number\":209551,\"url\":\"https://github.com/elastic/kibana/pull/209551\",\"mergeCommit\":{\"message\":\"[Security\nSolution]
Added concurrency limits and request throttling to prebuilt\nrule routes
(#209551)\\n\\n**Resolves:\nhttps://github.com//issues/208357**\\n**Resolves:\nhttps://github.com//issues/208355**\\n\\n##
Summary \\n\\nTo\nprevent possible OOM errors, we need to limit
concurrent requests\nto\\nprebuilt rule routes (see attached tickets for
more details).\\n\\n-\n`installation/_perform` and `upgrade/_perform`
endpoints\\n- Concurrency\nis limited to one parallel call. If another
call is\nmade\\nsimultaneously, the server responds with 429 Too Many
Requests.\\n-\nOn the front end, all rule install and upgrade operations
are\nretried\\nin case of a 429 response. This ensures proper handling
when a\nuser\\nclicks multiple times an update or install rule
buttons\\n\\n-\n`prebuilt_rules/_bootstrap` endpoint\\n- Install
prebuilt rules and\nendpoint packages sequentially instead of\\nin
parallel to prevent from\nhaving them both downloaded into
memory\\nsimultaneously.\\n- Added a\n30-minute socket timeout to
prevent the proxy from closing\nthe\\nconnection while rule installation
is in progress.\\n- Introduced a\n`throttleRequests` wrapper, ensuring
the endpoint handler\\nis called\nonly once when multiple concurrent
requests are received.\\n- The first\nrequest triggers the handler,
while subsequent requests wait\\nfor the\nfirst one to complete and
reuse its result.\\n- This prevents costly\nprebuilt rule package
installation from running\\nin parallel.\\n- Reusing\nthe response
ensures the frontend correctly invalidates cached\\nprebuilt\nrule
queries. Since concurrent frontend requests should receive\\nthe\nsame
installed package information, responding with 421 and
using\\nthe\nretry logic as in cases above is not an option here because
the\\nsecond\nrequest would receive a package installation skipped
response\\nleading\nto no cache invalidation.\\n\\n-
`installation/_review` and\n`upgrade/_review` endpoints\\n- Concurrency
is limited to one parallel\ncall. If another call is
made\\nsimultaneously, the server responds with\n429 Too Many
Requests.\\n- On the front end, all rule install and upgrade\noperations
are retried\\nin case of a 429 response. This ensures proper\nhandling
when a user\\nclicks multiple times an update or install
rule\nbuttons\",\"sha\":\"c5557f33213f699acd9bb656af9166b1449d18f9\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[\"9.0\",\"8.18\",\"8.x\",\"8.17\"],\"targetPullRequestStates\":[{\"branch\":\"9.0\",\"label\":\"v9.0.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"8.18\",\"label\":\"v8.18.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"main\",\"label\":\"v9.1.0\",\"branchLabelMappingKey\":\"^v9.1.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/209551\",\"number\":209551,\"mergeCommit\":{\"message\":\"[Security\nSolution]
Added concurrency limits and request throttling to prebuilt\nrule routes
(#209551)\\n\\n**Resolves:\nhttps://github.com//issues/208357**\\n**Resolves:\nhttps://github.com//issues/208355**\\n\\n##
Summary \\n\\nTo\nprevent possible OOM errors, we need to limit
concurrent requests\nto\\nprebuilt rule routes (see attached tickets for
more details).\\n\\n-\n`installation/_perform` and `upgrade/_perform`
endpoints\\n- Concurrency\nis limited to one parallel call. If another
call is\nmade\\nsimultaneously, the server responds with 429 Too Many
Requests.\\n-\nOn the front end, all rule install and upgrade operations
are\nretried\\nin case of a 429 response. This ensures proper handling
when a\nuser\\nclicks multiple times an update or install rule
buttons\\n\\n-\n`prebuilt_rules/_bootstrap` endpoint\\n- Install
prebuilt rules and\nendpoint packages sequentially instead of\\nin
parallel to prevent from\nhaving them both downloaded into
memory\\nsimultaneously.\\n- Added a\n30-minute socket timeout to
prevent the proxy from closing\nthe\\nconnection while rule installation
is in progress.\\n- Introduced a\n`throttleRequests` wrapper, ensuring
the endpoint handler\\nis called\nonly once when multiple concurrent
requests are received.\\n- The first\nrequest triggers the handler,
while subsequent requests wait\\nfor the\nfirst one to complete and
reuse its result.\\n- This prevents costly\nprebuilt rule package
installation from running\\nin parallel.\\n- Reusing\nthe response
ensures the frontend correctly invalidates cached\\nprebuilt\nrule
queries. Since concurrent frontend requests should receive\\nthe\nsame
installed package information, responding with 421 and
using\\nthe\nretry logic as in cases above is not an option here because
the\\nsecond\nrequest would receive a package installation skipped
response\\nleading\nto no cache invalidation.\\n\\n-
`installation/_review` and\n`upgrade/_review` endpoints\\n- Concurrency
is limited to one parallel\ncall. If another call is
made\\nsimultaneously, the server responds with\n429 Too Many
Requests.\\n- On the front end, all rule install and upgrade\noperations
are retried\\nin case of a 429 response. This ensures proper\nhandling
when a user\\nclicks multiple times an update or install
rule\nbuttons\",\"sha\":\"c5557f33213f699acd9bb656af9166b1449d18f9\"}},{\"branch\":\"8.x\",\"label\":\"v8.19.0\",\"branchLabelMappingKey\":\"^v8.19.0$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"8.17\",\"label\":\"v8.17.3\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"}]}]\nBACKPORT-->\n\nCo-authored-by:
Dmitrii Shevchenko
<dmitrii.shevchenko@elastic.co>"}},{"branch":"8.17","label":"v8.17.3","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
  • Loading branch information
xcrzx authored Feb 12, 2025
1 parent 3014b85 commit 6259401
Show file tree
Hide file tree
Showing 21 changed files with 565 additions and 263 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

const MAX_BACKOFF = 30000;

/**
* Calculates a backoff delay using an exponential growth formula, capped by a
* predefined maximum value.
*
* @param failedAttempts - The number of consecutive failed attempts.
* @returns The calculated backoff delay, in milliseconds.
*/
export const cappedExponentialBackoff = (failedAttempts: number) => {
const backoff = Math.min(1000 * 2 ** failedAttempts, MAX_BACKOFF);
return backoff;
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { get } from 'lodash';

/*
Prebuilt rule operations like install and upgrade are rate limited to one at a
time. In most cases it is fine to wait for the other operation to finish using
the retry logic.
429 can be caused by a user clicking multiple times on the install or upgrade
rule buttons and in most cases the operations can be performed in succession
without any conflicts.
*/
export const retryOnRateLimitedError = (failureCount: number, error: unknown) => {
const statusCode = get(error, 'response.status');
return statusCode === 429;
};
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,18 @@
*/
import type { UseMutationOptions } from '@tanstack/react-query';
import { useMutation } from '@tanstack/react-query';
import { BOOTSTRAP_PREBUILT_RULES_URL } from '../../../../../common/api/detection_engine';
import type { BootstrapPrebuiltRulesResponse } from '../../../../../common/api/detection_engine/prebuilt_rules/bootstrap_prebuilt_rules/bootstrap_prebuilt_rules.gen';
import { PREBUILT_RULES_PACKAGE_NAME } from '../../../../../common/detection_engine/constants';
import { bootstrapPrebuiltRules } from '../api';
import { useInvalidateFetchPrebuiltRulesInstallReviewQuery } from './prebuilt_rules/use_fetch_prebuilt_rules_install_review_query';
import { useInvalidateFetchPrebuiltRulesStatusQuery } from './prebuilt_rules/use_fetch_prebuilt_rules_status_query';
import { useInvalidateFetchPrebuiltRulesUpgradeReviewQuery } from './prebuilt_rules/use_fetch_prebuilt_rules_upgrade_review_query';
import { BOOTSTRAP_PREBUILT_RULES_URL } from '../../../../../../common/api/detection_engine';
import type { BootstrapPrebuiltRulesResponse } from '../../../../../../common/api/detection_engine/prebuilt_rules/bootstrap_prebuilt_rules/bootstrap_prebuilt_rules.gen';
import { PREBUILT_RULES_PACKAGE_NAME } from '../../../../../../common/detection_engine/constants';
import { bootstrapPrebuiltRules } from '../../api';
import { useInvalidateFetchPrebuiltRulesInstallReviewQuery } from './use_fetch_prebuilt_rules_install_review_query';
import { useInvalidateFetchPrebuiltRulesStatusQuery } from './use_fetch_prebuilt_rules_status_query';
import { useInvalidateFetchPrebuiltRulesUpgradeReviewQuery } from './use_fetch_prebuilt_rules_upgrade_review_query';

export const BOOTSTRAP_PREBUILT_RULES_KEY = ['POST', BOOTSTRAP_PREBUILT_RULES_URL];

export const useBootstrapPrebuiltRulesMutation = (
options?: UseMutationOptions<BootstrapPrebuiltRulesResponse, Error>
options?: UseMutationOptions<BootstrapPrebuiltRulesResponse>
) => {
const invalidatePrePackagedRulesStatus = useInvalidateFetchPrebuiltRulesStatusQuery();
const invalidatePrebuiltRulesInstallReview = useInvalidateFetchPrebuiltRulesInstallReviewQuery();
Expand All @@ -26,7 +26,7 @@ export const useBootstrapPrebuiltRulesMutation = (
return useMutation(() => bootstrapPrebuiltRules(), {
...options,
mutationKey: BOOTSTRAP_PREBUILT_RULES_KEY,
onSettled: (...args) => {
onSuccess: (...args) => {
const response = args[0];
if (
response?.packages.find((pkg) => pkg.name === PREBUILT_RULES_PACKAGE_NAME)?.status ===
Expand All @@ -40,8 +40,8 @@ export const useBootstrapPrebuiltRulesMutation = (
invalidatePrebuiltRulesUpdateReview();
}

if (options?.onSettled) {
options.onSettled(...args);
if (options?.onSuccess) {
options.onSuccess(...args);
}
},
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ import { reviewRuleInstall } from '../../api';
import { REVIEW_RULE_INSTALLATION_URL } from '../../../../../../common/api/detection_engine/prebuilt_rules/urls';
import type { ReviewRuleInstallationResponseBody } from '../../../../../../common/api/detection_engine/prebuilt_rules';
import { DEFAULT_QUERY_OPTIONS } from '../constants';
import { retryOnRateLimitedError } from './retry_on_rate_limited_error';
import { cappedExponentialBackoff } from './capped_exponential_backoff';

export const REVIEW_RULE_INSTALLATION_QUERY_KEY = ['POST', REVIEW_RULE_INSTALLATION_URL];

Expand All @@ -26,6 +28,8 @@ export const useFetchPrebuiltRulesInstallReviewQuery = (
{
...DEFAULT_QUERY_OPTIONS,
...options,
retry: retryOnRateLimitedError,
retryDelay: cappedExponentialBackoff,
}
);
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ import { reviewRuleUpgrade } from '../../api';
import { REVIEW_RULE_UPGRADE_URL } from '../../../../../../common/api/detection_engine/prebuilt_rules/urls';
import type { ReviewRuleUpgradeResponseBody } from '../../../../../../common/api/detection_engine/prebuilt_rules';
import { DEFAULT_QUERY_OPTIONS } from '../constants';
import { retryOnRateLimitedError } from './retry_on_rate_limited_error';
import { cappedExponentialBackoff } from './capped_exponential_backoff';

export const REVIEW_RULE_UPGRADE_QUERY_KEY = ['POST', REVIEW_RULE_UPGRADE_URL];

Expand All @@ -26,6 +28,8 @@ export const useFetchPrebuiltRulesUpgradeReviewQuery = (
{
...DEFAULT_QUERY_OPTIONS,
...options,
retry: retryOnRateLimitedError,
retryDelay: cappedExponentialBackoff,
}
);
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,15 @@ import type { UseMutationOptions } from '@tanstack/react-query';
import { useMutation } from '@tanstack/react-query';
import type { PerformRuleInstallationResponseBody } from '../../../../../../common/api/detection_engine/prebuilt_rules';
import { PERFORM_RULE_INSTALLATION_URL } from '../../../../../../common/api/detection_engine/prebuilt_rules/urls';
import { useInvalidateFetchPrebuiltRulesStatusQuery } from './use_fetch_prebuilt_rules_status_query';
import { useInvalidateFindRulesQuery } from '../use_find_rules_query';
import { performInstallAllRules } from '../../api';
import { useInvalidateFetchCoverageOverviewQuery } from '../use_fetch_coverage_overview_query';
import { useInvalidateFetchRuleManagementFiltersQuery } from '../use_fetch_rule_management_filters_query';
import { useInvalidateFetchRulesSnoozeSettingsQuery } from '../use_fetch_rules_snooze_settings_query';
import { useInvalidateFindRulesQuery } from '../use_find_rules_query';
import { retryOnRateLimitedError } from './retry_on_rate_limited_error';
import { useInvalidateFetchPrebuiltRulesInstallReviewQuery } from './use_fetch_prebuilt_rules_install_review_query';
import { performInstallAllRules } from '../../api';
import { useInvalidateFetchCoverageOverviewQuery } from '../use_fetch_coverage_overview_query';
import { useInvalidateFetchPrebuiltRulesStatusQuery } from './use_fetch_prebuilt_rules_status_query';
import { cappedExponentialBackoff } from './capped_exponential_backoff';

export const PERFORM_ALL_RULES_INSTALLATION_KEY = [
'POST',
Expand All @@ -23,7 +25,7 @@ export const PERFORM_ALL_RULES_INSTALLATION_KEY = [
];

export const usePerformAllRulesInstallMutation = (
options?: UseMutationOptions<PerformRuleInstallationResponseBody, Error>
options?: UseMutationOptions<PerformRuleInstallationResponseBody>
) => {
const invalidateFindRulesQuery = useInvalidateFindRulesQuery();
const invalidateFetchRulesSnoozeSettings = useInvalidateFetchRulesSnoozeSettingsQuery();
Expand All @@ -33,7 +35,7 @@ export const usePerformAllRulesInstallMutation = (
const invalidateRuleStatus = useInvalidateFetchPrebuiltRulesStatusQuery();
const invalidateFetchCoverageOverviewQuery = useInvalidateFetchCoverageOverviewQuery();

return useMutation<PerformRuleInstallationResponseBody, Error>(() => performInstallAllRules(), {
return useMutation<PerformRuleInstallationResponseBody>(() => performInstallAllRules(), {
...options,
mutationKey: PERFORM_ALL_RULES_INSTALLATION_KEY,
onSettled: (...args) => {
Expand All @@ -49,5 +51,7 @@ export const usePerformAllRulesInstallMutation = (
options.onSettled(...args);
}
},
retry: retryOnRateLimitedError,
retryDelay: cappedExponentialBackoff,
});
};
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,17 @@ import type {
PerformRuleInstallationResponseBody,
} from '../../../../../../common/api/detection_engine/prebuilt_rules';
import { PERFORM_RULE_INSTALLATION_URL } from '../../../../../../common/api/detection_engine/prebuilt_rules/urls';
import { useInvalidateFetchPrebuiltRulesStatusQuery } from './use_fetch_prebuilt_rules_status_query';
import { useInvalidateFindRulesQuery } from '../use_find_rules_query';
import { useInvalidateFetchRuleManagementFiltersQuery } from '../use_fetch_rule_management_filters_query';
import { useInvalidateFetchRulesSnoozeSettingsQuery } from '../use_fetch_rules_snooze_settings_query';
import { useInvalidateFetchPrebuiltRulesInstallReviewQuery } from './use_fetch_prebuilt_rules_install_review_query';
import type { BulkAction } from '../../api';
import { performInstallSpecificRules } from '../../api';
import { useInvalidateFetchCoverageOverviewQuery } from '../use_fetch_coverage_overview_query';
import { useBulkActionMutation } from '../use_bulk_action_mutation';
import { useInvalidateFetchCoverageOverviewQuery } from '../use_fetch_coverage_overview_query';
import { useInvalidateFetchRuleManagementFiltersQuery } from '../use_fetch_rule_management_filters_query';
import { useInvalidateFetchRulesSnoozeSettingsQuery } from '../use_fetch_rules_snooze_settings_query';
import { useInvalidateFindRulesQuery } from '../use_find_rules_query';
import { retryOnRateLimitedError } from './retry_on_rate_limited_error';
import { useInvalidateFetchPrebuiltRulesInstallReviewQuery } from './use_fetch_prebuilt_rules_install_review_query';
import { useInvalidateFetchPrebuiltRulesStatusQuery } from './use_fetch_prebuilt_rules_status_query';
import { cappedExponentialBackoff } from './capped_exponential_backoff';

export const PERFORM_SPECIFIC_RULES_INSTALLATION_KEY = [
'POST',
Expand All @@ -35,7 +37,7 @@ export interface UsePerformSpecificRulesInstallParams {
export const usePerformSpecificRulesInstallMutation = (
options?: UseMutationOptions<
PerformRuleInstallationResponseBody,
Error,
unknown,
UsePerformSpecificRulesInstallParams
>
) => {
Expand All @@ -51,7 +53,7 @@ export const usePerformSpecificRulesInstallMutation = (

return useMutation<
PerformRuleInstallationResponseBody,
Error,
unknown,
UsePerformSpecificRulesInstallParams
>(
(rulesToInstall: UsePerformSpecificRulesInstallParams) =>
Expand Down Expand Up @@ -81,6 +83,8 @@ export const usePerformSpecificRulesInstallMutation = (
options.onSettled(...args);
}
},
retry: retryOnRateLimitedError,
retryDelay: cappedExponentialBackoff,
}
);
};
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,15 @@ import type {
UpgradeSpecificRulesRequest,
} from '../../../../../../common/api/detection_engine/prebuilt_rules';
import { PERFORM_RULE_UPGRADE_URL } from '../../../../../../common/api/detection_engine/prebuilt_rules/urls';
import { useInvalidateFetchPrebuiltRulesStatusQuery } from './use_fetch_prebuilt_rules_status_query';
import { useInvalidateFindRulesQuery } from '../use_find_rules_query';
import { performUpgradeSpecificRules } from '../../api';
import { useInvalidateFetchCoverageOverviewQuery } from '../use_fetch_coverage_overview_query';
import { useInvalidateFetchRuleManagementFiltersQuery } from '../use_fetch_rule_management_filters_query';
import { useInvalidateFetchRulesSnoozeSettingsQuery } from '../use_fetch_rules_snooze_settings_query';
import { performUpgradeSpecificRules } from '../../api';
import { useInvalidateFindRulesQuery } from '../use_find_rules_query';
import { useInvalidateFetchPrebuiltRulesStatusQuery } from './use_fetch_prebuilt_rules_status_query';
import { useInvalidateFetchPrebuiltRulesUpgradeReviewQuery } from './use_fetch_prebuilt_rules_upgrade_review_query';
import { useInvalidateFetchCoverageOverviewQuery } from '../use_fetch_coverage_overview_query';
import { retryOnRateLimitedError } from './retry_on_rate_limited_error';
import { cappedExponentialBackoff } from './capped_exponential_backoff';

export const PERFORM_SPECIFIC_RULES_UPGRADE_KEY = [
'POST',
Expand Down Expand Up @@ -64,6 +66,8 @@ export const usePerformSpecificRulesUpgradeMutation = (
options.onSettled(...args);
}
},
retry: retryOnRateLimitedError,
retryDelay: cappedExponentialBackoff,
}
);
};
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import { useEffect } from 'react';
import {
BOOTSTRAP_PREBUILT_RULES_KEY,
useBootstrapPrebuiltRulesMutation,
} from '../api/hooks/use_bootstrap_prebuilt_rules';
} from '../api/hooks/prebuilt_rules/use_bootstrap_prebuilt_rules';

/**
* Install or upgrade the security packages (endpoint and prebuilt rules)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,8 @@ export const AddPrebuiltRulesTableContextProvider = ({
rules: [{ rule_id: ruleId, version: rule.version }],
enable,
});
} catch {
// Error is handled by the mutation's onError callback, so no need to do anything here
} finally {
setLoadingRules((prev) => prev.filter((id) => id !== ruleId));
}
Expand All @@ -182,6 +184,8 @@ export const AddPrebuiltRulesTableContextProvider = ({
setLoadingRules((prev) => [...prev, ...rulesToUpgrade.map((r) => r.rule_id)]);
try {
await installSpecificRulesRequest({ rules: rulesToUpgrade, enable });
} catch {
// Error is handled by the mutation's onError callback, so no need to do anything here
} finally {
setLoadingRules((prev) =>
prev.filter((id) => !rulesToUpgrade.some((r) => r.rule_id === id))
Expand All @@ -197,6 +201,8 @@ export const AddPrebuiltRulesTableContextProvider = ({
setLoadingRules((prev) => [...prev, ...rules.map((r) => r.rule_id)]);
try {
await installAllRulesRequest();
} catch {
// Error is handled by the mutation's onError callback, so no need to do anything here
} finally {
setLoadingRules([]);
setSelectedRules([]);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,11 @@
* 2.0.
*/

import { transformError } from '@kbn/securitysolution-es-utils';
import type { IKibanaResponse } from '@kbn/core/server';
import { BOOTSTRAP_PREBUILT_RULES_URL } from '../../../../../../common/api/detection_engine/prebuilt_rules';
import type { BootstrapPrebuiltRulesResponse } from '../../../../../../common/api/detection_engine/prebuilt_rules/bootstrap_prebuilt_rules/bootstrap_prebuilt_rules.gen';
import type { SecuritySolutionPluginRouter } from '../../../../../types';
import { buildSiemResponse } from '../../../routes/utils';
import {
installEndpointPackage,
installPrebuiltRulesPackage,
} from '../install_prebuilt_rules_and_timelines/install_prebuilt_rules_package';
import { PREBUILT_RULES_OPERATION_SOCKET_TIMEOUT_MS } from '../../constants';
import { bootstrapPrebuiltRulesHandler } from './bootstrap_prebuilt_rules_handler';
import { throttleRequests } from '../../../../../utils/throttle_requests';

export const bootstrapPrebuiltRulesRoute = (router: SecuritySolutionPluginRouter) => {
router.versioned
Expand All @@ -26,43 +21,17 @@ export const bootstrapPrebuiltRulesRoute = (router: SecuritySolutionPluginRouter
requiredPrivileges: ['securitySolution'],
},
},
options: {
timeout: {
idleSocket: PREBUILT_RULES_OPERATION_SOCKET_TIMEOUT_MS,
},
},
})
.addVersion(
{
version: '1',
validate: {},
},
async (context, _, response): Promise<IKibanaResponse<BootstrapPrebuiltRulesResponse>> => {
const siemResponse = buildSiemResponse(response);

try {
const ctx = await context.resolve(['securitySolution']);
const securityContext = ctx.securitySolution;
const config = securityContext.getConfig();

const results = await Promise.all([
installPrebuiltRulesPackage(config, securityContext),
installEndpointPackage(config, securityContext),
]);

const responseBody: BootstrapPrebuiltRulesResponse = {
packages: results.map((result) => ({
name: result.package.name,
version: result.package.version,
status: result.status,
})),
};

return response.ok({
body: responseBody,
});
} catch (err) {
const error = transformError(err);
return siemResponse.error({
body: error.message,
statusCode: error.statusCode,
});
}
}
throttleRequests(bootstrapPrebuiltRulesHandler)
);
};
Loading

0 comments on commit 6259401

Please sign in to comment.