Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[8.17] [Security Solution] Added concurrency limits and request throt…
…tling to prebuilt rule routes (#209551) (#210773) # Backport This will backport the following commits from `main` to `8.17`: - [[Security Solution] Added concurrency limits and request throttling to prebuilt rule routes (#209551)](#209551) <!--- Backport version: 9.6.4 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Dmitrii Shevchenko","email":"dmitrii.shevchenko@elastic.co"},"sourceCommit":{"committedDate":"2025-02-11T17:12:03Z","message":"[Security Solution] Added concurrency limits and request throttling to prebuilt rule routes (#209551)\n\n**Resolves: https://github.com/elastic/kibana/issues/208357**\n**Resolves: https://github.com/elastic/kibana/issues/208355**\n\n## Summary \n\nTo prevent possible OOM errors, we need to limit concurrent requests to\nprebuilt rule routes (see attached tickets for more details).\n\n- `installation/_perform` and `upgrade/_perform` endpoints\n- Concurrency is limited to one parallel call. If another call is made\nsimultaneously, the server responds with 429 Too Many Requests.\n- On the front end, all rule install and upgrade operations are retried\nin case of a 429 response. This ensures proper handling when a user\nclicks multiple times an update or install rule buttons\n\n- `prebuilt_rules/_bootstrap` endpoint\n- Install prebuilt rules and endpoint packages sequentially instead of\nin parallel to prevent from having them both downloaded into memory\nsimultaneously.\n- Added a 30-minute socket timeout to prevent the proxy from closing the\nconnection while rule installation is in progress.\n- Introduced a `throttleRequests` wrapper, ensuring the endpoint handler\nis called only once when multiple concurrent requests are received.\n- The first request triggers the handler, while subsequent requests wait\nfor the first one to complete and reuse its result.\n- This prevents costly prebuilt rule package installation from running\nin parallel.\n- Reusing the response ensures the frontend correctly invalidates cached\nprebuilt rule queries. Since concurrent frontend requests should receive\nthe same installed package information, responding with 421 and using\nthe retry logic as in cases above is not an option here because the\nsecond request would receive a package installation skipped response\nleading to no cache invalidation.\n\n- `installation/_review` and `upgrade/_review` endpoints\n- Concurrency is limited to one parallel call. If another call is made\nsimultaneously, the server responds with 429 Too Many Requests.\n- On the front end, all rule install and upgrade operations are retried\nin case of a 429 response. This ensures proper handling when a user\nclicks multiple times an update or install rule buttons","sha":"c5557f33213f699acd9bb656af9166b1449d18f9","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","performance","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0","v8.17.3"],"title":"[Security Solution] Added concurrency limits and request throttling to prebuilt rule routes","number":209551,"url":"https://github.com/elastic/kibana/pull/209551","mergeCommit":{"message":"[Security Solution] Added concurrency limits and request throttling to prebuilt rule routes (#209551)\n\n**Resolves: https://github.com/elastic/kibana/issues/208357**\n**Resolves: https://github.com/elastic/kibana/issues/208355**\n\n## Summary \n\nTo prevent possible OOM errors, we need to limit concurrent requests to\nprebuilt rule routes (see attached tickets for more details).\n\n- `installation/_perform` and `upgrade/_perform` endpoints\n- Concurrency is limited to one parallel call. If another call is made\nsimultaneously, the server responds with 429 Too Many Requests.\n- On the front end, all rule install and upgrade operations are retried\nin case of a 429 response. This ensures proper handling when a user\nclicks multiple times an update or install rule buttons\n\n- `prebuilt_rules/_bootstrap` endpoint\n- Install prebuilt rules and endpoint packages sequentially instead of\nin parallel to prevent from having them both downloaded into memory\nsimultaneously.\n- Added a 30-minute socket timeout to prevent the proxy from closing the\nconnection while rule installation is in progress.\n- Introduced a `throttleRequests` wrapper, ensuring the endpoint handler\nis called only once when multiple concurrent requests are received.\n- The first request triggers the handler, while subsequent requests wait\nfor the first one to complete and reuse its result.\n- This prevents costly prebuilt rule package installation from running\nin parallel.\n- Reusing the response ensures the frontend correctly invalidates cached\nprebuilt rule queries. Since concurrent frontend requests should receive\nthe same installed package information, responding with 421 and using\nthe retry logic as in cases above is not an option here because the\nsecond request would receive a package installation skipped response\nleading to no cache invalidation.\n\n- `installation/_review` and `upgrade/_review` endpoints\n- Concurrency is limited to one parallel call. If another call is made\nsimultaneously, the server responds with 429 Too Many Requests.\n- On the front end, all rule install and upgrade operations are retried\nin case of a 429 response. This ensures proper handling when a user\nclicks multiple times an update or install rule buttons","sha":"c5557f33213f699acd9bb656af9166b1449d18f9"}},"sourceBranch":"main","suggestedTargetBranches":["8.17"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/210642","number":210642,"state":"MERGED","mergeCommit":{"sha":"df87081d8a574c6b0bb2d9fc026776502622dc11","message":"[9.0] [Security Solution] Added concurrency limits and request throttling to prebuilt rule routes (#209551) (#210642)\n\n# Backport\n\nThis will backport the following commits from `main` to `9.0`:\n- [[Security Solution] Added concurrency limits and request throttling\nto prebuilt rule routes\n(#209551)](https://github.com/elastic/kibana/pull/209551)\n\n<!--- Backport version: 9.4.3 -->\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT [{\"author\":{\"name\":\"Dmitrii\nShevchenko\",\"email\":\"dmitrii.shevchenko@elastic.co\"},\"sourceCommit\":{\"committedDate\":\"2025-02-11T17:12:03Z\",\"message\":\"[Security\nSolution] Added concurrency limits and request throttling to prebuilt\nrule routes (#209551)\\n\\n**Resolves:\nhttps://github.com//issues/208357**\\n**Resolves:\nhttps://github.com//issues/208355**\\n\\n## Summary \\n\\nTo\nprevent possible OOM errors, we need to limit concurrent requests\nto\\nprebuilt rule routes (see attached tickets for more details).\\n\\n-\n`installation/_perform` and `upgrade/_perform` endpoints\\n- Concurrency\nis limited to one parallel call. If another call is\nmade\\nsimultaneously, the server responds with 429 Too Many Requests.\\n-\nOn the front end, all rule install and upgrade operations are\nretried\\nin case of a 429 response. This ensures proper handling when a\nuser\\nclicks multiple times an update or install rule buttons\\n\\n-\n`prebuilt_rules/_bootstrap` endpoint\\n- Install prebuilt rules and\nendpoint packages sequentially instead of\\nin parallel to prevent from\nhaving them both downloaded into memory\\nsimultaneously.\\n- Added a\n30-minute socket timeout to prevent the proxy from closing\nthe\\nconnection while rule installation is in progress.\\n- Introduced a\n`throttleRequests` wrapper, ensuring the endpoint handler\\nis called\nonly once when multiple concurrent requests are received.\\n- The first\nrequest triggers the handler, while subsequent requests wait\\nfor the\nfirst one to complete and reuse its result.\\n- This prevents costly\nprebuilt rule package installation from running\\nin parallel.\\n- Reusing\nthe response ensures the frontend correctly invalidates cached\\nprebuilt\nrule queries. Since concurrent frontend requests should receive\\nthe\nsame installed package information, responding with 421 and using\\nthe\nretry logic as in cases above is not an option here because the\\nsecond\nrequest would receive a package installation skipped response\\nleading\nto no cache invalidation.\\n\\n- `installation/_review` and\n`upgrade/_review` endpoints\\n- Concurrency is limited to one parallel\ncall. If another call is made\\nsimultaneously, the server responds with\n429 Too Many Requests.\\n- On the front end, all rule install and upgrade\noperations are retried\\nin case of a 429 response. This ensures proper\nhandling when a user\\nclicks multiple times an update or install rule\nbuttons\",\"sha\":\"c5557f33213f699acd9bb656af9166b1449d18f9\",\"branchLabelMapping\":{\"^v9.1.0$\":\"main\",\"^v8.19.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"release_note:fix\",\"performance\",\"v9.0.0\",\"Team:Detections\nand Resp\",\"Team: SecuritySolution\",\"Team:Detection Rule\nManagement\",\"Feature:Prebuilt Detection\nRules\",\"backport:version\",\"v8.18.0\",\"v9.1.0\",\"v8.19.0\",\"v8.17.3\"],\"title\":\"[Security\nSolution] Added concurrency limits and request throttling to prebuilt\nrule\nroutes\",\"number\":209551,\"url\":\"https://github.com/elastic/kibana/pull/209551\",\"mergeCommit\":{\"message\":\"[Security\nSolution] Added concurrency limits and request throttling to prebuilt\nrule routes (#209551)\\n\\n**Resolves:\nhttps://github.com//issues/208357**\\n**Resolves:\nhttps://github.com//issues/208355**\\n\\n## Summary \\n\\nTo\nprevent possible OOM errors, we need to limit concurrent requests\nto\\nprebuilt rule routes (see attached tickets for more details).\\n\\n-\n`installation/_perform` and `upgrade/_perform` endpoints\\n- Concurrency\nis limited to one parallel call. If another call is\nmade\\nsimultaneously, the server responds with 429 Too Many Requests.\\n-\nOn the front end, all rule install and upgrade operations are\nretried\\nin case of a 429 response. This ensures proper handling when a\nuser\\nclicks multiple times an update or install rule buttons\\n\\n-\n`prebuilt_rules/_bootstrap` endpoint\\n- Install prebuilt rules and\nendpoint packages sequentially instead of\\nin parallel to prevent from\nhaving them both downloaded into memory\\nsimultaneously.\\n- Added a\n30-minute socket timeout to prevent the proxy from closing\nthe\\nconnection while rule installation is in progress.\\n- Introduced a\n`throttleRequests` wrapper, ensuring the endpoint handler\\nis called\nonly once when multiple concurrent requests are received.\\n- The first\nrequest triggers the handler, while subsequent requests wait\\nfor the\nfirst one to complete and reuse its result.\\n- This prevents costly\nprebuilt rule package installation from running\\nin parallel.\\n- Reusing\nthe response ensures the frontend correctly invalidates cached\\nprebuilt\nrule queries. Since concurrent frontend requests should receive\\nthe\nsame installed package information, responding with 421 and using\\nthe\nretry logic as in cases above is not an option here because the\\nsecond\nrequest would receive a package installation skipped response\\nleading\nto no cache invalidation.\\n\\n- `installation/_review` and\n`upgrade/_review` endpoints\\n- Concurrency is limited to one parallel\ncall. If another call is made\\nsimultaneously, the server responds with\n429 Too Many Requests.\\n- On the front end, all rule install and upgrade\noperations are retried\\nin case of a 429 response. This ensures proper\nhandling when a user\\nclicks multiple times an update or install rule\nbuttons\",\"sha\":\"c5557f33213f699acd9bb656af9166b1449d18f9\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[\"9.0\",\"8.18\",\"8.x\",\"8.17\"],\"targetPullRequestStates\":[{\"branch\":\"9.0\",\"label\":\"v9.0.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"8.18\",\"label\":\"v8.18.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"main\",\"label\":\"v9.1.0\",\"branchLabelMappingKey\":\"^v9.1.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/209551\",\"number\":209551,\"mergeCommit\":{\"message\":\"[Security\nSolution] Added concurrency limits and request throttling to prebuilt\nrule routes (#209551)\\n\\n**Resolves:\nhttps://github.com//issues/208357**\\n**Resolves:\nhttps://github.com//issues/208355**\\n\\n## Summary \\n\\nTo\nprevent possible OOM errors, we need to limit concurrent requests\nto\\nprebuilt rule routes (see attached tickets for more details).\\n\\n-\n`installation/_perform` and `upgrade/_perform` endpoints\\n- Concurrency\nis limited to one parallel call. If another call is\nmade\\nsimultaneously, the server responds with 429 Too Many Requests.\\n-\nOn the front end, all rule install and upgrade operations are\nretried\\nin case of a 429 response. This ensures proper handling when a\nuser\\nclicks multiple times an update or install rule buttons\\n\\n-\n`prebuilt_rules/_bootstrap` endpoint\\n- Install prebuilt rules and\nendpoint packages sequentially instead of\\nin parallel to prevent from\nhaving them both downloaded into memory\\nsimultaneously.\\n- Added a\n30-minute socket timeout to prevent the proxy from closing\nthe\\nconnection while rule installation is in progress.\\n- Introduced a\n`throttleRequests` wrapper, ensuring the endpoint handler\\nis called\nonly once when multiple concurrent requests are received.\\n- The first\nrequest triggers the handler, while subsequent requests wait\\nfor the\nfirst one to complete and reuse its result.\\n- This prevents costly\nprebuilt rule package installation from running\\nin parallel.\\n- Reusing\nthe response ensures the frontend correctly invalidates cached\\nprebuilt\nrule queries. Since concurrent frontend requests should receive\\nthe\nsame installed package information, responding with 421 and using\\nthe\nretry logic as in cases above is not an option here because the\\nsecond\nrequest would receive a package installation skipped response\\nleading\nto no cache invalidation.\\n\\n- `installation/_review` and\n`upgrade/_review` endpoints\\n- Concurrency is limited to one parallel\ncall. If another call is made\\nsimultaneously, the server responds with\n429 Too Many Requests.\\n- On the front end, all rule install and upgrade\noperations are retried\\nin case of a 429 response. This ensures proper\nhandling when a user\\nclicks multiple times an update or install rule\nbuttons\",\"sha\":\"c5557f33213f699acd9bb656af9166b1449d18f9\"}},{\"branch\":\"8.x\",\"label\":\"v8.19.0\",\"branchLabelMappingKey\":\"^v8.19.0$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"8.17\",\"label\":\"v8.17.3\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"}]}]\nBACKPORT-->\n\nCo-authored-by: Dmitrii Shevchenko <dmitrii.shevchenko@elastic.co>"}},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/210640","number":210640,"state":"MERGED","mergeCommit":{"sha":"2bd85b19aa84575ec0745ba2ad24d4c7718d824f","message":"[8.18] [Security Solution] Added concurrency limits and request throttling to prebuilt rule routes (#209551) (#210640)\n\n# Backport\n\nThis will backport the following commits from `main` to `8.18`:\n- [[Security Solution] Added concurrency limits and request throttling\nto prebuilt rule routes\n(#209551)](https://github.com/elastic/kibana/pull/209551)\n\n<!--- Backport version: 9.4.3 -->\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT [{\"author\":{\"name\":\"Dmitrii\nShevchenko\",\"email\":\"dmitrii.shevchenko@elastic.co\"},\"sourceCommit\":{\"committedDate\":\"2025-02-11T17:12:03Z\",\"message\":\"[Security\nSolution] Added concurrency limits and request throttling to prebuilt\nrule routes (#209551)\\n\\n**Resolves:\nhttps://github.com//issues/208357**\\n**Resolves:\nhttps://github.com//issues/208355**\\n\\n## Summary \\n\\nTo\nprevent possible OOM errors, we need to limit concurrent requests\nto\\nprebuilt rule routes (see attached tickets for more details).\\n\\n-\n`installation/_perform` and `upgrade/_perform` endpoints\\n- Concurrency\nis limited to one parallel call. If another call is\nmade\\nsimultaneously, the server responds with 429 Too Many Requests.\\n-\nOn the front end, all rule install and upgrade operations are\nretried\\nin case of a 429 response. This ensures proper handling when a\nuser\\nclicks multiple times an update or install rule buttons\\n\\n-\n`prebuilt_rules/_bootstrap` endpoint\\n- Install prebuilt rules and\nendpoint packages sequentially instead of\\nin parallel to prevent from\nhaving them both downloaded into memory\\nsimultaneously.\\n- Added a\n30-minute socket timeout to prevent the proxy from closing\nthe\\nconnection while rule installation is in progress.\\n- Introduced a\n`throttleRequests` wrapper, ensuring the endpoint handler\\nis called\nonly once when multiple concurrent requests are received.\\n- The first\nrequest triggers the handler, while subsequent requests wait\\nfor the\nfirst one to complete and reuse its result.\\n- This prevents costly\nprebuilt rule package installation from running\\nin parallel.\\n- Reusing\nthe response ensures the frontend correctly invalidates cached\\nprebuilt\nrule queries. Since concurrent frontend requests should receive\\nthe\nsame installed package information, responding with 421 and using\\nthe\nretry logic as in cases above is not an option here because the\\nsecond\nrequest would receive a package installation skipped response\\nleading\nto no cache invalidation.\\n\\n- `installation/_review` and\n`upgrade/_review` endpoints\\n- Concurrency is limited to one parallel\ncall. If another call is made\\nsimultaneously, the server responds with\n429 Too Many Requests.\\n- On the front end, all rule install and upgrade\noperations are retried\\nin case of a 429 response. This ensures proper\nhandling when a user\\nclicks multiple times an update or install rule\nbuttons\",\"sha\":\"c5557f33213f699acd9bb656af9166b1449d18f9\",\"branchLabelMapping\":{\"^v9.1.0$\":\"main\",\"^v8.19.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"release_note:fix\",\"performance\",\"v9.0.0\",\"Team:Detections\nand Resp\",\"Team: SecuritySolution\",\"Team:Detection Rule\nManagement\",\"Feature:Prebuilt Detection\nRules\",\"backport:version\",\"v8.18.0\",\"v9.1.0\",\"v8.19.0\",\"v8.17.3\"],\"title\":\"[Security\nSolution] Added concurrency limits and request throttling to prebuilt\nrule\nroutes\",\"number\":209551,\"url\":\"https://github.com/elastic/kibana/pull/209551\",\"mergeCommit\":{\"message\":\"[Security\nSolution] Added concurrency limits and request throttling to prebuilt\nrule routes (#209551)\\n\\n**Resolves:\nhttps://github.com//issues/208357**\\n**Resolves:\nhttps://github.com//issues/208355**\\n\\n## Summary \\n\\nTo\nprevent possible OOM errors, we need to limit concurrent requests\nto\\nprebuilt rule routes (see attached tickets for more details).\\n\\n-\n`installation/_perform` and `upgrade/_perform` endpoints\\n- Concurrency\nis limited to one parallel call. If another call is\nmade\\nsimultaneously, the server responds with 429 Too Many Requests.\\n-\nOn the front end, all rule install and upgrade operations are\nretried\\nin case of a 429 response. This ensures proper handling when a\nuser\\nclicks multiple times an update or install rule buttons\\n\\n-\n`prebuilt_rules/_bootstrap` endpoint\\n- Install prebuilt rules and\nendpoint packages sequentially instead of\\nin parallel to prevent from\nhaving them both downloaded into memory\\nsimultaneously.\\n- Added a\n30-minute socket timeout to prevent the proxy from closing\nthe\\nconnection while rule installation is in progress.\\n- Introduced a\n`throttleRequests` wrapper, ensuring the endpoint handler\\nis called\nonly once when multiple concurrent requests are received.\\n- The first\nrequest triggers the handler, while subsequent requests wait\\nfor the\nfirst one to complete and reuse its result.\\n- This prevents costly\nprebuilt rule package installation from running\\nin parallel.\\n- Reusing\nthe response ensures the frontend correctly invalidates cached\\nprebuilt\nrule queries. Since concurrent frontend requests should receive\\nthe\nsame installed package information, responding with 421 and using\\nthe\nretry logic as in cases above is not an option here because the\\nsecond\nrequest would receive a package installation skipped response\\nleading\nto no cache invalidation.\\n\\n- `installation/_review` and\n`upgrade/_review` endpoints\\n- Concurrency is limited to one parallel\ncall. If another call is made\\nsimultaneously, the server responds with\n429 Too Many Requests.\\n- On the front end, all rule install and upgrade\noperations are retried\\nin case of a 429 response. This ensures proper\nhandling when a user\\nclicks multiple times an update or install rule\nbuttons\",\"sha\":\"c5557f33213f699acd9bb656af9166b1449d18f9\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[\"9.0\",\"8.18\",\"8.x\",\"8.17\"],\"targetPullRequestStates\":[{\"branch\":\"9.0\",\"label\":\"v9.0.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"8.18\",\"label\":\"v8.18.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"main\",\"label\":\"v9.1.0\",\"branchLabelMappingKey\":\"^v9.1.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/209551\",\"number\":209551,\"mergeCommit\":{\"message\":\"[Security\nSolution] Added concurrency limits and request throttling to prebuilt\nrule routes (#209551)\\n\\n**Resolves:\nhttps://github.com//issues/208357**\\n**Resolves:\nhttps://github.com//issues/208355**\\n\\n## Summary \\n\\nTo\nprevent possible OOM errors, we need to limit concurrent requests\nto\\nprebuilt rule routes (see attached tickets for more details).\\n\\n-\n`installation/_perform` and `upgrade/_perform` endpoints\\n- Concurrency\nis limited to one parallel call. If another call is\nmade\\nsimultaneously, the server responds with 429 Too Many Requests.\\n-\nOn the front end, all rule install and upgrade operations are\nretried\\nin case of a 429 response. This ensures proper handling when a\nuser\\nclicks multiple times an update or install rule buttons\\n\\n-\n`prebuilt_rules/_bootstrap` endpoint\\n- Install prebuilt rules and\nendpoint packages sequentially instead of\\nin parallel to prevent from\nhaving them both downloaded into memory\\nsimultaneously.\\n- Added a\n30-minute socket timeout to prevent the proxy from closing\nthe\\nconnection while rule installation is in progress.\\n- Introduced a\n`throttleRequests` wrapper, ensuring the endpoint handler\\nis called\nonly once when multiple concurrent requests are received.\\n- The first\nrequest triggers the handler, while subsequent requests wait\\nfor the\nfirst one to complete and reuse its result.\\n- This prevents costly\nprebuilt rule package installation from running\\nin parallel.\\n- Reusing\nthe response ensures the frontend correctly invalidates cached\\nprebuilt\nrule queries. Since concurrent frontend requests should receive\\nthe\nsame installed package information, responding with 421 and using\\nthe\nretry logic as in cases above is not an option here because the\\nsecond\nrequest would receive a package installation skipped response\\nleading\nto no cache invalidation.\\n\\n- `installation/_review` and\n`upgrade/_review` endpoints\\n- Concurrency is limited to one parallel\ncall. If another call is made\\nsimultaneously, the server responds with\n429 Too Many Requests.\\n- On the front end, all rule install and upgrade\noperations are retried\\nin case of a 429 response. This ensures proper\nhandling when a user\\nclicks multiple times an update or install rule\nbuttons\",\"sha\":\"c5557f33213f699acd9bb656af9166b1449d18f9\"}},{\"branch\":\"8.x\",\"label\":\"v8.19.0\",\"branchLabelMappingKey\":\"^v8.19.0$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"8.17\",\"label\":\"v8.17.3\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"}]}]\nBACKPORT-->\n\nCo-authored-by: Dmitrii Shevchenko <dmitrii.shevchenko@elastic.co>"}},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/209551","number":209551,"mergeCommit":{"message":"[Security Solution] Added concurrency limits and request throttling to prebuilt rule routes (#209551)\n\n**Resolves: https://github.com/elastic/kibana/issues/208357**\n**Resolves: https://github.com/elastic/kibana/issues/208355**\n\n## Summary \n\nTo prevent possible OOM errors, we need to limit concurrent requests to\nprebuilt rule routes (see attached tickets for more details).\n\n- `installation/_perform` and `upgrade/_perform` endpoints\n- Concurrency is limited to one parallel call. If another call is made\nsimultaneously, the server responds with 429 Too Many Requests.\n- On the front end, all rule install and upgrade operations are retried\nin case of a 429 response. This ensures proper handling when a user\nclicks multiple times an update or install rule buttons\n\n- `prebuilt_rules/_bootstrap` endpoint\n- Install prebuilt rules and endpoint packages sequentially instead of\nin parallel to prevent from having them both downloaded into memory\nsimultaneously.\n- Added a 30-minute socket timeout to prevent the proxy from closing the\nconnection while rule installation is in progress.\n- Introduced a `throttleRequests` wrapper, ensuring the endpoint handler\nis called only once when multiple concurrent requests are received.\n- The first request triggers the handler, while subsequent requests wait\nfor the first one to complete and reuse its result.\n- This prevents costly prebuilt rule package installation from running\nin parallel.\n- Reusing the response ensures the frontend correctly invalidates cached\nprebuilt rule queries. Since concurrent frontend requests should receive\nthe same installed package information, responding with 421 and using\nthe retry logic as in cases above is not an option here because the\nsecond request would receive a package installation skipped response\nleading to no cache invalidation.\n\n- `installation/_review` and `upgrade/_review` endpoints\n- Concurrency is limited to one parallel call. If another call is made\nsimultaneously, the server responds with 429 Too Many Requests.\n- On the front end, all rule install and upgrade operations are retried\nin case of a 429 response. This ensures proper handling when a user\nclicks multiple times an update or install rule buttons","sha":"c5557f33213f699acd9bb656af9166b1449d18f9"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/210641","number":210641,"state":"MERGED","mergeCommit":{"sha":"8018c82f7d92b1d4798dea3c0a677bae740d4f8a","message":"[8.x] [Security Solution] Added concurrency limits and request throttling to prebuilt rule routes (#209551) (#210641)\n\n# Backport\n\nThis will backport the following commits from `main` to `8.x`:\n- [[Security Solution] Added concurrency limits and request throttling\nto prebuilt rule routes\n(#209551)](https://github.com/elastic/kibana/pull/209551)\n\n<!--- Backport version: 9.4.3 -->\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT [{\"author\":{\"name\":\"Dmitrii\nShevchenko\",\"email\":\"dmitrii.shevchenko@elastic.co\"},\"sourceCommit\":{\"committedDate\":\"2025-02-11T17:12:03Z\",\"message\":\"[Security\nSolution] Added concurrency limits and request throttling to prebuilt\nrule routes (#209551)\\n\\n**Resolves:\nhttps://github.com//issues/208357**\\n**Resolves:\nhttps://github.com//issues/208355**\\n\\n## Summary \\n\\nTo\nprevent possible OOM errors, we need to limit concurrent requests\nto\\nprebuilt rule routes (see attached tickets for more details).\\n\\n-\n`installation/_perform` and `upgrade/_perform` endpoints\\n- Concurrency\nis limited to one parallel call. If another call is\nmade\\nsimultaneously, the server responds with 429 Too Many Requests.\\n-\nOn the front end, all rule install and upgrade operations are\nretried\\nin case of a 429 response. This ensures proper handling when a\nuser\\nclicks multiple times an update or install rule buttons\\n\\n-\n`prebuilt_rules/_bootstrap` endpoint\\n- Install prebuilt rules and\nendpoint packages sequentially instead of\\nin parallel to prevent from\nhaving them both downloaded into memory\\nsimultaneously.\\n- Added a\n30-minute socket timeout to prevent the proxy from closing\nthe\\nconnection while rule installation is in progress.\\n- Introduced a\n`throttleRequests` wrapper, ensuring the endpoint handler\\nis called\nonly once when multiple concurrent requests are received.\\n- The first\nrequest triggers the handler, while subsequent requests wait\\nfor the\nfirst one to complete and reuse its result.\\n- This prevents costly\nprebuilt rule package installation from running\\nin parallel.\\n- Reusing\nthe response ensures the frontend correctly invalidates cached\\nprebuilt\nrule queries. Since concurrent frontend requests should receive\\nthe\nsame installed package information, responding with 421 and using\\nthe\nretry logic as in cases above is not an option here because the\\nsecond\nrequest would receive a package installation skipped response\\nleading\nto no cache invalidation.\\n\\n- `installation/_review` and\n`upgrade/_review` endpoints\\n- Concurrency is limited to one parallel\ncall. If another call is made\\nsimultaneously, the server responds with\n429 Too Many Requests.\\n- On the front end, all rule install and upgrade\noperations are retried\\nin case of a 429 response. This ensures proper\nhandling when a user\\nclicks multiple times an update or install rule\nbuttons\",\"sha\":\"c5557f33213f699acd9bb656af9166b1449d18f9\",\"branchLabelMapping\":{\"^v9.1.0$\":\"main\",\"^v8.19.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"release_note:fix\",\"performance\",\"v9.0.0\",\"Team:Detections\nand Resp\",\"Team: SecuritySolution\",\"Team:Detection Rule\nManagement\",\"Feature:Prebuilt Detection\nRules\",\"backport:version\",\"v8.18.0\",\"v9.1.0\",\"v8.19.0\",\"v8.17.3\"],\"title\":\"[Security\nSolution] Added concurrency limits and request throttling to prebuilt\nrule\nroutes\",\"number\":209551,\"url\":\"https://github.com/elastic/kibana/pull/209551\",\"mergeCommit\":{\"message\":\"[Security\nSolution] Added concurrency limits and request throttling to prebuilt\nrule routes (#209551)\\n\\n**Resolves:\nhttps://github.com//issues/208357**\\n**Resolves:\nhttps://github.com//issues/208355**\\n\\n## Summary \\n\\nTo\nprevent possible OOM errors, we need to limit concurrent requests\nto\\nprebuilt rule routes (see attached tickets for more details).\\n\\n-\n`installation/_perform` and `upgrade/_perform` endpoints\\n- Concurrency\nis limited to one parallel call. If another call is\nmade\\nsimultaneously, the server responds with 429 Too Many Requests.\\n-\nOn the front end, all rule install and upgrade operations are\nretried\\nin case of a 429 response. This ensures proper handling when a\nuser\\nclicks multiple times an update or install rule buttons\\n\\n-\n`prebuilt_rules/_bootstrap` endpoint\\n- Install prebuilt rules and\nendpoint packages sequentially instead of\\nin parallel to prevent from\nhaving them both downloaded into memory\\nsimultaneously.\\n- Added a\n30-minute socket timeout to prevent the proxy from closing\nthe\\nconnection while rule installation is in progress.\\n- Introduced a\n`throttleRequests` wrapper, ensuring the endpoint handler\\nis called\nonly once when multiple concurrent requests are received.\\n- The first\nrequest triggers the handler, while subsequent requests wait\\nfor the\nfirst one to complete and reuse its result.\\n- This prevents costly\nprebuilt rule package installation from running\\nin parallel.\\n- Reusing\nthe response ensures the frontend correctly invalidates cached\\nprebuilt\nrule queries. Since concurrent frontend requests should receive\\nthe\nsame installed package information, responding with 421 and using\\nthe\nretry logic as in cases above is not an option here because the\\nsecond\nrequest would receive a package installation skipped response\\nleading\nto no cache invalidation.\\n\\n- `installation/_review` and\n`upgrade/_review` endpoints\\n- Concurrency is limited to one parallel\ncall. If another call is made\\nsimultaneously, the server responds with\n429 Too Many Requests.\\n- On the front end, all rule install and upgrade\noperations are retried\\nin case of a 429 response. This ensures proper\nhandling when a user\\nclicks multiple times an update or install rule\nbuttons\",\"sha\":\"c5557f33213f699acd9bb656af9166b1449d18f9\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[\"9.0\",\"8.18\",\"8.x\",\"8.17\"],\"targetPullRequestStates\":[{\"branch\":\"9.0\",\"label\":\"v9.0.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"8.18\",\"label\":\"v8.18.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"main\",\"label\":\"v9.1.0\",\"branchLabelMappingKey\":\"^v9.1.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/209551\",\"number\":209551,\"mergeCommit\":{\"message\":\"[Security\nSolution] Added concurrency limits and request throttling to prebuilt\nrule routes (#209551)\\n\\n**Resolves:\nhttps://github.com//issues/208357**\\n**Resolves:\nhttps://github.com//issues/208355**\\n\\n## Summary \\n\\nTo\nprevent possible OOM errors, we need to limit concurrent requests\nto\\nprebuilt rule routes (see attached tickets for more details).\\n\\n-\n`installation/_perform` and `upgrade/_perform` endpoints\\n- Concurrency\nis limited to one parallel call. If another call is\nmade\\nsimultaneously, the server responds with 429 Too Many Requests.\\n-\nOn the front end, all rule install and upgrade operations are\nretried\\nin case of a 429 response. This ensures proper handling when a\nuser\\nclicks multiple times an update or install rule buttons\\n\\n-\n`prebuilt_rules/_bootstrap` endpoint\\n- Install prebuilt rules and\nendpoint packages sequentially instead of\\nin parallel to prevent from\nhaving them both downloaded into memory\\nsimultaneously.\\n- Added a\n30-minute socket timeout to prevent the proxy from closing\nthe\\nconnection while rule installation is in progress.\\n- Introduced a\n`throttleRequests` wrapper, ensuring the endpoint handler\\nis called\nonly once when multiple concurrent requests are received.\\n- The first\nrequest triggers the handler, while subsequent requests wait\\nfor the\nfirst one to complete and reuse its result.\\n- This prevents costly\nprebuilt rule package installation from running\\nin parallel.\\n- Reusing\nthe response ensures the frontend correctly invalidates cached\\nprebuilt\nrule queries. Since concurrent frontend requests should receive\\nthe\nsame installed package information, responding with 421 and using\\nthe\nretry logic as in cases above is not an option here because the\\nsecond\nrequest would receive a package installation skipped response\\nleading\nto no cache invalidation.\\n\\n- `installation/_review` and\n`upgrade/_review` endpoints\\n- Concurrency is limited to one parallel\ncall. If another call is made\\nsimultaneously, the server responds with\n429 Too Many Requests.\\n- On the front end, all rule install and upgrade\noperations are retried\\nin case of a 429 response. This ensures proper\nhandling when a user\\nclicks multiple times an update or install rule\nbuttons\",\"sha\":\"c5557f33213f699acd9bb656af9166b1449d18f9\"}},{\"branch\":\"8.x\",\"label\":\"v8.19.0\",\"branchLabelMappingKey\":\"^v8.19.0$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"8.17\",\"label\":\"v8.17.3\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"}]}]\nBACKPORT-->\n\nCo-authored-by: Dmitrii Shevchenko <dmitrii.shevchenko@elastic.co>"}},{"branch":"8.17","label":"v8.17.3","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT-->
- Loading branch information