[Security Solution] Tests for prebuilt rule upgrade workflow #202078
Assignees
Labels
8.18 candidate
Feature:Prebuilt Detection Rules
Security Solution Prebuilt Detection Rules area
Team:Detection Rule Management
Security Detection Rule Management Team
Team:Detections and Resp
Security Detection Response Team
Team: SecuritySolution
Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.
test
test-coverage
issues & PRs for improving code test coverage
test-plan
v8.18.0
Comments
banderror
added
8.18 candidate
Feature:Prebuilt Detection Rules
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
|
Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management) |
92 tasks
banderror
changed the title
|
@jpdjere @maximpn @nikitaindik I'd suggest us to split the work on the upgrade test coverage between @maximpn and @jpdjere. Considering the fact that @maximpn and @nikitaindik are busy with finalizing the ThreeWayDiff UI and fixing bugs in it, maybe @jpdjere could address most of this ticket, especially because he is very familiar with the requirements. Let's sync up on it when @jpdjere is back next week. |
13 tasks
|
cc @pborgonovi |
This was referenced Dec 10, 2024
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Dec 19, 2024
Partially resolves: elastic#202078 ## Summary This PR refactors the test plans involving the Installation and Upgrade Workflows: - breaks down the gigantic `installation_and_upgrade.md` file into three files: - `installation.md` covering exclusively prebuilt rule install workflows - `upgrade.md` covering exclusively prebuilt rule upgrade workflows - `package_installation_and_upgrade.md` which covers scenarios for the installation and upgrades of the Prebuilt Rules package itself. The *Upgrade of Prebuilt Rules* (`upgrade.md`) was modified as follows: - expanded to cover scenarios detailed in the linked ticket (`Upgrading prebuilt rules one-by-one without preview` and `Upgrading prebuilt rules in bulk` in the Rule Updates table, including cases with and without conflicts, as well as a mixture of both) - internally refactored so that sections that will become legacy with the release of Milestone 3 were pushed to the end and marked as pending deletion once the feature is live. - from point above, the test plan now is in a better position to add the scenarios for the Three-Way-Diff upgrade workflow. cc @maximpn @nikitaindik ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co> (cherry picked from commit cd74ecf)
|
The ticket was closed automatically after merging #203331. Reopening it since all scope is not covered yet. |
JoseLuisGJ
pushed a commit
to JoseLuisGJ/kibana
that referenced
this issue
Dec 19, 2024
Partially resolves: elastic#202078 ## Summary This PR refactors the test plans involving the Installation and Upgrade Workflows: - breaks down the gigantic `installation_and_upgrade.md` file into three files: - `installation.md` covering exclusively prebuilt rule install workflows - `upgrade.md` covering exclusively prebuilt rule upgrade workflows - `package_installation_and_upgrade.md` which covers scenarios for the installation and upgrades of the Prebuilt Rules package itself. The *Upgrade of Prebuilt Rules* (`upgrade.md`) was modified as follows: - expanded to cover scenarios detailed in the linked ticket (`Upgrading prebuilt rules one-by-one without preview` and `Upgrading prebuilt rules in bulk` in the Rule Updates table, including cases with and without conflicts, as well as a mixture of both) - internally refactored so that sections that will become legacy with the release of Milestone 3 were pushed to the end and marked as pending deletion once the feature is live. - from point above, the test plan now is in a better position to add the scenarios for the Three-Way-Diff upgrade workflow. cc @maximpn @nikitaindik ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
2 tasks
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Feb 14, 2025
…part 1 (elastic#211300) **Epic:** elastic#174168 **Partially addresses:** elastic#202068, elastic#202078, elastic#202079 ## Summary We're cleaning up and refactoring our existing test plans for prebuilt rule customization, upgrade, and export/import workflows. Specifically, in this PR we're consistently: - Changing the file names and test plan titles. - Adding or expanding test plan summaries. - Regenerating tables of contents using the [Markdown All in One](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one) VS Code extension. No "functional" changes have been made to any test plans, such as adding, removing, or updating any scenarios. This refactoring prepares the test plans for being "functionally" changed and improved in follow-up PRs. For example, we're going to cover the logic described in elastic#210358 and address any other gaps in coverage. (cherry picked from commit 9600de4)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Feb 14, 2025
…part 1 (elastic#211300) **Epic:** elastic#174168 **Partially addresses:** elastic#202068, elastic#202078, elastic#202079 ## Summary We're cleaning up and refactoring our existing test plans for prebuilt rule customization, upgrade, and export/import workflows. Specifically, in this PR we're consistently: - Changing the file names and test plan titles. - Adding or expanding test plan summaries. - Regenerating tables of contents using the [Markdown All in One](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one) VS Code extension. No "functional" changes have been made to any test plans, such as adding, removing, or updating any scenarios. This refactoring prepares the test plans for being "functionally" changed and improved in follow-up PRs. For example, we're going to cover the logic described in elastic#210358 and address any other gaps in coverage. (cherry picked from commit 9600de4)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Feb 14, 2025
…part 1 (elastic#211300) **Epic:** elastic#174168 **Partially addresses:** elastic#202068, elastic#202078, elastic#202079 ## Summary We're cleaning up and refactoring our existing test plans for prebuilt rule customization, upgrade, and export/import workflows. Specifically, in this PR we're consistently: - Changing the file names and test plan titles. - Adding or expanding test plan summaries. - Regenerating tables of contents using the [Markdown All in One](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one) VS Code extension. No "functional" changes have been made to any test plans, such as adding, removing, or updating any scenarios. This refactoring prepares the test plans for being "functionally" changed and improved in follow-up PRs. For example, we're going to cover the logic described in elastic#210358 and address any other gaps in coverage. (cherry picked from commit 9600de4)
kibanamachine
added a commit
that referenced
this issue
Feb 14, 2025
…lans, part 1 (#211300) (#211307) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)](#211300) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Georgii Gorbachev","email":"georgii.gorbachev@elastic.co"},"sourceCommit":{"committedDate":"2025-02-14T18:22:45Z","message":"[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, in this PR we're consistently:\r\n\r\n- Changing the file names and test plan titles.\r\n- Adding or expanding test plan summaries.\r\n- Regenerating tables of contents using the [Markdown All in\r\nOne](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one)\r\nVS Code extension.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.","sha":"9600de4338edc123dbac7c2cb3403ede2f016752","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Refactor prebuilt rule customization test plans, part 1","number":211300,"url":"https://github.com/elastic/kibana/pull/211300","mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, in this PR we're consistently:\r\n\r\n- Changing the file names and test plan titles.\r\n- Adding or expanding test plan summaries.\r\n- Regenerating tables of contents using the [Markdown All in\r\nOne](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one)\r\nVS Code extension.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.","sha":"9600de4338edc123dbac7c2cb3403ede2f016752"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/211300","number":211300,"mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, in this PR we're consistently:\r\n\r\n- Changing the file names and test plan titles.\r\n- Adding or expanding test plan summaries.\r\n- Regenerating tables of contents using the [Markdown All in\r\nOne](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one)\r\nVS Code extension.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.","sha":"9600de4338edc123dbac7c2cb3403ede2f016752"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
kibanamachine
added a commit
that referenced
this issue
Feb 14, 2025
…lans, part 1 (#211300) (#211308) # Backport This will backport the following commits from `main` to `9.0`: - [[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)](#211300) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Georgii Gorbachev","email":"georgii.gorbachev@elastic.co"},"sourceCommit":{"committedDate":"2025-02-14T18:22:45Z","message":"[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, in this PR we're consistently:\r\n\r\n- Changing the file names and test plan titles.\r\n- Adding or expanding test plan summaries.\r\n- Regenerating tables of contents using the [Markdown All in\r\nOne](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one)\r\nVS Code extension.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.","sha":"9600de4338edc123dbac7c2cb3403ede2f016752","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Refactor prebuilt rule customization test plans, part 1","number":211300,"url":"https://github.com/elastic/kibana/pull/211300","mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, in this PR we're consistently:\r\n\r\n- Changing the file names and test plan titles.\r\n- Adding or expanding test plan summaries.\r\n- Regenerating tables of contents using the [Markdown All in\r\nOne](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one)\r\nVS Code extension.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.","sha":"9600de4338edc123dbac7c2cb3403ede2f016752"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/211300","number":211300,"mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, in this PR we're consistently:\r\n\r\n- Changing the file names and test plan titles.\r\n- Adding or expanding test plan summaries.\r\n- Regenerating tables of contents using the [Markdown All in\r\nOne](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one)\r\nVS Code extension.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.","sha":"9600de4338edc123dbac7c2cb3403ede2f016752"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
kibanamachine
added a commit
that referenced
this issue
Feb 14, 2025
…plans, part 1 (#211300) (#211306) # Backport This will backport the following commits from `main` to `8.18`: - [[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)](#211300) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Georgii Gorbachev","email":"georgii.gorbachev@elastic.co"},"sourceCommit":{"committedDate":"2025-02-14T18:22:45Z","message":"[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, in this PR we're consistently:\r\n\r\n- Changing the file names and test plan titles.\r\n- Adding or expanding test plan summaries.\r\n- Regenerating tables of contents using the [Markdown All in\r\nOne](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one)\r\nVS Code extension.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.","sha":"9600de4338edc123dbac7c2cb3403ede2f016752","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Refactor prebuilt rule customization test plans, part 1","number":211300,"url":"https://github.com/elastic/kibana/pull/211300","mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, in this PR we're consistently:\r\n\r\n- Changing the file names and test plan titles.\r\n- Adding or expanding test plan summaries.\r\n- Regenerating tables of contents using the [Markdown All in\r\nOne](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one)\r\nVS Code extension.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.","sha":"9600de4338edc123dbac7c2cb3403ede2f016752"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/211300","number":211300,"mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, in this PR we're consistently:\r\n\r\n- Changing the file names and test plan titles.\r\n- Adding or expanding test plan summaries.\r\n- Regenerating tables of contents using the [Markdown All in\r\nOne](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one)\r\nVS Code extension.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.","sha":"9600de4338edc123dbac7c2cb3403ede2f016752"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
banderror
added a commit
that referenced
this issue
Feb 18, 2025
…part 2 (#211472) **Epic:** #174168 **Partially addresses:** #202068, #202078, #202079 **Follow-up to:** #211300 ## Summary We're cleaning up and refactoring our existing test plans for prebuilt rule customization, upgrade, and export/import workflows. Specifically, this PR: - Creates an "entrypoint" file that should help navigate all the test plans for prebuilt rules. - Creates a file for keeping common information about prebuilt rules that can be shared between the test plans. - Extracts duplicated terminology to the file with common information. - Extracts duplicated assumptions to the file with common information. - Extracts duplicated non-functional requirements to the file with common information. - Adds user stories to each test plan. - Updates links to tickets in every test plan. No "functional" changes have been made to any test plans, such as adding, removing, or updating any scenarios. This refactoring prepares the test plans for being "functionally" changed and improved in follow-up PRs. For example, we're going to cover the logic described in #210358 and address any other gaps in coverage. ## Review tip It might be easier to review this PR commit-by-commit as each of them contains logically cohesive changes.
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Feb 18, 2025
…part 2 (elastic#211472) **Epic:** elastic#174168 **Partially addresses:** elastic#202068, elastic#202078, elastic#202079 **Follow-up to:** elastic#211300 ## Summary We're cleaning up and refactoring our existing test plans for prebuilt rule customization, upgrade, and export/import workflows. Specifically, this PR: - Creates an "entrypoint" file that should help navigate all the test plans for prebuilt rules. - Creates a file for keeping common information about prebuilt rules that can be shared between the test plans. - Extracts duplicated terminology to the file with common information. - Extracts duplicated assumptions to the file with common information. - Extracts duplicated non-functional requirements to the file with common information. - Adds user stories to each test plan. - Updates links to tickets in every test plan. No "functional" changes have been made to any test plans, such as adding, removing, or updating any scenarios. This refactoring prepares the test plans for being "functionally" changed and improved in follow-up PRs. For example, we're going to cover the logic described in elastic#210358 and address any other gaps in coverage. ## Review tip It might be easier to review this PR commit-by-commit as each of them contains logically cohesive changes. (cherry picked from commit 786df79)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Feb 18, 2025
…part 2 (elastic#211472) **Epic:** elastic#174168 **Partially addresses:** elastic#202068, elastic#202078, elastic#202079 **Follow-up to:** elastic#211300 ## Summary We're cleaning up and refactoring our existing test plans for prebuilt rule customization, upgrade, and export/import workflows. Specifically, this PR: - Creates an "entrypoint" file that should help navigate all the test plans for prebuilt rules. - Creates a file for keeping common information about prebuilt rules that can be shared between the test plans. - Extracts duplicated terminology to the file with common information. - Extracts duplicated assumptions to the file with common information. - Extracts duplicated non-functional requirements to the file with common information. - Adds user stories to each test plan. - Updates links to tickets in every test plan. No "functional" changes have been made to any test plans, such as adding, removing, or updating any scenarios. This refactoring prepares the test plans for being "functionally" changed and improved in follow-up PRs. For example, we're going to cover the logic described in elastic#210358 and address any other gaps in coverage. ## Review tip It might be easier to review this PR commit-by-commit as each of them contains logically cohesive changes. (cherry picked from commit 786df79)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Feb 18, 2025
…part 2 (elastic#211472) **Epic:** elastic#174168 **Partially addresses:** elastic#202068, elastic#202078, elastic#202079 **Follow-up to:** elastic#211300 ## Summary We're cleaning up and refactoring our existing test plans for prebuilt rule customization, upgrade, and export/import workflows. Specifically, this PR: - Creates an "entrypoint" file that should help navigate all the test plans for prebuilt rules. - Creates a file for keeping common information about prebuilt rules that can be shared between the test plans. - Extracts duplicated terminology to the file with common information. - Extracts duplicated assumptions to the file with common information. - Extracts duplicated non-functional requirements to the file with common information. - Adds user stories to each test plan. - Updates links to tickets in every test plan. No "functional" changes have been made to any test plans, such as adding, removing, or updating any scenarios. This refactoring prepares the test plans for being "functionally" changed and improved in follow-up PRs. For example, we're going to cover the logic described in elastic#210358 and address any other gaps in coverage. ## Review tip It might be easier to review this PR commit-by-commit as each of them contains logically cohesive changes. (cherry picked from commit 786df79)
kibanamachine
added a commit
that referenced
this issue
Feb 18, 2025
…plans, part 2 (#211472) (#211649) # Backport This will backport the following commits from `main` to `8.18`: - [[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)](#211472) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Georgii Gorbachev","email":"georgii.gorbachev@elastic.co"},"sourceCommit":{"committedDate":"2025-02-18T20:34:15Z","message":"[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n**Follow-up to:** https://github.com/elastic/kibana/pull/211300\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, this PR:\r\n\r\n- Creates an \"entrypoint\" file that should help navigate all the test\r\nplans for prebuilt rules.\r\n- Creates a file for keeping common information about prebuilt rules\r\nthat can be shared between the test plans.\r\n- Extracts duplicated terminology to the file with common information.\r\n- Extracts duplicated assumptions to the file with common information.\r\n- Extracts duplicated non-functional requirements to the file with\r\ncommon information.\r\n- Adds user stories to each test plan.\r\n- Updates links to tickets in every test plan.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.\r\n\r\n## Review tip\r\n\r\nIt might be easier to review this PR commit-by-commit as each of them\r\ncontains logically cohesive changes.","sha":"786df79f37642b8ddf214e086a3375204ec63a3b","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Refactor prebuilt rule customization test plans, part 2","number":211472,"url":"https://github.com/elastic/kibana/pull/211472","mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n**Follow-up to:** https://github.com/elastic/kibana/pull/211300\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, this PR:\r\n\r\n- Creates an \"entrypoint\" file that should help navigate all the test\r\nplans for prebuilt rules.\r\n- Creates a file for keeping common information about prebuilt rules\r\nthat can be shared between the test plans.\r\n- Extracts duplicated terminology to the file with common information.\r\n- Extracts duplicated assumptions to the file with common information.\r\n- Extracts duplicated non-functional requirements to the file with\r\ncommon information.\r\n- Adds user stories to each test plan.\r\n- Updates links to tickets in every test plan.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.\r\n\r\n## Review tip\r\n\r\nIt might be easier to review this PR commit-by-commit as each of them\r\ncontains logically cohesive changes.","sha":"786df79f37642b8ddf214e086a3375204ec63a3b"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/211472","number":211472,"mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n**Follow-up to:** https://github.com/elastic/kibana/pull/211300\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, this PR:\r\n\r\n- Creates an \"entrypoint\" file that should help navigate all the test\r\nplans for prebuilt rules.\r\n- Creates a file for keeping common information about prebuilt rules\r\nthat can be shared between the test plans.\r\n- Extracts duplicated terminology to the file with common information.\r\n- Extracts duplicated assumptions to the file with common information.\r\n- Extracts duplicated non-functional requirements to the file with\r\ncommon information.\r\n- Adds user stories to each test plan.\r\n- Updates links to tickets in every test plan.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.\r\n\r\n## Review tip\r\n\r\nIt might be easier to review this PR commit-by-commit as each of them\r\ncontains logically cohesive changes.","sha":"786df79f37642b8ddf214e086a3375204ec63a3b"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
kibanamachine
added a commit
that referenced
this issue
Feb 18, 2025
…lans, part 2 (#211472) (#211651) # Backport This will backport the following commits from `main` to `9.0`: - [[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)](#211472) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Georgii Gorbachev","email":"georgii.gorbachev@elastic.co"},"sourceCommit":{"committedDate":"2025-02-18T20:34:15Z","message":"[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n**Follow-up to:** https://github.com/elastic/kibana/pull/211300\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, this PR:\r\n\r\n- Creates an \"entrypoint\" file that should help navigate all the test\r\nplans for prebuilt rules.\r\n- Creates a file for keeping common information about prebuilt rules\r\nthat can be shared between the test plans.\r\n- Extracts duplicated terminology to the file with common information.\r\n- Extracts duplicated assumptions to the file with common information.\r\n- Extracts duplicated non-functional requirements to the file with\r\ncommon information.\r\n- Adds user stories to each test plan.\r\n- Updates links to tickets in every test plan.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.\r\n\r\n## Review tip\r\n\r\nIt might be easier to review this PR commit-by-commit as each of them\r\ncontains logically cohesive changes.","sha":"786df79f37642b8ddf214e086a3375204ec63a3b","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Refactor prebuilt rule customization test plans, part 2","number":211472,"url":"https://github.com/elastic/kibana/pull/211472","mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n**Follow-up to:** https://github.com/elastic/kibana/pull/211300\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, this PR:\r\n\r\n- Creates an \"entrypoint\" file that should help navigate all the test\r\nplans for prebuilt rules.\r\n- Creates a file for keeping common information about prebuilt rules\r\nthat can be shared between the test plans.\r\n- Extracts duplicated terminology to the file with common information.\r\n- Extracts duplicated assumptions to the file with common information.\r\n- Extracts duplicated non-functional requirements to the file with\r\ncommon information.\r\n- Adds user stories to each test plan.\r\n- Updates links to tickets in every test plan.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.\r\n\r\n## Review tip\r\n\r\nIt might be easier to review this PR commit-by-commit as each of them\r\ncontains logically cohesive changes.","sha":"786df79f37642b8ddf214e086a3375204ec63a3b"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/211472","number":211472,"mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n**Follow-up to:** https://github.com/elastic/kibana/pull/211300\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, this PR:\r\n\r\n- Creates an \"entrypoint\" file that should help navigate all the test\r\nplans for prebuilt rules.\r\n- Creates a file for keeping common information about prebuilt rules\r\nthat can be shared between the test plans.\r\n- Extracts duplicated terminology to the file with common information.\r\n- Extracts duplicated assumptions to the file with common information.\r\n- Extracts duplicated non-functional requirements to the file with\r\ncommon information.\r\n- Adds user stories to each test plan.\r\n- Updates links to tickets in every test plan.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.\r\n\r\n## Review tip\r\n\r\nIt might be easier to review this PR commit-by-commit as each of them\r\ncontains logically cohesive changes.","sha":"786df79f37642b8ddf214e086a3375204ec63a3b"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
kibanamachine
added a commit
that referenced
this issue
Feb 18, 2025
…lans, part 2 (#211472) (#211650) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)](#211472) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Georgii Gorbachev","email":"georgii.gorbachev@elastic.co"},"sourceCommit":{"committedDate":"2025-02-18T20:34:15Z","message":"[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n**Follow-up to:** https://github.com/elastic/kibana/pull/211300\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, this PR:\r\n\r\n- Creates an \"entrypoint\" file that should help navigate all the test\r\nplans for prebuilt rules.\r\n- Creates a file for keeping common information about prebuilt rules\r\nthat can be shared between the test plans.\r\n- Extracts duplicated terminology to the file with common information.\r\n- Extracts duplicated assumptions to the file with common information.\r\n- Extracts duplicated non-functional requirements to the file with\r\ncommon information.\r\n- Adds user stories to each test plan.\r\n- Updates links to tickets in every test plan.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.\r\n\r\n## Review tip\r\n\r\nIt might be easier to review this PR commit-by-commit as each of them\r\ncontains logically cohesive changes.","sha":"786df79f37642b8ddf214e086a3375204ec63a3b","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Refactor prebuilt rule customization test plans, part 2","number":211472,"url":"https://github.com/elastic/kibana/pull/211472","mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n**Follow-up to:** https://github.com/elastic/kibana/pull/211300\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, this PR:\r\n\r\n- Creates an \"entrypoint\" file that should help navigate all the test\r\nplans for prebuilt rules.\r\n- Creates a file for keeping common information about prebuilt rules\r\nthat can be shared between the test plans.\r\n- Extracts duplicated terminology to the file with common information.\r\n- Extracts duplicated assumptions to the file with common information.\r\n- Extracts duplicated non-functional requirements to the file with\r\ncommon information.\r\n- Adds user stories to each test plan.\r\n- Updates links to tickets in every test plan.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.\r\n\r\n## Review tip\r\n\r\nIt might be easier to review this PR commit-by-commit as each of them\r\ncontains logically cohesive changes.","sha":"786df79f37642b8ddf214e086a3375204ec63a3b"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/211472","number":211472,"mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n**Follow-up to:** https://github.com/elastic/kibana/pull/211300\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, this PR:\r\n\r\n- Creates an \"entrypoint\" file that should help navigate all the test\r\nplans for prebuilt rules.\r\n- Creates a file for keeping common information about prebuilt rules\r\nthat can be shared between the test plans.\r\n- Extracts duplicated terminology to the file with common information.\r\n- Extracts duplicated assumptions to the file with common information.\r\n- Extracts duplicated non-functional requirements to the file with\r\ncommon information.\r\n- Adds user stories to each test plan.\r\n- Updates links to tickets in every test plan.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.\r\n\r\n## Review tip\r\n\r\nIt might be easier to review this PR commit-by-commit as each of them\r\ncontains logically cohesive changes.","sha":"786df79f37642b8ddf214e086a3375204ec63a3b"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
maximpn
added a commit
that referenced
this issue
Feb 21, 2025
…t rule customization feature (#209260) **Addresses:** #202078 **Partially implements test plan:** #205922 ## Summary This PR implements the Rule Upgrade test plan added in #205922 and #203331. ## Details Tests cover only enabled customization and organized in a following way - `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.** - `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. - `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. This PR supersedes #205217 and #205339. * `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`. ## Flaky test runner Flaky test runs for 100 iterations were successful (#209260 (comment) and #209260 (comment)).
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Feb 21, 2025
…t rule customization feature (elastic#209260) **Addresses:** elastic#202078 **Partially implements test plan:** elastic#205922 ## Summary This PR implements the Rule Upgrade test plan added in elastic#205922 and elastic#203331. ## Details Tests cover only enabled customization and organized in a following way - `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.** - `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. - `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. This PR supersedes elastic#205217 and elastic#205339. * `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`. ## Flaky test runner Flaky test runs for 100 iterations were successful (elastic#209260 (comment) and elastic#209260 (comment)). (cherry picked from commit 4909770)
maximpn
added a commit
to maximpn/kibana
that referenced
this issue
Feb 24, 2025
…t rule customization feature (elastic#209260) **Addresses:** elastic#202078 **Partially implements test plan:** elastic#205922 ## Summary This PR implements the Rule Upgrade test plan added in elastic#205922 and elastic#203331. ## Details Tests cover only enabled customization and organized in a following way - `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.** - `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. - `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. This PR supersedes elastic#205217 and elastic#205339. * `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`. ## Flaky test runner Flaky test runs for 100 iterations were successful (elastic#209260 (comment) and elastic#209260 (comment)). (cherry picked from commit 4909770) # Conflicts: # x-pack/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/prebuilt_rule_customization/customization_enabled/upgrade_perform_prebuilt_rules.all_rules_mode.ts # x-pack/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/prebuilt_rule_customization/customization_enabled/upgrade_perform_prebuilt_rules.specific_rules_mode.ts
banderror
added a commit
that referenced
this issue
Feb 26, 2025
…part 3 (#212420) **Epic:** #174168 **Partially addresses:** #202068, #202078, #202079 **Follow-up to:** #211472 ## Summary We're cleaning up and refactoring our existing test plans for prebuilt rule customization, upgrade, and export/import workflows. Specifically, this PR: - Updates the common structure of test plans: - Adds a tip to each test plan's header to help with navigating all our test plans for prebuilt rules. - Introduces a new `Requirements` section to each one and consolidates assumptions, technical requirements and product requirements under it. - Each `Product requirements` section now contains relevant user stories and other product details. In the document flow this is kept close to the `Scenarios` section. - Updates the test plan template according to the updated structure. - Moves the documentation about customizable and non-customizable rule fields to the common info document. - Restores the lost changes from commit bde4612. No "functional" changes have been made to any test plans, such as adding, removing, or updating any scenarios. The only functional-like changes were made to the user stories when creating/updating the product requirements sections. This PR wraps up the refactoring efforts aimed to prepare the test plans for being "functionally" changed and improved in follow-up PRs. In the following PRs, we're going to cover the logic described in #210358, address any gaps in the coverage, and improve the scenarios structure and wording. ## Review tip It might be easier to review this PR commit-by-commit as each of them contains logically cohesive changes.
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Feb 26, 2025
…part 3 (elastic#212420) **Epic:** elastic#174168 **Partially addresses:** elastic#202068, elastic#202078, elastic#202079 **Follow-up to:** elastic#211472 ## Summary We're cleaning up and refactoring our existing test plans for prebuilt rule customization, upgrade, and export/import workflows. Specifically, this PR: - Updates the common structure of test plans: - Adds a tip to each test plan's header to help with navigating all our test plans for prebuilt rules. - Introduces a new `Requirements` section to each one and consolidates assumptions, technical requirements and product requirements under it. - Each `Product requirements` section now contains relevant user stories and other product details. In the document flow this is kept close to the `Scenarios` section. - Updates the test plan template according to the updated structure. - Moves the documentation about customizable and non-customizable rule fields to the common info document. - Restores the lost changes from commit elastic@bde4612. No "functional" changes have been made to any test plans, such as adding, removing, or updating any scenarios. The only functional-like changes were made to the user stories when creating/updating the product requirements sections. This PR wraps up the refactoring efforts aimed to prepare the test plans for being "functionally" changed and improved in follow-up PRs. In the following PRs, we're going to cover the logic described in elastic#210358, address any gaps in the coverage, and improve the scenarios structure and wording. ## Review tip It might be easier to review this PR commit-by-commit as each of them contains logically cohesive changes. (cherry picked from commit c961659)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Feb 26, 2025
…part 3 (elastic#212420) **Epic:** elastic#174168 **Partially addresses:** elastic#202068, elastic#202078, elastic#202079 **Follow-up to:** elastic#211472 ## Summary We're cleaning up and refactoring our existing test plans for prebuilt rule customization, upgrade, and export/import workflows. Specifically, this PR: - Updates the common structure of test plans: - Adds a tip to each test plan's header to help with navigating all our test plans for prebuilt rules. - Introduces a new `Requirements` section to each one and consolidates assumptions, technical requirements and product requirements under it. - Each `Product requirements` section now contains relevant user stories and other product details. In the document flow this is kept close to the `Scenarios` section. - Updates the test plan template according to the updated structure. - Moves the documentation about customizable and non-customizable rule fields to the common info document. - Restores the lost changes from commit elastic@bde4612. No "functional" changes have been made to any test plans, such as adding, removing, or updating any scenarios. The only functional-like changes were made to the user stories when creating/updating the product requirements sections. This PR wraps up the refactoring efforts aimed to prepare the test plans for being "functionally" changed and improved in follow-up PRs. In the following PRs, we're going to cover the logic described in elastic#210358, address any gaps in the coverage, and improve the scenarios structure and wording. ## Review tip It might be easier to review this PR commit-by-commit as each of them contains logically cohesive changes. (cherry picked from commit c961659)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Feb 26, 2025
…part 3 (elastic#212420) **Epic:** elastic#174168 **Partially addresses:** elastic#202068, elastic#202078, elastic#202079 **Follow-up to:** elastic#211472 ## Summary We're cleaning up and refactoring our existing test plans for prebuilt rule customization, upgrade, and export/import workflows. Specifically, this PR: - Updates the common structure of test plans: - Adds a tip to each test plan's header to help with navigating all our test plans for prebuilt rules. - Introduces a new `Requirements` section to each one and consolidates assumptions, technical requirements and product requirements under it. - Each `Product requirements` section now contains relevant user stories and other product details. In the document flow this is kept close to the `Scenarios` section. - Updates the test plan template according to the updated structure. - Moves the documentation about customizable and non-customizable rule fields to the common info document. - Restores the lost changes from commit elastic@bde4612. No "functional" changes have been made to any test plans, such as adding, removing, or updating any scenarios. The only functional-like changes were made to the user stories when creating/updating the product requirements sections. This PR wraps up the refactoring efforts aimed to prepare the test plans for being "functionally" changed and improved in follow-up PRs. In the following PRs, we're going to cover the logic described in elastic#210358, address any gaps in the coverage, and improve the scenarios structure and wording. ## Review tip It might be easier to review this PR commit-by-commit as each of them contains logically cohesive changes. (cherry picked from commit c961659)
kibanamachine
added a commit
that referenced
this issue
Feb 26, 2025
…lans, part 3 (#212420) (#212481) # Backport This will backport the following commits from `main` to `9.0`: - [[Security Solution] Refactor prebuilt rule customization test plans, part 3 (#212420)](#212420) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Georgii Gorbachev","email":"georgii.gorbachev@elastic.co"},"sourceCommit":{"committedDate":"2025-02-26T08:39:32Z","message":"[Security Solution] Refactor prebuilt rule customization test plans, part 3 (#212420)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\n**Partially addresses:**\nhttps://github.com//issues/202068,\nhttps://github.com//issues/202078,\nhttps://github.com//issues/202079\n**Follow-up to:** https://github.com/elastic/kibana/pull/211472\n\n## Summary\n\nWe're cleaning up and refactoring our existing test plans for prebuilt\nrule customization, upgrade, and export/import workflows.\n\nSpecifically, this PR:\n\n- Updates the common structure of test plans:\n- Adds a tip to each test plan's header to help with navigating all our\ntest plans for prebuilt rules.\n- Introduces a new `Requirements` section to each one and consolidates\nassumptions, technical requirements and product requirements under it.\n- Each `Product requirements` section now contains relevant user stories\nand other product details. In the document flow this is kept close to\nthe `Scenarios` section.\n - Updates the test plan template according to the updated structure.\n- Moves the documentation about customizable and non-customizable rule\nfields to the common info document.\n- Restores the lost changes from commit\nhttps://github.com/elastic/kibana/commit/bde4612e5256b3b779961736291ccae2e328c310.\n\nNo \"functional\" changes have been made to any test plans, such as\nadding, removing, or updating any scenarios. The only functional-like\nchanges were made to the user stories when creating/updating the product\nrequirements sections.\n\nThis PR wraps up the refactoring efforts aimed to prepare the test plans\nfor being \"functionally\" changed and improved in follow-up PRs. In the\nfollowing PRs, we're going to cover the logic described in\nhttps://github.com//issues/210358, address any gaps in the\ncoverage, and improve the scenarios structure and wording.\n\n## Review tip\n\nIt might be easier to review this PR commit-by-commit as each of them\ncontains logically cohesive changes.","sha":"c9616593c1deeb52f73fa5720f16288073943f3e","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Refactor prebuilt rule customization test plans, part 3","number":212420,"url":"https://github.com/elastic/kibana/pull/212420","mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 3 (#212420)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\n**Partially addresses:**\nhttps://github.com//issues/202068,\nhttps://github.com//issues/202078,\nhttps://github.com//issues/202079\n**Follow-up to:** https://github.com/elastic/kibana/pull/211472\n\n## Summary\n\nWe're cleaning up and refactoring our existing test plans for prebuilt\nrule customization, upgrade, and export/import workflows.\n\nSpecifically, this PR:\n\n- Updates the common structure of test plans:\n- Adds a tip to each test plan's header to help with navigating all our\ntest plans for prebuilt rules.\n- Introduces a new `Requirements` section to each one and consolidates\nassumptions, technical requirements and product requirements under it.\n- Each `Product requirements` section now contains relevant user stories\nand other product details. In the document flow this is kept close to\nthe `Scenarios` section.\n - Updates the test plan template according to the updated structure.\n- Moves the documentation about customizable and non-customizable rule\nfields to the common info document.\n- Restores the lost changes from commit\nhttps://github.com/elastic/kibana/commit/bde4612e5256b3b779961736291ccae2e328c310.\n\nNo \"functional\" changes have been made to any test plans, such as\nadding, removing, or updating any scenarios. The only functional-like\nchanges were made to the user stories when creating/updating the product\nrequirements sections.\n\nThis PR wraps up the refactoring efforts aimed to prepare the test plans\nfor being \"functionally\" changed and improved in follow-up PRs. In the\nfollowing PRs, we're going to cover the logic described in\nhttps://github.com//issues/210358, address any gaps in the\ncoverage, and improve the scenarios structure and wording.\n\n## Review tip\n\nIt might be easier to review this PR commit-by-commit as each of them\ncontains logically cohesive changes.","sha":"c9616593c1deeb52f73fa5720f16288073943f3e"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/212420","number":212420,"mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 3 (#212420)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\n**Partially addresses:**\nhttps://github.com//issues/202068,\nhttps://github.com//issues/202078,\nhttps://github.com//issues/202079\n**Follow-up to:** https://github.com/elastic/kibana/pull/211472\n\n## Summary\n\nWe're cleaning up and refactoring our existing test plans for prebuilt\nrule customization, upgrade, and export/import workflows.\n\nSpecifically, this PR:\n\n- Updates the common structure of test plans:\n- Adds a tip to each test plan's header to help with navigating all our\ntest plans for prebuilt rules.\n- Introduces a new `Requirements` section to each one and consolidates\nassumptions, technical requirements and product requirements under it.\n- Each `Product requirements` section now contains relevant user stories\nand other product details. In the document flow this is kept close to\nthe `Scenarios` section.\n - Updates the test plan template according to the updated structure.\n- Moves the documentation about customizable and non-customizable rule\nfields to the common info document.\n- Restores the lost changes from commit\nhttps://github.com/elastic/kibana/commit/bde4612e5256b3b779961736291ccae2e328c310.\n\nNo \"functional\" changes have been made to any test plans, such as\nadding, removing, or updating any scenarios. The only functional-like\nchanges were made to the user stories when creating/updating the product\nrequirements sections.\n\nThis PR wraps up the refactoring efforts aimed to prepare the test plans\nfor being \"functionally\" changed and improved in follow-up PRs. In the\nfollowing PRs, we're going to cover the logic described in\nhttps://github.com//issues/210358, address any gaps in the\ncoverage, and improve the scenarios structure and wording.\n\n## Review tip\n\nIt might be easier to review this PR commit-by-commit as each of them\ncontains logically cohesive changes.","sha":"c9616593c1deeb52f73fa5720f16288073943f3e"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
kibanamachine
added a commit
that referenced
this issue
Feb 26, 2025
…plans, part 3 (#212420) (#212479) # Backport This will backport the following commits from `main` to `8.18`: - [[Security Solution] Refactor prebuilt rule customization test plans, part 3 (#212420)](#212420) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Georgii Gorbachev","email":"georgii.gorbachev@elastic.co"},"sourceCommit":{"committedDate":"2025-02-26T08:39:32Z","message":"[Security Solution] Refactor prebuilt rule customization test plans, part 3 (#212420)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\n**Partially addresses:**\nhttps://github.com//issues/202068,\nhttps://github.com//issues/202078,\nhttps://github.com//issues/202079\n**Follow-up to:** https://github.com/elastic/kibana/pull/211472\n\n## Summary\n\nWe're cleaning up and refactoring our existing test plans for prebuilt\nrule customization, upgrade, and export/import workflows.\n\nSpecifically, this PR:\n\n- Updates the common structure of test plans:\n- Adds a tip to each test plan's header to help with navigating all our\ntest plans for prebuilt rules.\n- Introduces a new `Requirements` section to each one and consolidates\nassumptions, technical requirements and product requirements under it.\n- Each `Product requirements` section now contains relevant user stories\nand other product details. In the document flow this is kept close to\nthe `Scenarios` section.\n - Updates the test plan template according to the updated structure.\n- Moves the documentation about customizable and non-customizable rule\nfields to the common info document.\n- Restores the lost changes from commit\nhttps://github.com/elastic/kibana/commit/bde4612e5256b3b779961736291ccae2e328c310.\n\nNo \"functional\" changes have been made to any test plans, such as\nadding, removing, or updating any scenarios. The only functional-like\nchanges were made to the user stories when creating/updating the product\nrequirements sections.\n\nThis PR wraps up the refactoring efforts aimed to prepare the test plans\nfor being \"functionally\" changed and improved in follow-up PRs. In the\nfollowing PRs, we're going to cover the logic described in\nhttps://github.com//issues/210358, address any gaps in the\ncoverage, and improve the scenarios structure and wording.\n\n## Review tip\n\nIt might be easier to review this PR commit-by-commit as each of them\ncontains logically cohesive changes.","sha":"c9616593c1deeb52f73fa5720f16288073943f3e","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Refactor prebuilt rule customization test plans, part 3","number":212420,"url":"https://github.com/elastic/kibana/pull/212420","mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 3 (#212420)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\n**Partially addresses:**\nhttps://github.com//issues/202068,\nhttps://github.com//issues/202078,\nhttps://github.com//issues/202079\n**Follow-up to:** https://github.com/elastic/kibana/pull/211472\n\n## Summary\n\nWe're cleaning up and refactoring our existing test plans for prebuilt\nrule customization, upgrade, and export/import workflows.\n\nSpecifically, this PR:\n\n- Updates the common structure of test plans:\n- Adds a tip to each test plan's header to help with navigating all our\ntest plans for prebuilt rules.\n- Introduces a new `Requirements` section to each one and consolidates\nassumptions, technical requirements and product requirements under it.\n- Each `Product requirements` section now contains relevant user stories\nand other product details. In the document flow this is kept close to\nthe `Scenarios` section.\n - Updates the test plan template according to the updated structure.\n- Moves the documentation about customizable and non-customizable rule\nfields to the common info document.\n- Restores the lost changes from commit\nhttps://github.com/elastic/kibana/commit/bde4612e5256b3b779961736291ccae2e328c310.\n\nNo \"functional\" changes have been made to any test plans, such as\nadding, removing, or updating any scenarios. The only functional-like\nchanges were made to the user stories when creating/updating the product\nrequirements sections.\n\nThis PR wraps up the refactoring efforts aimed to prepare the test plans\nfor being \"functionally\" changed and improved in follow-up PRs. In the\nfollowing PRs, we're going to cover the logic described in\nhttps://github.com//issues/210358, address any gaps in the\ncoverage, and improve the scenarios structure and wording.\n\n## Review tip\n\nIt might be easier to review this PR commit-by-commit as each of them\ncontains logically cohesive changes.","sha":"c9616593c1deeb52f73fa5720f16288073943f3e"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/212420","number":212420,"mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 3 (#212420)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\n**Partially addresses:**\nhttps://github.com//issues/202068,\nhttps://github.com//issues/202078,\nhttps://github.com//issues/202079\n**Follow-up to:** https://github.com/elastic/kibana/pull/211472\n\n## Summary\n\nWe're cleaning up and refactoring our existing test plans for prebuilt\nrule customization, upgrade, and export/import workflows.\n\nSpecifically, this PR:\n\n- Updates the common structure of test plans:\n- Adds a tip to each test plan's header to help with navigating all our\ntest plans for prebuilt rules.\n- Introduces a new `Requirements` section to each one and consolidates\nassumptions, technical requirements and product requirements under it.\n- Each `Product requirements` section now contains relevant user stories\nand other product details. In the document flow this is kept close to\nthe `Scenarios` section.\n - Updates the test plan template according to the updated structure.\n- Moves the documentation about customizable and non-customizable rule\nfields to the common info document.\n- Restores the lost changes from commit\nhttps://github.com/elastic/kibana/commit/bde4612e5256b3b779961736291ccae2e328c310.\n\nNo \"functional\" changes have been made to any test plans, such as\nadding, removing, or updating any scenarios. The only functional-like\nchanges were made to the user stories when creating/updating the product\nrequirements sections.\n\nThis PR wraps up the refactoring efforts aimed to prepare the test plans\nfor being \"functionally\" changed and improved in follow-up PRs. In the\nfollowing PRs, we're going to cover the logic described in\nhttps://github.com//issues/210358, address any gaps in the\ncoverage, and improve the scenarios structure and wording.\n\n## Review tip\n\nIt might be easier to review this PR commit-by-commit as each of them\ncontains logically cohesive changes.","sha":"c9616593c1deeb52f73fa5720f16288073943f3e"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
kibanamachine
added a commit
that referenced
this issue
Feb 26, 2025
…lans, part 3 (#212420) (#212480) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Refactor prebuilt rule customization test plans, part 3 (#212420)](#212420) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Georgii Gorbachev","email":"georgii.gorbachev@elastic.co"},"sourceCommit":{"committedDate":"2025-02-26T08:39:32Z","message":"[Security Solution] Refactor prebuilt rule customization test plans, part 3 (#212420)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\n**Partially addresses:**\nhttps://github.com//issues/202068,\nhttps://github.com//issues/202078,\nhttps://github.com//issues/202079\n**Follow-up to:** https://github.com/elastic/kibana/pull/211472\n\n## Summary\n\nWe're cleaning up and refactoring our existing test plans for prebuilt\nrule customization, upgrade, and export/import workflows.\n\nSpecifically, this PR:\n\n- Updates the common structure of test plans:\n- Adds a tip to each test plan's header to help with navigating all our\ntest plans for prebuilt rules.\n- Introduces a new `Requirements` section to each one and consolidates\nassumptions, technical requirements and product requirements under it.\n- Each `Product requirements` section now contains relevant user stories\nand other product details. In the document flow this is kept close to\nthe `Scenarios` section.\n - Updates the test plan template according to the updated structure.\n- Moves the documentation about customizable and non-customizable rule\nfields to the common info document.\n- Restores the lost changes from commit\nhttps://github.com/elastic/kibana/commit/bde4612e5256b3b779961736291ccae2e328c310.\n\nNo \"functional\" changes have been made to any test plans, such as\nadding, removing, or updating any scenarios. The only functional-like\nchanges were made to the user stories when creating/updating the product\nrequirements sections.\n\nThis PR wraps up the refactoring efforts aimed to prepare the test plans\nfor being \"functionally\" changed and improved in follow-up PRs. In the\nfollowing PRs, we're going to cover the logic described in\nhttps://github.com//issues/210358, address any gaps in the\ncoverage, and improve the scenarios structure and wording.\n\n## Review tip\n\nIt might be easier to review this PR commit-by-commit as each of them\ncontains logically cohesive changes.","sha":"c9616593c1deeb52f73fa5720f16288073943f3e","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Refactor prebuilt rule customization test plans, part 3","number":212420,"url":"https://github.com/elastic/kibana/pull/212420","mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 3 (#212420)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\n**Partially addresses:**\nhttps://github.com//issues/202068,\nhttps://github.com//issues/202078,\nhttps://github.com//issues/202079\n**Follow-up to:** https://github.com/elastic/kibana/pull/211472\n\n## Summary\n\nWe're cleaning up and refactoring our existing test plans for prebuilt\nrule customization, upgrade, and export/import workflows.\n\nSpecifically, this PR:\n\n- Updates the common structure of test plans:\n- Adds a tip to each test plan's header to help with navigating all our\ntest plans for prebuilt rules.\n- Introduces a new `Requirements` section to each one and consolidates\nassumptions, technical requirements and product requirements under it.\n- Each `Product requirements` section now contains relevant user stories\nand other product details. In the document flow this is kept close to\nthe `Scenarios` section.\n - Updates the test plan template according to the updated structure.\n- Moves the documentation about customizable and non-customizable rule\nfields to the common info document.\n- Restores the lost changes from commit\nhttps://github.com/elastic/kibana/commit/bde4612e5256b3b779961736291ccae2e328c310.\n\nNo \"functional\" changes have been made to any test plans, such as\nadding, removing, or updating any scenarios. The only functional-like\nchanges were made to the user stories when creating/updating the product\nrequirements sections.\n\nThis PR wraps up the refactoring efforts aimed to prepare the test plans\nfor being \"functionally\" changed and improved in follow-up PRs. In the\nfollowing PRs, we're going to cover the logic described in\nhttps://github.com//issues/210358, address any gaps in the\ncoverage, and improve the scenarios structure and wording.\n\n## Review tip\n\nIt might be easier to review this PR commit-by-commit as each of them\ncontains logically cohesive changes.","sha":"c9616593c1deeb52f73fa5720f16288073943f3e"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/212420","number":212420,"mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 3 (#212420)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\n**Partially addresses:**\nhttps://github.com//issues/202068,\nhttps://github.com//issues/202078,\nhttps://github.com//issues/202079\n**Follow-up to:** https://github.com/elastic/kibana/pull/211472\n\n## Summary\n\nWe're cleaning up and refactoring our existing test plans for prebuilt\nrule customization, upgrade, and export/import workflows.\n\nSpecifically, this PR:\n\n- Updates the common structure of test plans:\n- Adds a tip to each test plan's header to help with navigating all our\ntest plans for prebuilt rules.\n- Introduces a new `Requirements` section to each one and consolidates\nassumptions, technical requirements and product requirements under it.\n- Each `Product requirements` section now contains relevant user stories\nand other product details. In the document flow this is kept close to\nthe `Scenarios` section.\n - Updates the test plan template according to the updated structure.\n- Moves the documentation about customizable and non-customizable rule\nfields to the common info document.\n- Restores the lost changes from commit\nhttps://github.com/elastic/kibana/commit/bde4612e5256b3b779961736291ccae2e328c310.\n\nNo \"functional\" changes have been made to any test plans, such as\nadding, removing, or updating any scenarios. The only functional-like\nchanges were made to the user stories when creating/updating the product\nrequirements sections.\n\nThis PR wraps up the refactoring efforts aimed to prepare the test plans\nfor being \"functionally\" changed and improved in follow-up PRs. In the\nfollowing PRs, we're going to cover the logic described in\nhttps://github.com//issues/210358, address any gaps in the\ncoverage, and improve the scenarios structure and wording.\n\n## Review tip\n\nIt might be easier to review this PR commit-by-commit as each of them\ncontains logically cohesive changes.","sha":"c9616593c1deeb52f73fa5720f16288073943f3e"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
JoseLuisGJ
pushed a commit
to JoseLuisGJ/kibana
that referenced
this issue
Feb 27, 2025
…t rule customization feature (elastic#209260) **Addresses:** elastic#202078 **Partially implements test plan:** elastic#205922 ## Summary This PR implements the Rule Upgrade test plan added in elastic#205922 and elastic#203331. ## Details Tests cover only enabled customization and organized in a following way - `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.** - `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. - `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. This PR supersedes elastic#205217 and elastic#205339. * `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`. ## Flaky test runner Flaky test runs for 100 iterations were successful (elastic#209260 (comment) and elastic#209260 (comment)).
JoseLuisGJ
pushed a commit
to JoseLuisGJ/kibana
that referenced
this issue
Feb 27, 2025
…part 3 (elastic#212420) **Epic:** elastic#174168 **Partially addresses:** elastic#202068, elastic#202078, elastic#202079 **Follow-up to:** elastic#211472 ## Summary We're cleaning up and refactoring our existing test plans for prebuilt rule customization, upgrade, and export/import workflows. Specifically, this PR: - Updates the common structure of test plans: - Adds a tip to each test plan's header to help with navigating all our test plans for prebuilt rules. - Introduces a new `Requirements` section to each one and consolidates assumptions, technical requirements and product requirements under it. - Each `Product requirements` section now contains relevant user stories and other product details. In the document flow this is kept close to the `Scenarios` section. - Updates the test plan template according to the updated structure. - Moves the documentation about customizable and non-customizable rule fields to the common info document. - Restores the lost changes from commit elastic@bde4612. No "functional" changes have been made to any test plans, such as adding, removing, or updating any scenarios. The only functional-like changes were made to the user stories when creating/updating the product requirements sections. This PR wraps up the refactoring efforts aimed to prepare the test plans for being "functionally" changed and improved in follow-up PRs. In the following PRs, we're going to cover the logic described in elastic#210358, address any gaps in the coverage, and improve the scenarios structure and wording. ## Review tip It might be easier to review this PR commit-by-commit as each of them contains logically cohesive changes.
SoniaSanzV
pushed a commit
to SoniaSanzV/kibana
that referenced
this issue
Mar 4, 2025
…lans, part 3 (elastic#212420) (elastic#212480) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Refactor prebuilt rule customization test plans, part 3 (elastic#212420)](elastic#212420) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Georgii Gorbachev","email":"georgii.gorbachev@elastic.co"},"sourceCommit":{"committedDate":"2025-02-26T08:39:32Z","message":"[Security Solution] Refactor prebuilt rule customization test plans, part 3 (elastic#212420)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\n**Partially addresses:**\nhttps://github.com/elastic/issues/202068,\nhttps://github.com/elastic/issues/202078,\nhttps://github.com/elastic/issues/202079\n**Follow-up to:** https://github.com/elastic/kibana/pull/211472\n\n## Summary\n\nWe're cleaning up and refactoring our existing test plans for prebuilt\nrule customization, upgrade, and export/import workflows.\n\nSpecifically, this PR:\n\n- Updates the common structure of test plans:\n- Adds a tip to each test plan's header to help with navigating all our\ntest plans for prebuilt rules.\n- Introduces a new `Requirements` section to each one and consolidates\nassumptions, technical requirements and product requirements under it.\n- Each `Product requirements` section now contains relevant user stories\nand other product details. In the document flow this is kept close to\nthe `Scenarios` section.\n - Updates the test plan template according to the updated structure.\n- Moves the documentation about customizable and non-customizable rule\nfields to the common info document.\n- Restores the lost changes from commit\nhttps://github.com/elastic/kibana/commit/bde4612e5256b3b779961736291ccae2e328c310.\n\nNo \"functional\" changes have been made to any test plans, such as\nadding, removing, or updating any scenarios. The only functional-like\nchanges were made to the user stories when creating/updating the product\nrequirements sections.\n\nThis PR wraps up the refactoring efforts aimed to prepare the test plans\nfor being \"functionally\" changed and improved in follow-up PRs. In the\nfollowing PRs, we're going to cover the logic described in\nhttps://github.com/elastic/issues/210358, address any gaps in the\ncoverage, and improve the scenarios structure and wording.\n\n## Review tip\n\nIt might be easier to review this PR commit-by-commit as each of them\ncontains logically cohesive changes.","sha":"c9616593c1deeb52f73fa5720f16288073943f3e","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Refactor prebuilt rule customization test plans, part 3","number":212420,"url":"https://github.com/elastic/kibana/pull/212420","mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 3 (elastic#212420)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\n**Partially addresses:**\nhttps://github.com/elastic/issues/202068,\nhttps://github.com/elastic/issues/202078,\nhttps://github.com/elastic/issues/202079\n**Follow-up to:** https://github.com/elastic/kibana/pull/211472\n\n## Summary\n\nWe're cleaning up and refactoring our existing test plans for prebuilt\nrule customization, upgrade, and export/import workflows.\n\nSpecifically, this PR:\n\n- Updates the common structure of test plans:\n- Adds a tip to each test plan's header to help with navigating all our\ntest plans for prebuilt rules.\n- Introduces a new `Requirements` section to each one and consolidates\nassumptions, technical requirements and product requirements under it.\n- Each `Product requirements` section now contains relevant user stories\nand other product details. In the document flow this is kept close to\nthe `Scenarios` section.\n - Updates the test plan template according to the updated structure.\n- Moves the documentation about customizable and non-customizable rule\nfields to the common info document.\n- Restores the lost changes from commit\nhttps://github.com/elastic/kibana/commit/bde4612e5256b3b779961736291ccae2e328c310.\n\nNo \"functional\" changes have been made to any test plans, such as\nadding, removing, or updating any scenarios. The only functional-like\nchanges were made to the user stories when creating/updating the product\nrequirements sections.\n\nThis PR wraps up the refactoring efforts aimed to prepare the test plans\nfor being \"functionally\" changed and improved in follow-up PRs. In the\nfollowing PRs, we're going to cover the logic described in\nhttps://github.com/elastic/issues/210358, address any gaps in the\ncoverage, and improve the scenarios structure and wording.\n\n## Review tip\n\nIt might be easier to review this PR commit-by-commit as each of them\ncontains logically cohesive changes.","sha":"c9616593c1deeb52f73fa5720f16288073943f3e"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/212420","number":212420,"mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 3 (elastic#212420)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\n**Partially addresses:**\nhttps://github.com/elastic/issues/202068,\nhttps://github.com/elastic/issues/202078,\nhttps://github.com/elastic/issues/202079\n**Follow-up to:** https://github.com/elastic/kibana/pull/211472\n\n## Summary\n\nWe're cleaning up and refactoring our existing test plans for prebuilt\nrule customization, upgrade, and export/import workflows.\n\nSpecifically, this PR:\n\n- Updates the common structure of test plans:\n- Adds a tip to each test plan's header to help with navigating all our\ntest plans for prebuilt rules.\n- Introduces a new `Requirements` section to each one and consolidates\nassumptions, technical requirements and product requirements under it.\n- Each `Product requirements` section now contains relevant user stories\nand other product details. In the document flow this is kept close to\nthe `Scenarios` section.\n - Updates the test plan template according to the updated structure.\n- Moves the documentation about customizable and non-customizable rule\nfields to the common info document.\n- Restores the lost changes from commit\nhttps://github.com/elastic/kibana/commit/bde4612e5256b3b779961736291ccae2e328c310.\n\nNo \"functional\" changes have been made to any test plans, such as\nadding, removing, or updating any scenarios. The only functional-like\nchanges were made to the user stories when creating/updating the product\nrequirements sections.\n\nThis PR wraps up the refactoring efforts aimed to prepare the test plans\nfor being \"functionally\" changed and improved in follow-up PRs. In the\nfollowing PRs, we're going to cover the logic described in\nhttps://github.com/elastic/issues/210358, address any gaps in the\ncoverage, and improve the scenarios structure and wording.\n\n## Review tip\n\nIt might be easier to review this PR commit-by-commit as each of them\ncontains logically cohesive changes.","sha":"c9616593c1deeb52f73fa5720f16288073943f3e"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Epics: https://github.com/elastic/security-team/issues/1974 (internal), #174168
Summary
Write test plans (one or a few) and create a comprehensive suite of automated tests for the workflow of upgrading prebuilt rules.
Features to cover:
Please cover both the features under the feature flag turned ON and OFF.
Related tickets
Related functional tickets to cover with tests:
POST /upgrade/_reviewAPI endpoint's contract and functionality #180153/upgrade/_reviewupgrade workflow #180393POST /upgrade/_performAPI endpoint's contract and functionality #166376/upgrade/_performendpoint upgrade workflow #186544exceptions_list,authorandlicensefrom Diffable Rule #196213Related bugs to cover with tests:
baseversion #200904Related PRs
Test plans for diff algorithms
data_sourcefield diff algorithm test plan #189669queryfields diff algorithm #192529typefield diff algorithm #193372Test plans for prebuilt rule upgrade workflow
Test coverage for prebuilt rule upgrade workflow
The text was updated successfully, but these errors were encountered: