diff --git a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/README.md b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/README.md index 610c9a98947c9..f411a60d32b47 100644 --- a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/README.md +++ b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/README.md @@ -37,6 +37,6 @@ Each subdomain folder should be owned by a single GitHub team in the `.github/CO Some examples for reference: - [Test plan template](./test_plan_template.md). -- [Installation and Upgrade of Prebuilt Rules](./detection_response/prebuilt_rules/installation_and_upgrade.md). +- [Test plans for prebuilt rules](./detection_response/prebuilt_rules/prebuilt_rules.md). Feel free to tune the structure whenever it makes sense and improves readability or maintainability of your plan: add more sections to `Useful info`, add more top-level sections in addition to `Useful info` and `Scenarios`, etc. diff --git a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_customization.md b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_customization.md index bbd7e9de91075..bfd887265afb7 100644 --- a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_customization.md +++ b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_customization.md @@ -2,6 +2,9 @@ **Status**: `in progress`, matches [Milestone 3](https://github.com/elastic/kibana/issues/174168). +> [!TIP] +> If you're new to prebuilt rules, get started [here](./prebuilt_rules.md) and check an overview of the features of prebuilt rules in [this section](./prebuilt_rules_common_info.md#features). + ## Summary This is a test plan for the workflows of customizing prebuilt rules via: @@ -24,10 +27,11 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one - [Useful information](#useful-information) - [Tickets](#tickets) - - [User stories](#user-stories) - [Terminology](#terminology) +- [Requirements](#requirements) - [Assumptions](#assumptions) - - [Non-functional requirements](#non-functional-requirements) + - [Technical requirements](#technical-requirements) + - [Product requirements](#product-requirements) - [Scenarios](#scenarios) - [Editing prebuilt rules](#editing-prebuilt-rules) - [**Scenario: User can edit a non-customized prebuilt rule from the rule edit page**](#scenario-user-can-edit-a-non-customized-prebuilt-rule-from-the-rule-edit-page) @@ -60,9 +64,35 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one - [Users can Customize Prebuilt Detection Rules: Milestone 3](https://github.com/elastic/kibana/issues/174168) - [Tests for prebuilt rule customization workflow](https://github.com/elastic/kibana/issues/202068) -### User stories +### Terminology + +- [Common terminology](./prebuilt_rules_common_info.md#common-terminology). +- **Rule source**, or **`ruleSource`**: a rule field that defines the rule's origin. Can be `internal` or `external`. Currently, custom rules have `internal` rule source and prebuilt rules have `external` rule source. +- **`is_customized`**: a field within `ruleSource` that exists when rule source is set to `external`. It is a boolean value based on if the rule has been changed from its base version. +- **non-semantic change**: a change to a rule field that is functionally different. We normalize certain fields so for a time-related field such as `from`, `1m` vs `60s` are treated as the same value. We also trim leading and trailing whitespace for query fields. + +## Requirements + +### Assumptions + +Assumptions about test environments and scenarios outlined in this test plan. + +- [Common assumptions](./prebuilt_rules_common_info.md#common-assumptions). +- Rule package used will have all previous rule versions present (no missing base versions). + +### Technical requirements + +Non-functional requirements for the functionality outlined in this test plan. + +- [Common technical requirements](./prebuilt_rules_common_info.md#common-technical-requirements). -**Prebuilt rule customization workflow:** +### Product requirements + +Functional requirements for the functionality outlined in this test plan. + +- [Common product requirements](./prebuilt_rules_common_info.md#common-product-requirements). + +User stories: - User can edit a single prebuilt rule from the Rule Details page. - User can edit single prebuilt rules one-by-one from the Rule Management page. @@ -76,22 +106,6 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one - User can see which rules are customized on the Rule Management page in the Upgrade table. - User can un-customize a prebuilt rule by editing it and reverting its parameters back to their original values. -### Terminology - -- [Common terminology](./prebuilt_rules_common_info.md#common-terminology). -- **Rule source**, or **`ruleSource`**: a rule field that defines the rule's origin. Can be `internal` or `external`. Currently, custom rules have `internal` rule source and prebuilt rules have `external` rule source. -- **`is_customized`**: a field within `ruleSource` that exists when rule source is set to `external`. It is a boolean value based on if the rule has been changed from its base version. -- **non-semantic change**: a change to a rule field that is functionally different. We normalize certain fields so for a time-related field such as `from`, `1m` vs `60s` are treated as the same value. We also trim leading and trailing whitespace for query fields. - -### Assumptions - -- [Common assumptions](./prebuilt_rules_common_info.md#common-assumptions). -- Rule package used will have all previous rule versions present (no missing base versions) - -### Non-functional requirements - -- [Common non-functional requirements](./prebuilt_rules_common_info.md#common-non-functional-requirements). - ## Scenarios ### Editing prebuilt rules diff --git a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_export.md b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_export.md index 8819a7db15466..81870f3d349eb 100644 --- a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_export.md +++ b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_export.md @@ -2,6 +2,9 @@ **Status**: `in progress`, matches [Milestone 3](https://github.com/elastic/kibana/issues/174168). +> [!TIP] +> If you're new to prebuilt rules, get started [here](./prebuilt_rules.md) and check an overview of the features of prebuilt rules in [this section](./prebuilt_rules_common_info.md#features). + ## Summary This is a test plan for the workflows of: @@ -25,10 +28,11 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one - [Useful information](#useful-information) - [Tickets](#tickets) - - [User stories](#user-stories) - [Terminology](#terminology) +- [Requirements](#requirements) - [Assumptions](#assumptions) - - [Non-functional requirements](#non-functional-requirements) + - [Technical requirements](#technical-requirements) + - [Product requirements](#product-requirements) - [Scenarios](#scenarios) - [Core Functionality](#core-functionality) - [Scenario: Exporting prebuilt rule individually from rule details page](#scenario-exporting-prebuilt-rule-individually-from-rule-details-page) @@ -52,28 +56,38 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one - [Support exporting prebuilt rules from the Rule Details page](https://github.com/elastic/kibana/issues/180176) - [Tests for prebuilt rule import/export workflow](https://github.com/elastic/kibana/issues/202079) -### User stories - -**Prebuilt rule export workflow:** - -- User can export a single prebuilt rule from the Rule Details page. -- User can export multiple prebuilt rules one-by-one from the Rule Management page. -- User can export multiple prebuilt rules in bulk from the Rule Management page via bulk actions. -- User can export prebuilt non-customized rules. -- User can export prebuilt customized rules. -- User can export any combination of prebuilt non-customized, prebuilt customized, and custom rules. - ### Terminology - [Common terminology](./prebuilt_rules_common_info.md#common-terminology). +## Requirements + ### Assumptions +Assumptions about test environments and scenarios outlined in this test plan. + - [Common assumptions](./prebuilt_rules_common_info.md#common-assumptions). -### Non-functional requirements +### Technical requirements -- [Common non-functional requirements](./prebuilt_rules_common_info.md#common-non-functional-requirements). +Non-functional requirements for the functionality outlined in this test plan. + +- [Common technical requirements](./prebuilt_rules_common_info.md#common-technical-requirements). + +### Product requirements + +Functional requirements for the functionality outlined in this test plan. + +- [Common product requirements](./prebuilt_rules_common_info.md#common-product-requirements). + +User stories: + +- User can export a single prebuilt rule from the Rule Details page. +- User can export multiple prebuilt rules one-by-one from the Rule Management page. +- User can export multiple prebuilt rules in bulk from the Rule Management page via bulk actions. +- User can export prebuilt non-customized rules. +- User can export prebuilt customized rules. +- User can export any combination of prebuilt non-customized, prebuilt customized, and custom rules. ## Scenarios diff --git a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_import.md b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_import.md index 4896c54d7dd26..ab45ce56e9670 100644 --- a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_import.md +++ b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_import.md @@ -2,6 +2,9 @@ **Status**: `in progress`, matches [Milestone 3](https://github.com/elastic/kibana/issues/174168). +> [!TIP] +> If you're new to prebuilt rules, get started [here](./prebuilt_rules.md) and check an overview of the features of prebuilt rules in [this section](./prebuilt_rules_common_info.md#features). + ## Summary This is a test plan for the workflows of: @@ -21,26 +24,28 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one - [Useful information](#useful-information) - [Tickets](#tickets) - - [User stories](#user-stories) - [Terminology](#terminology) +- [Requirements](#requirements) - [Assumptions](#assumptions) - - [Non-functional requirements](#non-functional-requirements) + - [Technical requirements](#technical-requirements) + - [Product requirements](#product-requirements) - [Scenarios](#scenarios) - [Core Functionality](#core-functionality) - - [Scenario: Importing an unmodified prebuilt rule with a matching rule_id and version](#scenario-importing-an-unmodified-prebuilt-rule-with-a-matching-rule_id-and-version) - - [Scenario: Importing a customized prebuilt rule with a matching rule_id and version](#scenario-importing-a-customized-prebuilt-rule-with-a-matching-rule_id-and-version) - - [Scenario: Importing a custom rule with a matching rule_id and version](#scenario-importing-a-custom-rule-with-a-matching-rule_id-and-version) - - [Scenario: Importing a prebuilt rule with a matching rule_id but no matching version](#scenario-importing-a-prebuilt-rule-with-a-matching-rule_id-but-no-matching-version) - - [Scenario: Importing a prebuilt rule with a non-existent rule_id](#scenario-importing-a-prebuilt-rule-with-a-non-existent-rule_id) - - [Scenario: Importing a prebuilt rule without a rule_id field](#scenario-importing-a-prebuilt-rule-without-a-rule_id-field) - - [Scenario: Importing a prebuilt rule with a matching rule_id but missing a version field](#scenario-importing-a-prebuilt-rule-with-a-matching-rule_id-but-missing-a-version-field) - - [Scenario: Importing an existing custom rule missing a version field](#scenario-importing-an-existing-custom-rule-missing-a-version-field) - - [Scenario: Importing a new custom rule missing a version field](#scenario-importing-a-new-custom-rule-missing-a-version-field) - - [Scenario: Importing a rule with overwrite flag set to true](#scenario-importing-a-rule-with-overwrite-flag-set-to-true) - - [Scenario: Importing a rule with overwrite flag set to false](#scenario-importing-a-rule-with-overwrite-flag-set-to-false) + - [Scenario: Importing a non-customized prebuilt rule with a matching `rule_id` and `version`](#scenario-importing-a-non-customized-prebuilt-rule-with-a-matching-rule_id-and-version) + - [Scenario: Importing a customized prebuilt rule with a matching `rule_id` and `version`](#scenario-importing-a-customized-prebuilt-rule-with-a-matching-rule_id-and-version) + - [Scenario: Importing a custom rule with a matching prebuilt `rule_id` and `version`](#scenario-importing-a-custom-rule-with-a-matching-prebuilt-rule_id-and-version) + - [Scenario: Importing a custom rule with a matching custom `rule_id` and `version`](#scenario-importing-a-custom-rule-with-a-matching-custom-rule_id-and-version) + - [Scenario: Importing a prebuilt rule with a matching `rule_id` but no matching `version`](#scenario-importing-a-prebuilt-rule-with-a-matching-rule_id-but-no-matching-version) + - [Scenario: Importing a prebuilt rule with a non-existent `rule_id`](#scenario-importing-a-prebuilt-rule-with-a-non-existent-rule_id) + - [Scenario: Importing a prebuilt rule without a `rule_id` field](#scenario-importing-a-prebuilt-rule-without-a-rule_id-field) + - [Scenario: Importing a prebuilt rule with a matching `rule_id` but missing a `version` field](#scenario-importing-a-prebuilt-rule-with-a-matching-rule_id-but-missing-a-version-field) + - [Scenario: Importing an existing custom rule missing a `version` field](#scenario-importing-an-existing-custom-rule-missing-a-version-field) + - [Scenario: Importing a new custom rule missing a `version` field](#scenario-importing-a-new-custom-rule-missing-a-version-field) + - [Scenario: Importing a rule with `overwrite` flag set to true](#scenario-importing-a-rule-with-overwrite-flag-set-to-true) + - [Scenario: Importing a rule with `overwrite` flag set to false](#scenario-importing-a-rule-with-overwrite-flag-set-to-false) - [Scenario: Importing both custom and prebuilt rules](#scenario-importing-both-custom-and-prebuilt-rules) - - [Scenario: Importing prebuilt rules when the rules package is not installed](#scenario-importing-prebuilt-rules-when-the-rules-package-is-not-installed) - - [Scenario: User imports a custom rule before a prebuilt rule asset is created with the same rule_id](#scenario-user-imports-a-custom-rule-before-a-prebuilt-rule-asset-is-created-with-the-same-rule_id) + - [Scenario: Importing a prebuilt rule when the rules package is not installed](#scenario-importing-a-prebuilt-rule-when-the-rules-package-is-not-installed) + - [Scenario: Importing a custom rule before a prebuilt rule asset is created with the same `rule_id`](#scenario-importing-a-custom-rule-before-a-prebuilt-rule-asset-is-created-with-the-same-rule_id) ## Useful information @@ -52,50 +57,61 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one - [Benchmark performance of importing a large number of prebuilt rules](https://github.com/elastic/kibana/issues/195632) - [Tests for prebuilt rule import/export workflow](https://github.com/elastic/kibana/issues/202079) -### User stories - -**Prebuilt rule import workflow:** - -- User can import a single prebuilt rule on the Rule Management page. -- User can import multiple prebuilt rules on the Rule Management page. -- User can import prebuilt non-customized rules. -- User can import prebuilt customized rules. -- User can import any combination of prebuilt non-customized, prebuilt customized, and custom rules. - ### Terminology - [Common terminology](./prebuilt_rules_common_info.md#common-terminology). +## Requirements + ### Assumptions +Assumptions about test environments and scenarios outlined in this test plan. + - [Common assumptions](./prebuilt_rules_common_info.md#common-assumptions). -### Non-functional requirements +### Technical requirements + +Non-functional requirements for the functionality outlined in this test plan. + +- [Common technical requirements](./prebuilt_rules_common_info.md#common-technical-requirements). + +### Product requirements + +Functional requirements for the functionality outlined in this test plan. -- [Common non-functional requirements](./prebuilt_rules_common_info.md#common-non-functional-requirements). +- [Common product requirements](./prebuilt_rules_common_info.md#common-product-requirements). + +User stories: + +- User can import a single prebuilt rule on the Rule Management page. +- User can import multiple prebuilt rules on the Rule Management page. +- User can import prebuilt non-customized rules. +- User can import prebuilt customized rules. +- User can import any combination of prebuilt non-customized, prebuilt customized, and custom rules. ## Scenarios ### Core Functionality -#### Scenario: Importing an unmodified prebuilt rule with a matching rule_id and version +#### Scenario: Importing a non-customized prebuilt rule with a matching `rule_id` and `version` **Automation**: 1 cypress test and 1 integration test. ```Gherkin -Given the import payload contains an unmodified prebuilt rule +Given the import payload contains a non-customized prebuilt rule And its rule_id and version match a rule asset from the installed package +When the user imports the rule Then the rule should be created or updated And the ruleSource type should be "external" And isCustomized should be false ``` -#### Scenario: Importing a customized prebuilt rule with a matching rule_id and version +#### Scenario: Importing a customized prebuilt rule with a matching `rule_id` and `version` **Automation**: 1 cypress test and 1 integration test. ```Gherkin -Given the import payload contains a modified prebuilt rule +Given the import payload contains a customized prebuilt rule And its rule_id and version match a rule asset from the installed package When the user imports the rule Then the rule should be created or updated @@ -103,56 +119,59 @@ And the ruleSource type should be "external" And isCustomized should be true ``` -#### Scenario: Importing a custom rule with a matching prebuilt rule_id and version +#### Scenario: Importing a custom rule with a matching prebuilt `rule_id` and `version` **Automation**: 1 cypress test and 1 integration test. ```Gherkin -Given the import payload contains a custom rule with a matching rule_id and version +Given the import payload contains a custom rule +And its rule_id and version match a rule asset from the installed package When the user imports the rule Then the rule should be created or updated And the ruleSource type should be "external" +And isCustomized should be true ``` -#### Scenario: Importing a custom rule with a matching custom rule_id and version +#### Scenario: Importing a custom rule with a matching custom `rule_id` and `version` **Automation**: 1 cypress test and 1 integration test. ```Gherkin -Given the import payload contains a custom rule with a matching rule_id and version +Given the import payload contains a custom rule +And its rule_id and version match a rule asset from the installed package And the overwrite flag is set to true When the user imports the rule Then the rule should be created or updated And the ruleSource type should be "internal" ``` -#### Scenario: Importing a prebuilt rule with a matching rule_id but no matching version +#### Scenario: Importing a prebuilt rule with a matching `rule_id` but no matching `version` **Automation**: 1 integration test. ```Gherkin Given the import payload contains a prebuilt rule -And its rule_id matches a rule asset from the installed package -And the version does not match the rule asset's version +And its rule_id matches one or a few rule assets from the installed package +And its version does NOT match any of those rule assets When the user imports the rule Then the rule should be created or updated And the ruleSource type should be "external" And isCustomized should be true ``` -#### Scenario: Importing a prebuilt rule with a non-existent rule_id +#### Scenario: Importing a prebuilt rule with a non-existent `rule_id` **Automation**: 1 integration test. ```Gherkin Given the import payload contains a prebuilt rule -And its rule_id does NOT match a rule asset from the installed package +And its rule_id does NOT match any rule assets from the installed package When the user imports the rule Then the rule should be created And the ruleSource type should be "internal" ``` -#### Scenario: Importing a prebuilt rule without a rule_id field +#### Scenario: Importing a prebuilt rule without a `rule_id` field **Automation**: 1 integration test. @@ -162,41 +181,46 @@ When the user imports the rule Then the import should be rejected with a message "rule_id field is required" ``` -#### Scenario: Importing a prebuilt rule with a matching rule_id but missing a version field +#### Scenario: Importing a prebuilt rule with a matching `rule_id` but missing a `version` field **Automation**: 1 integration test. ```Gherkin Given the import payload contains a prebuilt rule without a version field +And its rule_id matches one or a few rule assets from the installed package When the user imports the rule Then the import should be rejected with a message "version field is required" ``` -#### Scenario: Importing an existing custom rule missing a version field +#### Scenario: Importing an existing custom rule missing a `version` field **Automation**: 1 integration test. ```Gherkin -Given the import payload contains an existing custom rule without a version field +Given the import payload contains a custom rule without a version field +And its rule_id does NOT match any rule assets from the installed package +And this custom rule has already been created When the user imports the rule Then the rule should be updated And the ruleSource type should be "internal" And the "version" field should be set to the existing rule's "version" ``` -#### Scenario: Importing a new custom rule missing a version field +#### Scenario: Importing a new custom rule missing a `version` field **Automation**: 1 integration test. ```Gherkin -Given the import payload contains a new custom rule without a version field +Given the import payload contains a custom rule without a version field +And its rule_id does NOT match any rule assets from the installed package +And this custom rule hasn't been created yet When the user imports the rule Then the rule should be created And the ruleSource type should be "internal" And the "version" field should be set to 1 ``` -#### Scenario: Importing a rule with overwrite flag set to true +#### Scenario: Importing a rule with `overwrite` flag set to true **Automation**: 1 integration test. @@ -209,7 +233,7 @@ Then the rule should be overwritten And the ruleSource should be based on rule_id and version ``` -#### Scenario: Importing a rule with overwrite flag set to false +#### Scenario: Importing a rule with `overwrite` flag set to false **Automation**: 1 integration test. @@ -228,25 +252,28 @@ CASE: should have the same outcome for all rule types **Automation**: 1 integration test. ```Gherkin -Given the import payload contains modified and unmodified, custom and prebuilt rules -When the user imports the rule +Given the import payload contains prebuilt non-customized, prebuilt customized, and custom rules +When the user imports these rules Then custom rules should be created or updated, with versions defaulted to 1 And prebuilt rules should be created or updated, And prebuilt rules missing versions should be rejected ``` -#### Scenario: Importing prebuilt rules when the rules package is not installed +#### Scenario: Importing a prebuilt rule when the rules package is not installed **Automation**: 1 integration test. ```Gherkin -Given the import payload contains prebuilt rules -And no rules package has been installed locally +Given the import payload contains a prebuilt rule +And its rule_id matches one or a few rule assets from the latest package +And the package hasn't been installed yet When the user imports the rule -Then the latest prebuilt rules package should get installed automatically +Then the latest package should get installed automatically +And the rule should be created or updated +And the ruleSource type should be "external" ``` -#### Scenario: User imports a custom rule before a prebuilt rule asset is created with the same rule_id +#### Scenario: Importing a custom rule before a prebuilt rule asset is created with the same `rule_id` **Automation**: 1 integration test. diff --git a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_installation.md b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_installation.md index 56de7b5d64d7c..53c8126b81c29 100644 --- a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_installation.md +++ b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_installation.md @@ -2,6 +2,9 @@ **Status**: `in progress`, matches [Milestone 3](https://github.com/elastic/kibana/issues/174168). +> [!TIP] +> If you're new to prebuilt rules, get started [here](./prebuilt_rules.md) and check an overview of the features of prebuilt rules in [this section](./prebuilt_rules_common_info.md#features). + ## Summary This is a test plan for the workflows of: @@ -20,11 +23,11 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one - [Useful information](#useful-information) - [Tickets](#tickets) - - [User stories](#user-stories) - [Terminology](#terminology) +- [Requirements](#requirements) - [Assumptions](#assumptions) - - [Non-functional requirements](#non-functional-requirements) - - [Functional requirements](#functional-requirements) + - [Technical requirements](#technical-requirements) + - [Product requirements](#product-requirements) - [Scenarios](#scenarios) - [Rule installation notifications on the Rule Management page](#rule-installation-notifications-on-the-rule-management-page) - [**Scenario: User is NOT notified when no prebuilt rules are installed and there are no prebuilt rules assets**](#scenario-user-is-not-notified-when-no-prebuilt-rules-are-installed-and-there-are-no-prebuilt-rules-assets) @@ -63,34 +66,46 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one - [Users can Customize Prebuilt Detection Rules: Milestone 2](https://github.com/elastic/kibana/issues/174167) - [Users can Customize Prebuilt Detection Rules: Milestone 3](https://github.com/elastic/kibana/issues/174168) -### User stories - -**Prebuilt rule installation workflow:** - -- User can install single prebuilt rules one-by-one from the Rule Installation page. -- User can install multiple prebuilt rules in bulk from the Rule Installation page. -- User can install all available prebuilt rules in bulk from the Rule Installation page. -- User can preview properties of a prebuilt rule before installing it. - ### Terminology - [Common terminology](./prebuilt_rules_common_info.md#common-terminology). - **CTA to install prebuilt rules**: a link button with a counter on the Rule Management page. - **CTA to upgrade prebuilt rules**: a tab with a counter on the Rule Management page. +## Requirements + ### Assumptions +Assumptions about test environments and scenarios outlined in this test plan. + - [Common assumptions](./prebuilt_rules_common_info.md#common-assumptions). -### Non-functional requirements +### Technical requirements + +Non-functional requirements for the functionality outlined in this test plan. + +- [Common technical requirements](./prebuilt_rules_common_info.md#common-technical-requirements). -- [Common non-functional requirements](./prebuilt_rules_common_info.md#common-non-functional-requirements). +### Product requirements -### Functional requirements +Functional requirements for the functionality outlined in this test plan. -- Users should be able to install prebuilt rules on the `Basic` license and higher. -- User should be able to install prebuilt rules with and without previewing what exactly they would install (rule properties). -- If user chooses to preview a prebuilt rule to be installed/upgraded, we currently show this preview in a flyout. +- [Common product requirements](./prebuilt_rules_common_info.md#common-product-requirements). + +User stories for the main rule installation features: + +- User can install single prebuilt rules one-by-one from the Rule Installation page. +- User can install multiple prebuilt rules in bulk from the Rule Installation page. +- User can install all available prebuilt rules in bulk from the Rule Installation page. +- User can preview properties of a prebuilt rule before installing it. + +User stories for licensing and RBAC: + +- User can install prebuilt rules on the `Basic` license and higher. + +Previewing properties of a prebuilt rule before installing it: + +- If user chooses to preview a prebuilt rule to be installed, we currently show this preview in a flyout. - In the prebuilt rule preview a tab that doesn't have any sections should not be displayed and a section that doesn't have any properties also should not be displayed. Examples of rule properties we show in the prebuilt rule preview flyout: @@ -555,4 +570,4 @@ And no prebuilt rules are installed When user opens the Add Rules page Then user should see prebuilt rules available to install But user should not be able to install them -``` \ No newline at end of file +``` diff --git a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_diff_algorithms.md b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_diff_algorithms.md index 381e207c3c47a..32a7e4660ddb3 100644 --- a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_diff_algorithms.md +++ b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_diff_algorithms.md @@ -2,6 +2,9 @@ **Status**: `in progress`, matches [Milestone 3](https://github.com/elastic/kibana/issues/174168). +> [!TIP] +> If you're new to prebuilt rules, get started [here](./prebuilt_rules.md) and check an overview of the features of prebuilt rules in [this section](./prebuilt_rules_common_info.md#features). + ## Summary This is a test plan for the diff algorithms used in the workflows of upgrading prebuilt rules and specifically in the `upgrade/_review` endpoint. @@ -18,8 +21,10 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one - [Useful information](#useful-information) - [Tickets](#tickets) - [Terminology](#terminology) +- [Requirements](#requirements) - [Assumptions](#assumptions) - - [Non-functional requirements](#non-functional-requirements) + - [Technical requirements](#technical-requirements) + - [Product requirements](#product-requirements) - [Scenarios](#scenarios) - [Rule field doesn't have an update and has no custom value - `AAA`](#rule-field-doesnt-have-an-update-and-has-no-custom-value---aaa) - [**Scenario: `AAA` - Rule field is any type**](#scenario-aaa---rule-field-is-any-type) @@ -76,15 +81,27 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one - `eql_query`: an object that contains a `query` string field, a `language` field that always has the value: `eql`, and a `filters` field that contains an array of kibana filters. - `esql_query`: an object that contains a `query` string field and a `language` field that always has the value: `esql`. +## Requirements + ### Assumptions +Assumptions about test environments and scenarios outlined in this test plan. + - [Common assumptions](./prebuilt_rules_common_info.md#common-assumptions). - All scenarios will contain at least 1 prebuilt rule installed in Kibana. - A new version will be available for rule(s). -### Non-functional requirements +### Technical requirements + +Non-functional requirements for the functionality outlined in this test plan. + +- [Common technical requirements](./prebuilt_rules_common_info.md#common-technical-requirements). + +### Product requirements + +Functional requirements for the functionality outlined in this test plan. -- [Common non-functional requirements](./prebuilt_rules_common_info.md#common-non-functional-requirements). +- [Common product requirements](./prebuilt_rules_common_info.md#common-product-requirements). ## Scenarios diff --git a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md index 759bd352b17da..0ad1a8a8ed0a8 100644 --- a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md +++ b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md @@ -2,6 +2,9 @@ **Status**: `in progress`, matches [Milestone 3](https://github.com/elastic/kibana/issues/174168). +> [!TIP] +> If you're new to prebuilt rules, get started [here](./prebuilt_rules.md) and check an overview of the features of prebuilt rules in [this section](./prebuilt_rules_common_info.md#features). + ## Summary This is a test plan for the workflow of: @@ -19,11 +22,11 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one - [Useful information](#useful-information) - [Tickets](#tickets) - - [User stories](#user-stories) - [Terminology](#terminology) +- [Requirements](#requirements) - [Assumptions](#assumptions) - - [Non-functional requirements](#non-functional-requirements) - - [Functional requirements](#functional-requirements) + - [Technical requirements](#technical-requirements) + - [Product requirements](#product-requirements) - [Scenarios](#scenarios) - [Rule upgrade field preview](#rule-upgrade-field-preview) - [Preview non-customized field that has an upgrade (AAB)](#preview-non-customized-field-that-has-an-upgrade-aab) @@ -68,51 +71,60 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one - [Users can Customize Prebuilt Detection Rules: Milestone 3](https://github.com/elastic/kibana/issues/174168) - [Tests for prebuilt rule upgrade workflow](https://github.com/elastic/kibana/issues/202078) -### User stories - -**Prebuilt rule upgrade workflow:** - -- User can upgrade a single prebuilt rule to its latest version from the Rule Upgrade table with previewing incoming updates from Elastic and user customizations in the Rule Upgrade flyout. -- Specifically, when upgrading a rule, in the Rule Upgrade flyout: - - User can preview updates from Elastic, for each rule field that has an update from Elastic. - - User can preview their customizations, for each rule field that was customized. - - User can compare their customizations with updates from Elastic and see if there are any conflicts between them, per each rule field. - - User can manually resolve conflicts between their customizations and updates from Elastic, per each rule field. - - User can edit the final field values before submitting the update. -- User can upgrade a rule if its type has been changed by Elastic in the latest version, but can only accept the incoming changes; any user customizations will be lost. - ### Terminology - [Common terminology](./prebuilt_rules_common_info.md#common-terminology). - **CTA to upgrade the prebuilt rule**: the button to upgrade the prebuilt rule currently shown in the Rule Upgrade flyout. +## Requirements + ### Assumptions +Assumptions about test environments and scenarios outlined in this test plan. + - [Common assumptions](./prebuilt_rules_common_info.md#common-assumptions). - A prebuilt rule is shown in the Rule Upgrade table when there's a newer version of this rule in the currently installed package with prebuilt rules. -### Non-functional requirements +### Technical requirements + +Non-functional requirements for the functionality outlined in this test plan. + +- [Common technical requirements](./prebuilt_rules_common_info.md#common-technical-requirements). + +### Product requirements + +Functional requirements for the functionality outlined in this test plan. + +- [Common product requirements](./prebuilt_rules_common_info.md#common-product-requirements). + +User stories: + +- User can upgrade a single prebuilt rule to its latest version from the Rule Upgrade table with previewing incoming updates from Elastic and user customizations in the Rule Upgrade flyout. See below user stories that describe this workflow in more detail. +- User can upgrade single prebuilt rules one-by-one via the Rule Upgrade flyout. + +User stories for upgrading a rule via the Rule Upgrade flyout: + +- User can preview what updates they would receive from Elastic in the latest rule version, per each rule field that has an update from Elastic. +- User can preview their customizations, and which of them will be retained in the rule or overwritten by updates from Elastic, per each rule field that was customized. +- User can compare their customizations with updates from Elastic and see if there are any conflicts between them, per each rule field. +- User can manually resolve conflicts between their customizations and updates from Elastic, per each rule field. +- User can edit the final field values before submitting the update. + +User stories for edge cases: -- [Common non-functional requirements](./prebuilt_rules_common_info.md#common-non-functional-requirements). +- User can upgrade a rule if its type has been changed by Elastic in the latest version, but can only accept the incoming changes. User customizations, if the rule has any, will be lost on upgrade. We force the user to preview such changes, meaning that users can only do this via the Rule Upgrade flyout, quick upgrade right from the table is not supported. -### Functional requirements +What should be inside the Rule Upgrade flyout: -- User should be able to upgrade prebuilt rules one-by-one with the ability to preview: - - what updates they would receive from Elastic in the latest rule version; - - what user customizations they would retain in the rule; - - are there any conflicts between the updates from Elastic and the user customizations; - - what they should pay attention to in case there are any conflicts. -- A preview should be shown in the Rule Upgrade flyout. - The Rule Upgrade flyout should contain a few tabs: - The "Updates" tab. - The "Overview" tab. - The "Investigation guide" tab. - On the "Updates" tab: - - We should show the updates from Elastic and the user-customized fields. - - We should show only those fields that are [customizable](./shared_assets/customizable_rule_fields.md). - - We shouldn't show technical fields and those that are [not customizable](./shared_assets/non_customizable_rule_fields.md). -- User should be able to upgrade a prebuilt rule that has some updates to [non-customizable fields](./shared_assets/non_customizable_rule_fields.md) in the latest version. -- Any other fields that are not involved in the rule upgrade workflow, such as `enabled` or `exceptions`, should stay unchanged after rule upgrade. + - We show the updates from Elastic and the user-customized fields. + - We show only those fields that are [customizable](./prebuilt_rules_common_info.md#customizable-rule-fields). + - We don't show technical fields and those that are [not customizable](./prebuilt_rules_common_info.md#non-customizable-rule-fields). They are handled by the upgrade workflow automatically under the hood. + - We don't show some non-technical fields (rule parameters) as they are also handled by the upgrade workflow automatically under the hood. For example, the `enabled` or `exception_list` fields stay unchanged after upgrade (we retain their values from the current rule version). ## Scenarios diff --git a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md index db1e1f2b8b66f..b5b3505ef9fab 100644 --- a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md +++ b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md @@ -2,6 +2,9 @@ **Status**: `in progress`, matches [Milestone 3](https://github.com/elastic/kibana/issues/174168). +> [!TIP] +> If you're new to prebuilt rules, get started [here](./prebuilt_rules.md) and check an overview of the features of prebuilt rules in [this section](./prebuilt_rules_common_info.md#features). + ## Summary This is a test plan for the workflows of: @@ -20,11 +23,11 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one - [Useful information](#useful-information) - [Tickets](#tickets) - - [User stories](#user-stories) - [Terminology](#terminology) +- [Requirements](#requirements) - [Assumptions](#assumptions) - - [Non-functional requirements](#non-functional-requirements) - - [Functional requirements](#functional-requirements) + - [Technical requirements](#technical-requirements) + - [Product requirements](#product-requirements) - [Scenarios](#scenarios) - [Rule upgrade notifications on the Rule Management page](#rule-upgrade-notifications-on-the-rule-management-page) - [**Scenario: User is NOT notified when all installed prebuilt rules are up to date**](#scenario-user-is-not-notified-when-all-installed-prebuilt-rules-are-up-to-date) @@ -86,84 +89,58 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one - [Users can Customize Prebuilt Detection Rules: Milestone 3](https://github.com/elastic/kibana/issues/174168) - [Tests for prebuilt rule upgrade workflow](https://github.com/elastic/kibana/issues/202078) -### User stories - -**Prebuilt rule upgrade workflow:** - -- User can upgrade a single prebuilt rule to its latest version without previewing the incoming updates. -- User can bulk upgrade multiple prebuilt rules to their latest versions without previewing the incoming updates. - ### Terminology - [Common terminology](./prebuilt_rules_common_info.md#common-terminology). - **CTA to install prebuilt rules**: a link button with a counter on the Rule Management page. - **CTA to upgrade prebuilt rules**: a tab with a counter on the Rule Management page. +## Requirements + ### Assumptions +Assumptions about test environments and scenarios outlined in this test plan. + - [Common assumptions](./prebuilt_rules_common_info.md#common-assumptions). -### Non-functional requirements - -- [Common non-functional requirements](./prebuilt_rules_common_info.md#common-non-functional-requirements). - -### Functional requirements - -User should be able to upgrade prebuilt rules with and without previewing what updates they would apply (rule properties of target rule versions). - -User should be able to upgrade prebuilt rules with and without customizations. Where the following fields support customizations: - -| Rule type | Field name in UI | Diffable rule field | -| ---------------- | ------------------------- | ------------------------- | -| All rule types | Rule name | `name` | -| All rule types | Rule description | `description` | -| All rule types | Tags | `tags` | -| All rule types | Default severity | `severity` | -| All rule types | Severity Override | `severity_mapping` | -| All rule types | Default risk score | `risk_score` | -| All rule types | Risk score override | `risk_score_mapping` | -| All rule types | Reference URLs | `references` | -| All rule types | False positive examples | `false_positives` | -| All rule types | MITRE ATT&CK™ threats | `threat` | -| All rule types | Setup guide | `setup` | -| All rule types | Investigation guide | `note` | -| All rule types | Related integrations | `related_integrations` | -| All rule types | Required fields | `required_fields` | -| All rule types | Rule schedule | `rule_schedule` | -| All rule types | Max alerts per run | `max_signals` | -| All rule types | Rule name override | `rule_name_override` | -| All rule types | Timestamp override | `timestamp_override` | -| All rule types | Timeline template | `timeline_template` | -| All rule types | Building block `*` | `building_block` | -| All rule types | Investigation fields | `investigation_fields` | -| All rule types | Data source `**` | `data_source` | -| All rule types | Suppress alerts | `alert_suppression` | -| Custom Query | Custom query | `kql_query` | -| Saved Query | Custom query | `kql_query` | -| EQL | EQL query | `eql_query` | -| ESQL | ESQL query | `esql_query` | -| Threat Match | Custom query | `kql_query` | -| Threat Match | Indicator index patterns | `threat_index` | -| Threat Match | Indicator index query | `threat_query` | -| Threat Match | Indicator mapping | `threat_mapping` | -| Threat Match | Indicator prefix override | `threat_indicator_path` | -| Threshold | Custom query | `kql_query` | -| Threshold | Threshold config | `threshold` | -| Machine Learning | Machine Learning job | `machine_learning_job_id` | -| Machine Learning | Anomaly score threshold | `anomaly_threshold` | -| New Terms | Custom query | `kql_query` | -| New Terms | Fields | `new_terms_fields` | -| New Terms | History Window Size | `history_window_start` | - -- `*` Building block field is used to mark alerts as building block alerts. -- `**` Data Source represents index patterns or a data view. Machine Learning rules don't have data_source field. - -User should be able to upgrade prebuilt rules with updates in the following non-customizable fields: - -| Field name | Diffable rule field | -| --------------------- | ------------------- | -| Rule type | `type` | -| Rule version | `version` | +### Technical requirements + +Non-functional requirements for the functionality outlined in this test plan. + +- [Common technical requirements](./prebuilt_rules_common_info.md#common-technical-requirements). + +### Product requirements + +Functional requirements for the functionality outlined in this test plan. + +- [Common product requirements](./prebuilt_rules_common_info.md#common-product-requirements). + +User stories for upgrading single prebuilt rules one-by-one: + +- User can upgrade a single prebuilt rule to its latest version without previewing the incoming updates: + - if the rule doesn't have any conflicts with its latest version. +- User can't upgrade a single prebuilt rule to its latest version without previewing the incoming updates: + - if the rule has any solvable conflicts with its latest version; + - if the rule has any non-solvable conflicts with its latest version; + - if the rule's type has been changed in the latest version by Elastic (this is considered a non-solvable conflict); + - in these situations user is required to upgrade the rule with preview. + +User stories for upgrading multiple prebuilt rules in bulk: + +- User can bulk upgrade multiple prebuilt rules to their latest versions without previewing the incoming updates, but only those: + - that don't have any conflicts with their latest versions; + - that have some solvable conflicts but don't have any non-solvable conflicts with their latest versions. +- User can't bulk upgrade multiple prebuilt rules to their latest versions without previewing the incoming updates: + - if these rules have any non-solvable conflicts with their latest versions; + - if these rules don't have any non-solvable conflicts, but have some solvable conflicts and the user doesn't confirm their intention to upgrade such rules; + - if these rules' types have been changed in their latest versions by Elastic (this is considered a non-solvable conflict); + - in these situations user is required to upgrade each rule one-by-one with preview. +- User can bulk upgrade prebuilt rules with solvable conflicts only if the user confirms their intention to upgrade such rules. +- User can "bulk upgrade" a single prebuilt rule via the bulk actions. In this case, the "user stories for upgrading multiple prebuilt rules in bulk" apply instead of the "user stories for upgrading single prebuilt rules one-by-one". + +User stories, misc: + +- In general, user can upgrade a prebuilt rule without preview regardless of the fact if the rule is customized or not. The ability to do so depends on the fact if this customization conflicts with the latest version or not, and if yes, is this conflict solvable or non-solvable. ## Scenarios diff --git a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rules.md b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rules.md index b66e991dd5ef5..566ea451b95fd 100644 --- a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rules.md +++ b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rules.md @@ -11,7 +11,7 @@ See [common information](./prebuilt_rules_common_info.md) about prebuilt rules, ## Test plans -Understand how we distribute prebuilt rules via a Fleet package and how prebuilt rules become available for installation and upgrade in the system. +Understand how we distribute prebuilt rules via a Fleet package and how prebuilt rules become available for installation and upgrade in the system: - [Test plan: prebuilt rules package](./prebuilt_rules_package.md) diff --git a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rules_common_info.md b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rules_common_info.md index cca246064fc47..f4e7a3ee2e3e4 100644 --- a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rules_common_info.md +++ b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rules_common_info.md @@ -1,5 +1,10 @@ # Common information about prebuilt rules +**Status**: `in progress`, matches [Milestone 3](https://github.com/elastic/kibana/issues/174168). + +> [!TIP] +> If you're new to prebuilt rules, get started [here](./prebuilt_rules.md) and check an overview of the features of prebuilt rules in [this section](#features). + ## Table of contents This is a test plan for the workflow of installing and updating the Fleet package with prebuilt rules. @@ -18,9 +21,10 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one - [Useful information](#useful-information) - [Tickets](#tickets) - [Terminology](#terminology) +- [Requirements](#requirements) - [Assumptions](#assumptions) - - [Non-functional requirements](#non-functional-requirements) - - [Functional requirements](#functional-requirements) + - [Technical requirements](#technical-requirements) + - [Product requirements](#product-requirements) - [Scenarios](#scenarios) - [Package installation](#package-installation) - [**Scenario: Package is installed via Fleet**](#scenario-package-is-installed-via-fleet) @@ -45,22 +49,34 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one - [Common terminology](./prebuilt_rules_common_info.md#common-terminology), see the terminology related to the package with prebuilt rules. +## Requirements + ### Assumptions +Assumptions about test environments and scenarios outlined in this test plan. + - [Common assumptions](./prebuilt_rules_common_info.md#common-assumptions). -### Non-functional requirements +### Technical requirements + +Non-functional requirements for the functionality outlined in this test plan. + +- [Common technical requirements](./prebuilt_rules_common_info.md#common-technical-requirements). + +### Product requirements + +Functional requirements for the functionality outlined in this test plan. -- [Common non-functional requirements](./prebuilt_rules_common_info.md#common-non-functional-requirements). +- [Common product requirements](./prebuilt_rules_common_info.md#common-product-requirements). -### Functional requirements +User stories: -- Package should be installable and updatable: +- Package can be installed and updated: - on any license in self-hosted and ECH environments; - on any tier in Serverless Security environments; - regardless of user privileges, as long as the user has access to Security Solution. -- User should be able to install prebuilt rules with and without previewing what exactly they would install (rule properties). -- User should be able to upgrade prebuilt rules with and without previewing what updates they would apply (rule properties of target rule versions). +- User can install prebuilt rules with and without previewing what exactly they would install (rule properties). +- User can upgrade prebuilt rules with and without previewing what updates they would apply (rule properties of target rule versions). ## Scenarios diff --git a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/shared_assets/customizable_rule_fields.md b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/shared_assets/customizable_rule_fields.md deleted file mode 100644 index f9b63b6de8165..0000000000000 --- a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/shared_assets/customizable_rule_fields.md +++ /dev/null @@ -1,48 +0,0 @@ -# Customizable rule fields - -These are fields in the detection rule schema that are able to be customized on a prebuilt rule. - -| Rule type | Field name in UI | Diffable rule field | -| ---------------- | ------------------------- | ------------------------- | -| All rule types | Rule name | `name` | -| All rule types | Rule description | `description` | -| All rule types | Tags | `tags` | -| All rule types | Default severity | `severity` | -| All rule types | Severity Override | `severity_mapping` | -| All rule types | Default risk score | `risk_score` | -| All rule types | Risk score override | `risk_score_mapping` | -| All rule types | Reference URLs | `references` | -| All rule types | False positive examples | `false_positives` | -| All rule types | MITRE ATT&CK™ threats | `threat` | -| All rule types | Setup guide | `setup` | -| All rule types | Investigation guide | `note` | -| All rule types | Related integrations | `related_integrations` | -| All rule types | Required fields | `required_fields` | -| All rule types | Rule schedule | `rule_schedule` | -| All rule types | Max alerts per run | `max_signals` | -| All rule types | Rule name override | `rule_name_override` | -| All rule types | Timestamp override | `timestamp_override` | -| All rule types | Timeline template | `timeline_template` | -| All rule types | Building block `*` | `building_block` | -| All rule types | Investigation fields | `investigation_fields` | -| All rule types | Data source `**` | `data_source` | -| All rule types | Suppress alerts | `alert_suppression` | -| Custom Query | Custom query | `kql_query` | -| Saved Query | Custom query | `kql_query` | -| EQL | EQL query | `eql_query` | -| ESQL | ESQL query | `esql_query` | -| Threat Match | Custom query | `kql_query` | -| Threat Match | Indicator index patterns | `threat_index` | -| Threat Match | Indicator index query | `threat_query` | -| Threat Match | Indicator mapping | `threat_mapping` | -| Threat Match | Indicator prefix override | `threat_indicator_path` | -| Threshold | Custom query | `kql_query` | -| Threshold | Threshold config | `threshold` | -| Machine Learning | Machine Learning job | `machine_learning_job_id` | -| Machine Learning | Anomaly score threshold | `anomaly_threshold` | -| New Terms | Custom query | `kql_query` | -| New Terms | Fields | `new_terms_fields` | -| New Terms | History Window Size | `history_window_start` | - -- `*` Building block field is used to mark alerts as building block alerts. -- `**` Data Source represents index patterns or a data view. Machine Learning rules don't have data_source field. diff --git a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/shared_assets/non_customizable_rule_fields.md b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/shared_assets/non_customizable_rule_fields.md deleted file mode 100644 index b408f1fab5698..0000000000000 --- a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/shared_assets/non_customizable_rule_fields.md +++ /dev/null @@ -1,10 +0,0 @@ -# Non-customizable rule fields - -These are fields in the detection rule schema that cannot be customized for a prebuilt rule. - -| Field name | Diffable rule field | -| ------------ | ------------------- | -| Rule type | `type` | -| Rule version | `version` | -| Rule author | `author` | -| Rule license | `license` | diff --git a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/test_plan_template.md b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/test_plan_template.md index 6f9a62ff4b612..c5a3cc9de2946 100644 --- a/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/test_plan_template.md +++ b/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/test_plan_template.md @@ -1,10 +1,33 @@ -# Awesome Feature +# Test plan: my awesome feature - + +**Status**: `in progress`. + +## Summary + + This is a test plan for ... - -Status: `in progress`. +## Table of contents + + + +- [Useful information](#useful-information) + - [Tickets](#tickets) + - [Terminology](#terminology) +- [Requirements](#requirements) + - [Assumptions](#assumptions) + - [Technical requirements](#technical-requirements) + - [Product requirements](#product-requirements) +- [Scenarios](#scenarios) + - [Section 1](#section-1) + - [**Scenario: Title of scenario 1.1**](#scenario-title-of-scenario-11) + - [**Scenario: Title of scenario 1.2**](#scenario-title-of-scenario-12) + - [Section 2](#section-2) + - [**Scenario: Title of scenario 2.1**](#scenario-title-of-scenario-21) ## Useful information @@ -12,7 +35,7 @@ Status: `in progress`. -- [Awesome Feature](https://github.com/elastic/security-team/issues/9999) epic +- [Awesome Feature's epic](https://github.com/elastic/security-team/issues/9999) - [Add tests for the new awesome feature](https://github.com/elastic/kibana/issues/999999) - [Document the new awesome feature](https://github.com/elastic/security-docs/issues/9999) @@ -24,23 +47,26 @@ Status: `in progress`. --> - **Term 1**: explanation. - - **Term 2**: explanation. +## Requirements + ### Assumptions - +Assumptions about test environments and scenarios outlined in this test plan. + - Assumption 1. - Assumption 2. -### Non-functional requirements +### Technical requirements - testing --> +Non-functional requirements for the functionality outlined in this test plan. + - Requirement 1. - Requirement 2. +### Product requirements + + + +Functional requirements for the functionality outlined in this test plan. + +User stories: + +- User can do X. +- User can do Y. + ## Scenarios ### Section 1 -#### **Scenario: Awesome feature works** +#### **Scenario: Title of scenario 1.1** **Automation**: X e2e tests + Y integration tests + unit tests. @@ -86,11 +130,21 @@ When ... Then ... ``` - + + +#### **Scenario: Title of scenario 1.2** + +**Automation**: X e2e tests + Y integration tests + unit tests. + +```Gherkin +Given ... +When ... +Then ... +``` ### Section 2 -#### **Scenario: ?** +#### **Scenario: Title of scenario 2.1** **Automation**: X e2e tests + Y integration tests + unit tests.