Skip to content

Commit

Permalink
[8.x] [Security Solution] Test plans for prebuilt rule import and exp…
Browse files Browse the repository at this point in the history 
…ort (#191116) (#199716)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Security Solution] Test plans for prebuilt rule import and export
(#191116)](#191116)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Ryland
Herrick","email":"ryalnd@gmail.com"},"sourceCommit":{"committedDate":"2024-11-11T21:01:46Z","message":"[Security
Solution] Test plans for prebuilt rule import and export (#191116)\n\n##
Summary\r\n\r\nThis PR introduces test plans for both [Prebuilt
Rule\r\nImport](#180168)
(corresponding\r\n[PR](#190198))
and [Prebuilt
Rule\r\nExport](#180167)
(corresponding\r\n[PR](#194498)).
Import is\r\nconsiderably more complicated as it is calculating new
values (for\r\n`rule_source`, `immutable`), while the export work is
mainly removing\r\nexisting restrictions (which allowed only custom
rules to be exported).\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic
Machine
<elasticmachine@users.noreply.github.com>","sha":"e4298492b5e48338396618d51168ea3e8427c103","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections
and Resp","Team: SecuritySolution","Team:Detection Rule
Management","Feature:Prebuilt Detection
Rules","backport:version","v8.17.0"],"title":"[Security Solution] Test
plans for prebuilt rule import and
export","number":191116,"url":"https://github.com/elastic/kibana/pull/191116","mergeCommit":{"message":"[Security
Solution] Test plans for prebuilt rule import and export (#191116)\n\n##
Summary\r\n\r\nThis PR introduces test plans for both [Prebuilt
Rule\r\nImport](#180168)
(corresponding\r\n[PR](#190198))
and [Prebuilt
Rule\r\nExport](#180167)
(corresponding\r\n[PR](#194498)).
Import is\r\nconsiderably more complicated as it is calculating new
values (for\r\n`rule_source`, `immutable`), while the export work is
mainly removing\r\nexisting restrictions (which allowed only custom
rules to be exported).\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic
Machine
<elasticmachine@users.noreply.github.com>","sha":"e4298492b5e48338396618d51168ea3e8427c103"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/191116","number":191116,"mergeCommit":{"message":"[Security
Solution] Test plans for prebuilt rule import and export (#191116)\n\n##
Summary\r\n\r\nThis PR introduces test plans for both [Prebuilt
Rule\r\nImport](#180168)
(corresponding\r\n[PR](#190198))
and [Prebuilt
Rule\r\nExport](#180167)
(corresponding\r\n[PR](#194498)).
Import is\r\nconsiderably more complicated as it is calculating new
values (for\r\n`rule_source`, `immutable`), while the export work is
mainly removing\r\nexisting restrictions (which allowed only custom
rules to be exported).\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic
Machine
<elasticmachine@users.noreply.github.com>","sha":"e4298492b5e48338396618d51168ea3e8427c103"}},{"branch":"8.x","label":"v8.17.0","branchLabelMappingKey":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Ryland Herrick <ryalnd@gmail.com>
  • Loading branch information
@kibanamachine @rylnd
kibanamachine and rylnd authored Nov 11, 2024
1 parent 8554fae commit 48c81df
Show file tree
Hide file tree
Showing 2 changed files with 192 additions and 0 deletions.
65 changes: 65 additions & 0 deletions ...solution/docs/testing/test_plans/detection_response/prebuilt_rules/exporting.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Prebuilt Rule Export

This is a test plan for the exporting of prebuilt rules. This feature is an aspect of `Milestone 2` of the [Rule Immutability/Customization](https://github.com/elastic/security-team/issues/1974) epic.

Status: `in progress`.

## Useful information

### Tickets

- [Rule Immutability/Customization](https://github.com/elastic/security-team/issues/1974)
- [Rule Exporting Feature](https://github.com/elastic/kibana/issues/180167#issue-2227974379)
- [Rule Export API PR](https://github.com/elastic/kibana/pull/194498)

### Terminology

- **prebuilt rule**: A rule contained in our `Prebuilt Security Detection Rules` integration in Fleet.
- **custom rule**: A rule defined by the user, which has no relation to the prebuilt rules
- **rule source, or ruleSource**: A field on the rule that defines the rule's categorization

## Scenarios

### Core Functionality

#### Scenario: Exporting prebuilt rule individually
```Gherkin
Given a space with prebuilt rules installed
When the user selects "Export rule" from the "All actions" dropdown on the rule's page
Then the rule should be exported as an NDJSON file
And it should include an "immutable" field with a value of true
And its "ruleSource" "type" should be "external"
And its "ruleSource" "isCustomized" value should depend on whether the rule was customized
```

#### Scenario: Exporting prebuilt rules in bulk
```Gherkin
Given a space with prebuilt rules installed
When the user selects prebuilt rules in the alerts table
And chooses "Export" from bulk actions
Then the selected rules should be exported as an NDJSON file
And they should include an "immutable" field with a value of true
And their "ruleSource" "type" should be "external"
And their "ruleSource" "isCustomized" should depend on whether the rule was customized
```

#### Scenario: Exporting both prebuilt and custom rules in bulk
```Gherkin
Given a space with prebuilt and custom rules installed
When the user selects prebuilt rules in the alerts table
And chooses "Export" from bulk actions
Then the selected rules should be exported as an NDJSON file
And the prebuilt rules should include an "immutable" field with a value of true
And the custom rules should include an "immutable" field with a value of false
And the prebuilt rules' "ruleSource" "type" should be "external"
And the custom rules' "ruleSource" "type" should be "internal"
```

### Error Handling

#### Scenario: Exporting beyond the export limit
```Gherkin
Given a space with prebuilt and custom rules installed
And the number of rules is greater than the export limit (defaults to 10_000)
Then the request should be rejected as a bad request
```
127 changes: 127 additions & 0 deletions ...solution/docs/testing/test_plans/detection_response/prebuilt_rules/importing.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# Prebuilt Rule Import

This is a test plan for the importing of prebuilt rules. This feature is an aspect of `Milestone 2` of the [Rule Immutability/Customization](https://github.com/elastic/security-team/issues/1974) epic.

Status: `in progress`.

## Useful information

### Tickets

- [Rule Immutability/Customization](https://github.com/elastic/security-team/issues/1974)
- [Rule Importing Feature](https://github.com/elastic/kibana/issues/180168)
- [Rule Import API PR](https://github.com/elastic/kibana/pull/190198)

### Terminology

- **prebuilt rule**: A rule contained in our `Prebuilt Security Detection Rules` integration in Fleet.
- **custom rule**: A rule defined by the user, which has no relation to the prebuilt rules
- **rule source, or ruleSource**: A field on the rule that defines the rule's categorization

## Scenarios

### Core Functionality

#### Scenario: Importing an unmodified prebuilt rule with a matching rule_id and version

```Gherkin
Given the import payload contains a prebuilt rule with a matching rule_id and version, identical to the published rule
When the user imports the rule
Then the rule should be created or updated
And the ruleSource type should be "external"
And isCustomized should be false
```

#### Scenario: Importing a customized prebuilt rule with a matching rule_id and version

```Gherkin
Given the import payload contains a prebuilt rule with a matching rule_id and version, modified from the published version
When the user imports the rule
Then the rule should be created or updated
And the ruleSource type should be "external"
And isCustomized should be true
```

#### Scenario: Importing a prebuilt rule with a matching rule_id but no matching version

```Gherkin
Given the import payload contains a prebuilt rule with a matching rule_id but no matching version
When the user imports the rule
Then the rule should be created or updated
And the ruleSource type should be "external"
And isCustomized should be true
```

#### Scenario: Importing a prebuilt rule with a non-existent rule_id

```Gherkin
Given the import payload contains a prebuilt rule with a non-existent rule_id
When the user imports the rule
Then the rule should be created
And the ruleSource type should be "internal"
```

#### Scenario: Importing a prebuilt rule without a rule_id field

```Gherkin
Given the import payload contains a prebuilt rule without a rule_id field
When the user imports the rule
Then the import should be rejected with a message "rule_id field is required"
```

#### Scenario: Importing a prebuilt rule with a matching rule_id but missing a version field

```Gherkin
Given the import payload contains a prebuilt rule without a version field
When the user imports the rule
Then the import should be rejected with a message "version field is required"
```

#### Scenario: Importing an existing custom rule missing a version field

```Gherkin
Given the import payload contains an existing custom rule without a version field
When the user imports the rule
Then the rule should be updated
And the ruleSource type should be "internal"
And the "version" field should be set to the existing rule's "version"
```

#### Scenario: Importing a new custom rule missing a version field

```Gherkin
Given the import payload contains a new custom rule without a version field
When the user imports the rule
Then the rule should be created
And the ruleSource type should be "internal"
And the "version" field should be set to 1
```

#### Scenario: Importing a rule with overwrite flag set to true

```Gherkin
Given the import payload contains a rule with an existing rule_id
And the overwrite flag is set to true
When the user imports the rule
Then the rule should be overwritten
And the ruleSource type should be calculated based on the rule_id and version
```

#### Scenario: Importing a rule with overwrite flag set to false

```Gherkin
Given the import payload contains a rule with an existing rule_id
And the overwrite flag is set to false
When the user imports the rule
Then the import should be rejected with a message "rule_id already exists"
```

#### Scenario: Importing both custom and prebuilt rules

```Gherkin
Given the import payload contains modified and unmodified, custom and prebuilt rules
When the user imports the rule
Then custom rules should be created or updated, with versions defaulted to 1
And prebuilt rules should be created or updated,
And prebuilt rules missing versions should be rejected
```

0 comments on commit 48c81df

Please sign in to comment.