Skip to content

Commit

Permalink
[8.x] Skip scheduling actions for the alerts without scheduledActions (
Browse files Browse the repository at this point in the history
…elastic#195948) (elastic#196458)

# Backport

This will backport the following commits from `main` to `8.x`:
- [Skip scheduling actions for the alerts without scheduledActions
(elastic#195948)](elastic#195948)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Ersin
Erdal","email":"92688503+ersin-erdal@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-10-15T23:40:19Z","message":"Skip
scheduling actions for the alerts without scheduledActions
(elastic#195948)\n\nResolves: elastic#190258\r\n\r\nAs a result of elastic#190258, we have
found out that the odd behaviour happens\r\nwhen an existing alert is
pushed above the max alerts limit by a
new\r\nalert.\r\n\r\nScenario:\r\n\r\n1. The rule type detects 4 alerts
(`alert-1`, `alert-2`, `alert-3`,\r\n`alert-4`),\r\nBut reports only the
first 3 as the max alerts limit is 3.\r\n\r\n2. `alert-2` becomes
recovered, therefore the rule type reports 3 active\r\n(`alert-1`,
`alert-3`, `alert-4`), 1 recovered (`alert-2`) alert.\r\n\r\n3. Alerts
`alert-1`, `alert-3`, `alert-4` are saved in the task state.\r\n\r\n4.
`alert-2` becomes active again (the others are still active)\r\n\r\n5.
Rule type reports 3 active alerts (`alert-1`, `alert-2`,
`alert-3`)\r\n\r\n6. As a result, the action scheduler tries to schedule
actions for\r\n`alert-1`, `alert-3`, `alert-4` as they are the existing
alerts.\r\nBut, since the rule type didn't report the `alert-4` it has
no scheduled\r\nactions, therefore the action scheduler assumes it is
recovered and\r\ntries to schedule a recovery action.\r\n\r\nThis PR
changes the actionScheduler to handle active and recovered\r\nalerts
separately.\r\nWith this change, no action would be scheduled for an
alert from\r\nprevious run (exists in the task state) and isn't reported
by the\r\nruleTypeExecutor due to max-alerts-limit but it would be kept
in the\r\ntask
state.","sha":"dd25bf8807c3ff3982d455f070b6e6c65233662d","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","Team:ResponseOps","v9.0.0","backport:prev-minor"],"title":"Skip
scheduling actions for the alerts without
scheduledActions","number":195948,"url":"https://github.com/elastic/kibana/pull/195948","mergeCommit":{"message":"Skip
scheduling actions for the alerts without scheduledActions
(elastic#195948)\n\nResolves: elastic#190258\r\n\r\nAs a result of elastic#190258, we have
found out that the odd behaviour happens\r\nwhen an existing alert is
pushed above the max alerts limit by a
new\r\nalert.\r\n\r\nScenario:\r\n\r\n1. The rule type detects 4 alerts
(`alert-1`, `alert-2`, `alert-3`,\r\n`alert-4`),\r\nBut reports only the
first 3 as the max alerts limit is 3.\r\n\r\n2. `alert-2` becomes
recovered, therefore the rule type reports 3 active\r\n(`alert-1`,
`alert-3`, `alert-4`), 1 recovered (`alert-2`) alert.\r\n\r\n3. Alerts
`alert-1`, `alert-3`, `alert-4` are saved in the task state.\r\n\r\n4.
`alert-2` becomes active again (the others are still active)\r\n\r\n5.
Rule type reports 3 active alerts (`alert-1`, `alert-2`,
`alert-3`)\r\n\r\n6. As a result, the action scheduler tries to schedule
actions for\r\n`alert-1`, `alert-3`, `alert-4` as they are the existing
alerts.\r\nBut, since the rule type didn't report the `alert-4` it has
no scheduled\r\nactions, therefore the action scheduler assumes it is
recovered and\r\ntries to schedule a recovery action.\r\n\r\nThis PR
changes the actionScheduler to handle active and recovered\r\nalerts
separately.\r\nWith this change, no action would be scheduled for an
alert from\r\nprevious run (exists in the task state) and isn't reported
by the\r\nruleTypeExecutor due to max-alerts-limit but it would be kept
in the\r\ntask
state.","sha":"dd25bf8807c3ff3982d455f070b6e6c65233662d"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/195948","number":195948,"mergeCommit":{"message":"Skip
scheduling actions for the alerts without scheduledActions
(elastic#195948)\n\nResolves: elastic#190258\r\n\r\nAs a result of elastic#190258, we have
found out that the odd behaviour happens\r\nwhen an existing alert is
pushed above the max alerts limit by a
new\r\nalert.\r\n\r\nScenario:\r\n\r\n1. The rule type detects 4 alerts
(`alert-1`, `alert-2`, `alert-3`,\r\n`alert-4`),\r\nBut reports only the
first 3 as the max alerts limit is 3.\r\n\r\n2. `alert-2` becomes
recovered, therefore the rule type reports 3 active\r\n(`alert-1`,
`alert-3`, `alert-4`), 1 recovered (`alert-2`) alert.\r\n\r\n3. Alerts
`alert-1`, `alert-3`, `alert-4` are saved in the task state.\r\n\r\n4.
`alert-2` becomes active again (the others are still active)\r\n\r\n5.
Rule type reports 3 active alerts (`alert-1`, `alert-2`,
`alert-3`)\r\n\r\n6. As a result, the action scheduler tries to schedule
actions for\r\n`alert-1`, `alert-3`, `alert-4` as they are the existing
alerts.\r\nBut, since the rule type didn't report the `alert-4` it has
no scheduled\r\nactions, therefore the action scheduler assumes it is
recovered and\r\ntries to schedule a recovery action.\r\n\r\nThis PR
changes the actionScheduler to handle active and recovered\r\nalerts
separately.\r\nWith this change, no action would be scheduled for an
alert from\r\nprevious run (exists in the task state) and isn't reported
by the\r\nruleTypeExecutor due to max-alerts-limit but it would be kept
in the\r\ntask
state.","sha":"dd25bf8807c3ff3982d455f070b6e6c65233662d"}}]}]
BACKPORT-->

Co-authored-by: Ersin Erdal <92688503+ersin-erdal@users.noreply.github.com>
  • Loading branch information
kibanamachine and ersin-erdal authored Oct 16, 2024
1 parent d5d39e5 commit b9820cc
Show file tree
Hide file tree
Showing 12 changed files with 542 additions and 196 deletions.
7 changes: 5 additions & 2 deletions x-pack/plugins/alerting/server/alerts_client/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,11 @@ export interface IAlertsClient<
processAlerts(opts: ProcessAlertsOpts): void;
logAlerts(opts: LogAlertsOpts): void;
getProcessedAlerts(
type: 'new' | 'active' | 'activeCurrent' | 'recovered' | 'recoveredCurrent'
): Record<string, LegacyAlert<State, Context, ActionGroupIds | RecoveryActionGroupId>>;
type: 'new' | 'active' | 'activeCurrent'
): Record<string, LegacyAlert<State, Context, ActionGroupIds>> | {};
getProcessedAlerts(
type: 'recovered' | 'recoveredCurrent'
): Record<string, LegacyAlert<State, Context, RecoveryActionGroupId>> | {};
persistAlerts(): Promise<{ alertIds: string[]; maintenanceWindowIds: string[] } | null>;
isTrackedAlert(id: string): boolean;
getSummarizedAlerts?(params: GetSummarizedAlertsParams): Promise<SummarizedAlerts>;
Expand Down
Loading

0 comments on commit b9820cc

Please sign in to comment.