Add new ML detection rules for Privileged Access Detection

[sodhikirti07]

Pull Request

Issue link(s):

• https://github.com/elastic/security-ml/issues/507

Summary - What I changed

• Added detection rules corresponding to AD jobs in the Privileged Access Detection (PAD) package. You could see the jobs config here.
  Note: Both updates are scheduled for release in version 8.18.0.

How To Test

Checklist

• Added a label for the type of pr: bug, enhancement, schema, maintenance, Rule: New, Rule: Deprecation, Rule: Tuning, Hunt: New, or Hunt: Tuning so guidelines can be generated
• Added the meta:rapid-merge label if planning to merge within 24 hours
• Secret and sensitive material has been managed correctly
• Automated testing was updated or added to match the most common scenarios
• Documentation and comments were added for features that require explanation

Contributor checklist

• Have you signed the contributor license agreement?
• Have you followed the contributor guidelines?

[tradebot-elastic]

⛔️ Tests failed:

• ❌ Spike in Group Privilege Change Events (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ Unusual Source IP for Windows Privileged Operations Detected (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ Spike in Special Logon Events (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ High Command Line Entropy Detected for Privileged Commands (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ Spike in Group Membership Events (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ Spike in User Lifecycle Management Change Events (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ Unusual Privilege Type assigned to a User (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ Unusual Host Name for Windows Privileged Operations Detected (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ Spike in Group Application Assignment Change Events (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ Spike in User Account Management Events (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ Unusual Process Detected for Privileged Commands by a User (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ Spike in Special Privilege Use Events (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ Spike in Group Management Events (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ Unusual Host Name for Okta Privileged Operations Detected (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ Unusual Spike in Concurrent Active Sessions by a User (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ Unusual Region Name for Okta Privileged Operations Detected (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ Spike in Group Lifecycle Change Events (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ Decline in host-based traffic (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ Spike in Privileged Command Execution by a User (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ Unusual Region Name for Windows Privileged Operations Detected (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ Unusual Source IP for Okta Privileged Operations Detected (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events
• ❌ Spike in host-based traffic (-)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
  • events_validation_missing: Not tested with events

[shashank-elastic]

Quick Review note

• PAD seems to be a new integration that we are adding.
• This would mean we need to pull in manifests and schemas for the same!
• When are targeting this we have a huge big PR for Prep work and a release scheduled this week once we are unblocked by the ML packages having 9.0.0.
• From the integrations part I can help you generate for PAD and we can sync on this

[shashank-elastic]

Post Syncing with @sodhikirti07
The Package is not published in the EPR - https://epr.elastic.co/search?package=pad
This is scheduled to release in Timeline of 8.18 and we will be able to pull the rule dev work and integrations only after having at-least a beta version of the package and the same has been communicated.

[terrancedejesus]

@shashank-elastic - If I remember correctly, we still need to validate ML job IDs in packages upstream. Thus this PR will not be able to merge until that the ML package is in EPR. We could manually add them to bypass this for now, but need to pull later when available

[tradebot-elastic]

⛔️ Tests failed:

[sodhikirti07]

@shashank-elastic Started a PR for Security:Host module here : #4519

[shashank-elastic]

Update

For this PR we have new integration PAD, and we wait for the Package to release add the package to our list of MACHINE_LEARNING packages here and then allow it pass through the normal dev cycle.