[Security Solution] Migrate legacy API docs owned by the Rule Management team

[banderror]

Epics: #200310, https://github.com/elastic/security-docs-internal/issues/57 (internal)
Related to: https://github.com/elastic/security-docs-internal/issues/60 (internal)

Summary

Historically, Security Solution APIs have been documented on the now-legacy docs site under Elastic Security APIs. These are pages that were written manually using the asciidoc syntax here. For example, source files for the Detections API docs can be found here.

Last year, as part of https://github.com/elastic/security-team/issues/8058 and https://github.com/elastic/security-team/issues/9400, we built a system that auto-generates API docs from the OpenAPI schemas that we have in the code. The new API docs pages are hosted on the bump.sh service and are available at:

• https://www.elastic.co/docs/api/doc/kibana/
• https://www.elastic.co/docs/api/doc/serverless/

While we have defined OpenAPI specs for all our public API endpoints, and added some descriptions in some cases, some of the content hasn't been migrated from the legacy API docs website to OpenAPI specs.

We should migrate the missing content by March 17th (internal). After this date the legacy API docs will be retired and become unavailable.

Details

Additional context from the Docs team: https://github.com/elastic/security-docs-internal/issues/57 (internal).

Our API docs on the legacy website:

• Detections API
• Create rule
• Get rule
• Update rule
• Delete rule
• Bulk rule actions
• Export rules
• Import rules
• Find rules
• Tags endpoint
• Prebuilt rules

Our API docs on the new website:

• Kibana API -> Security detections
• Kibana Serverless API -> Security detections

Work done by Yara for migrating legacy API docs owned by the Detection Engine team: https://github.com/elastic/security-docs-internal/issues/60 (internal).

Todo

Migrate pages content

Some of the legacy content is located on pages and doesn't apply to any particular API endpoint, or applies to multiple endpoints. We should try to find a place for it in the new docs.

• Detections API
  • Clarify authentication caveats. The old page mentions that it's possible to basic auth via username:pass, whereas new pages only mention API key auth: "If the API key used for authorization has different privileges than the key that created or most recently updated a rule, the rule behavior might change.". I believe this caveat also applies to basic auth. We should double-check that before documenting.
  • Migrate the role requirements, or even better link to a docs page that documents them.

Migrate endpoints content

Some of the legacy content, such as descriptions of API requests and responses and examples of API calls, applies to a single endpoint or a small group of endpoints. We should migrate this content to our OpenAPI specs.

We're also missing a lot of content in the legacy docs. We should:

• Add missing summaries and descriptions for API endpoints, their request and response schemas, as well as any types that those request/response schemas depend on ("components" in the OpenAPI terminology).
• Extend and clarify those descriptions and examples that are too short or unclear.
• (Advanced) Add more examples of API usage, those that are missing in the legacy API docs. Ideally, all the API use cases should be covered by at least one example.

Rules CRUD:

• POST /api/detection_engine/rules: old, new, new svl
  • Migrate text content from the old page. There, we explain what types of rules can be created, and what properties can be specified for them.
  • Document all request and response parameters. Migrate descriptions from the old page. Add missing summaries and descriptions.
  • Migrate examples from the old page.
  • (Advanced) Ideally, we should have at least one request example per each rule type, with only required request parameters. Plus, maybe some examples for setting optional parameters. We can discuss if/when/how we should do that.
  • (Advanced) Document the caveat about the privileges of the user who creates a rule via this endpoint.
• GET /api/detection_engine/rules: old, new, new svl
  • Migrate example from the old page.
  • Add example for fetching by rule_id.
  • Explain the difference between id and rule_id.
• PUT /api/detection_engine/rules: old, new, new svl
  • Document all request and response parameters. Migrate descriptions from the old page. Add missing summaries and descriptions.
  • Migrate examples from the old page.
  • (Advanced) Ideally, we should have at least one request example per each rule type, with only required request parameters.
  • (Advanced) Document the caveat about the privileges of the user who updates a rule via this endpoint.
• PATCH /api/detection_engine/rules: old, new, new svl
  • Same as PUT /api/detection_engine/rules
• DELETE /api/detection_engine/rules: old, new, new svl
  • Migrate example from the old page.
  • Add example for fetching by rule_id.
  • Explain the difference between id and rule_id.

Rules bulk CRUD. These endpoints have been disabled and removed from Kibana 9.0. We should understand how to reflect this in the docs: remove them from Kibana 9.0 API docs, and maybe update the pre-9.0 API docs to mention that they are removed in 9.0.

• [BREAKING CHANGE] Removing deprecated rules bulk CRUD API endpoints security-docs#5981
• POST /api/detection_engine/rules/_bulk_create: old, new
  • Migrate examples from the old page.
• PUT /api/detection_engine/rules/_bulk_update: old, new
  • Migrate examples from the old page.
• PATCH /api/detection_engine/rules/_bulk_update: old, new
  • Migrate examples from the old page.
• POST /api/detection_engine/rules/_bulk_delete: old, new
  • Migrate examples from the old page.
• DELETE /api/detection_engine/rules/_bulk_delete: old, new
  • Migrate examples from the old page.

Rules bulk actions:

• POST /api/detection_engine/rules/_bulk_action: old, new, new svl
  • Document all request and response parameters. Migrate descriptions from the old page. Add missing summaries and descriptions.
  • Migrate examples from the old page.
  • (Advanced) Add missing examples. Ideally, we should find a way to document each use case separately: enabling, disabling, duplicating, bulk editing index patterns, etc.
  • (Advanced) Document the caveat about the privileges of the user who updates rules via this endpoint.

Rules export and import:

• POST /api/detection_engine/rules/_export: old, new, new svl
  • Remove the "You cannot export prebuilt rules." info from the 8.18+ API docs. Users will be able to export prebuilt rules starting from 8.18.0.
  • Document all request and response parameters. Migrate descriptions from the old page. Add missing summaries and descriptions.
  • Migrate examples from the old page.
• POST /api/detection_engine/rules/_import: old, new, new svl
  • Migrate text content from the old page.
  • Document all request and response parameters. Migrate descriptions from the old page. Add missing summaries and descriptions.
  • Migrate examples from the old page.

Rules searching and filtering:

• GET /api/detection_engine/rules/_find: old, new, new svl
  • Document all request and response parameters. Migrate descriptions from the old page. Add missing summaries and descriptions.
  • Migrate examples from the old page.
• GET /api/detection_engine/tags: old, new, new svl
  • Migrate examples from the old page.

Prebuilt rules:

• GET /api/detection_engine/rules/prepackaged/_status: old, new
  • Elaborate on what this endpoint is used for.
  • Migrate examples from the old page.
• PUT /api/detection_engine/rules/prepackaged: old, new
  • Elaborate on what this endpoint does, when called.
  • Migrate examples from the old page.

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[elasticmachine]

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[banderror]

@jkelas I added elastic/security-docs#5981 to this ticket's todo list. The work on the removed bulk CRUD endpoints was already in the todo list, I just found that ticket that we created earlier to address that.