network_traffic: copy non-ECS fields to ECS-compliant locations for detection rules #8221
Conversation
efd6
added
enhancement
| # Detection Rules compatibility | ||
| - set: | ||
| tag: set_compatibility_request_authorization | ||
| field: network_traffic.http.request.headers.authorization |
There was a problem hiding this comment.
I considered using http.request.Headers.authorization instead, but I think a good path (depending on approval of this location) would be to reflect what is currently http.* (without the ECS enrichments) into network_traffic.http.* (and equivalently all <datastream>.* into network_traffic.<datastream>.* in the other datastreams).
There was a problem hiding this comment.
I think that network_traffic.<datastream>.* is a good safe pattern to follow.
There has been a long standing lack of guidance from ECS on the headers field (elastic/ecs#232). With us moving in the direction of open-telemetry I expect us to adapt to their naming conventions at some point. So I figured I should mention that the OTEL definition is singular http.{request,response}.header.<key> (with key lowercased). https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
🤖 GitHub commentsExpand to view the GitHub comments
To re-run your PR in the CI, just comment with:
|
🌐 Coverage report
|
|
Pinging @elastic/security-external-integrations (Team:Security-External Integrations) |
| @@ -1,4 +1,15 @@ | |||
| # newer versions go on top | |||
| - version: "1.26.0" | |||
| changes: | |||
| - description: Copy `type` field to `network.protocol` in ICPM datastream. | |||
There was a problem hiding this comment.
| - description: Copy `type` field to `network.protocol` in ICPM datastream. | |
| - description: Copy `type` field to `network.protocol` in ICMP datastream. |
Currently, detection rules depend on three non-ECS-compliant fields: - type (all except flows) - http.request.headers.authorization (http) - http.response.headers.content-type (http) With the exception of flows and icmp, type is already set in network.protocol as well. So make the following copies: - type field into network.protocol for icmp - http.request.headers.authorization to network_traffic.http.request.headers.authorization for http - http.response.headers.content-type to http.response.mime_type for http
There was a problem hiding this comment.
Changes look good from our end 👍 . We will make adjustments accordingly in elastic/detection-rules#3194
Thank you
| # Detection Rules compatibility | ||
| - set: | ||
| tag: set_compatibility_request_authorization | ||
| field: network_traffic.http.request.headers.authorization |
There was a problem hiding this comment.
I think that network_traffic.<datastream>.* is a good safe pattern to follow.
There has been a long standing lack of guidance from ECS on the headers field (elastic/ecs#232). With us moving in the direction of open-telemetry I expect us to adapt to their naming conventions at some point. So I figured I should mention that the OTEL definition is singular http.{request,response}.header.<key> (with key lowercased). https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/
|
Package network_traffic - 1.27.0 containing this change is available at https://epr.elastic.co/search?package=network_traffic |
2 similar comments
|
Package network_traffic - 1.27.0 containing this change is available at https://epr.elastic.co/search?package=network_traffic |
|
Package network_traffic - 1.27.0 containing this change is available at https://epr.elastic.co/search?package=network_traffic |
Proposed commit message
Currently, detection rules depend on three non-ECS-compliant fields:
type(all except flows)http.request.headers.authorization(http)http.response.headers.content-type(http)With the exception of flows and icmp,
typeis already set innetwork.protocolas well. So make the following copies:typefield intonetwork.protocolfor icmphttp.request.headers.authorizationtonetwork_traffic.http.request.headers.authorizationfor httphttp.response.headers.content-typetohttp.response.mime_typefor httpChecklist
changelog.ymlfile.Author's Checklist
network_traffic.*with http datastream. The issue here is that the field undernetwork_trafficwould behttpwhich is an ECS name, and I have seen some issues where use of ECS names is claimed to require ECS conformance even when in a namespaced location. I have doubts about the merits of this position in the long term given that is essentially prevents the use of perfectly cromulent labels.How to test this PR locally
Related issues
Screenshots