From 8be4ed7d2e7e89ed34566f324fb6d65e1055c27b Mon Sep 17 00:00:00 2001 From: Smriti <152067238+smriti0321@users.noreply.github.com> Date: Tue, 17 Sep 2024 15:55:29 +0200 Subject: [PATCH 1/2] [RFC] Stage 0: Introducing new fields in ECS vulnerability field set (#2331) * RFC to add new fields in ECS vulnerability field set RFC to add new fields in ECS vulnerability field set * Moving to separate file * set title and add stage 0 PR # * clean up fields table markdown * Moving to (rfcs/text) and renaming file to next number in series. * Resolving the comments from @trisch-me * Update rfcs/text/0045-additional-vulnerability-fields.md Co-authored-by: Alexandra Konrad * Update rfcs/text/0045-additional-vulnerability-fields.md Co-authored-by: Alexandra Konrad * Making changed to the date format as per comments from @trisch-me * Resolving @trisch-me comments * Resolving latest comments * Update rfcs/text/0045-additional-vulnerability-fields.md Co-authored-by: Alexandra Konrad --------- Co-authored-by: Eric Beahan Co-authored-by: Alexandra Konrad --- .../0045-additional-vulnerability-fields.md | 138 ++++++++++++++++++ 1 file changed, 138 insertions(+) create mode 100644 rfcs/text/0045-additional-vulnerability-fields.md diff --git a/rfcs/text/0045-additional-vulnerability-fields.md b/rfcs/text/0045-additional-vulnerability-fields.md new file mode 100644 index 0000000000..63b82105e9 --- /dev/null +++ b/rfcs/text/0045-additional-vulnerability-fields.md @@ -0,0 +1,138 @@ +# 0000: Additional Vulnerability Fields + + +- Stage: **0 (strawperson)** +- Date: **TBD** + + + + +This RFC proposes to expand the vulnerability fieldset to include more fields, the proposal takes into consideration various customer feedbacks provided to Security integration team, inputs from Infosec team managing vulnerabilities across Elastic and other companies. This will benefit our customers and internal product teams to provide more effective vulnerability management experience to end user. to come up with the list of fields, extensive research was done across various Vulnerability management products and schemas like OSV. It is a continuation of one of the previous RFC on similar topic- https://github.com/elastic/ecs/issues/1685 + + + + + +## Fields +The `vulnerabilities` fields being proposed are as follows: + +| Field | Type | Description / Use Case | +| ----- | ---- | ---------------------- | +| `vulnerability.vendor.id` | keyword | A vulnerability doesn't have necessary a CVE associated with it. It makes sense to seperate vulnerability ID (like CVEs) to the vendor/detection IDs. | +| `vulnerability.title` | keyword | Title/Name/Short Description for vulnerability, to be used in flyout and dashboards. | +| `vulnerability.mitigation` | text | Explains user how to fix or mitigate the problem, could be usefd to store resolution from the scanner vendor or document mitigation in place | +| `vulnerability.published` | date | The “published” field indicates the date when information about a specific vulnerability was publicly disclosed or made available.It represents the moment when details about the vulnerability were shared with the security community, vendors, and the public.This field helps security professionals track the timeline of vulnerability awareness, in ISO 8601 format - YYYY-MM-DD | +| `vulnerability.patch.*` | object | - | +| `vulnerability.patch.exists` | boolean | The “patch” field refers to whether a security fix or update (commonly known as a patch) is available to address the identified vulnerability. It indicates whether the software vendor or developer has released a solution to mitigate the vulnerability. | +| `vulnerability.patch.name` | text | Name of the patch | +| `vulnerability.patch.code` | keyword | Associated patch code for example ESA-2020-13 | +| `vulnerability.evidence` | text | A demonstration of the validity of a vulnerability claim, e.g. app.any.run replaying the exploitation of the vulnerability. | +| `vulnerability.status` | keyword | The status field helps security teams track vulnerabilities, prioritize actions, and communicate their progress effectively. Examples- open/ignored/patched/mitigated/false_positive/risk_accepted/reopened..| +| `vulnerability.tags` | keyword | This is different from cloud provider assigned resource tags, this is specifically for vulnerability. Vulnerability tags serve as a way to add custom metadata to vulnerabilities, enhancing their context and aiding in search and automation. | +| `vulnerability.first_found` | date | First time a vulnerability was found on the asset, in ISO 8601 format: 2016-05-23T08:05:34.853Z | +| `vulnerability.last_found` | date | Last time a vulnerability was found on the asset, in ISO 8601 format: 2016-05-23T08:05:34.853Z | +| `vulnerability.last_scanned` | date | Last time a scan was performed on the asset. It's important as some companies are scanning on a quarterly basis. If last_found and last_scanned are close, it means it's still an active vulnerability, in ISO 8601 format: 2016-05-23T08:05:34.853Z| +| `vulnerability.age` | long | Numbers of days since the vulnerability is active. It should be dynamically calculated (runtime fields, ingest, ...). It could either be then difference between the last_found date and the published date (preferred). It could also be the difference between the first_found and last_found dates. | +| `vulnerability.uid` | keyword | It's extremely important to be able to deduplicate different scans. It's often that we have different scanners showing the same vulnerability on the same asset. | +| `vulnerability.type` | keyword | To conclude if the vulnerability is confirmed or potential. | +| `vulnerability.exploitability.*` | object | - | +| `vulnerability.exploitability.exploited` | boolean | To indicate if the vulnerability has been exploited or not. | +| `vulnerability.exploitability.reference` | keyword | Exploitability databse for example CSA-KEV. | +| `vulnerability.exploitability.confidence` | keyword | Confidence measure the credibility of existence and exploitability. | +| `vulnerability.exploitability.first_seen` | date | First time of exploitability, in ISO 8601 format: 2016-05-23T08:05:34.853Z | +| `vulnerability.exploitability.last_seen` | date | Last time of exploitability, in ISO 8601 format: 2016-05-23T08:05:34.853Z | +| `vulnerability.affected.*` | object | The affected field is a JSON array containing objects that describes the affected package versions, meaning those that contain the vulnerability. | +| `vulnerability.affected.package` | array | Package field is a JSON object identifying the affected code library or command provided by the package. | +| `vulnerability.affected.severity` | array | This field applies to a specific package, in cases where affected packages have differing severities for the same vulnerability. | +| `vulnerability.affected.versions` | array | Affected version in whatever version syntax is used by the given package ecosystem. | + + + + + + +## Usage + + + +## Source data + + + + + + + +## Scope of impact + + + +## Concerns + + + + + + + +## People + +The following are the people that consulted on the contents of this RFC. + +* @smriti0321 | author +* @tinnytintin10 | Product Manager Cloud Security +* @oren-zohar | Engineering Manager Cloud Security +* @orouz | Engineer +* @clement-fouque | Information Security Analyst + + + +## References + + +previous RFC - https://github.com/elastic/ecs/issues/1685 +https://ossf.github.io/osv-schema/#affected-fields + +### RFC Pull Requests + + + +* Stage 0: https://github.com/elastic/ecs/pull/2331 + + From 220ecee12b6829515d2935e331d08c0f1579d012 Mon Sep 17 00:00:00 2001 From: Michael Wolf Date: Mon, 23 Sep 2024 11:19:19 -0700 Subject: [PATCH 2/2] Fix type in code signature (#2382) Change the type of code_signature.flags to keyword, which is what it should be. Also add a unit test that will verify all types are valid. --- docs/fields/field-details.asciidoc | 2 +- experimental/generated/beats/fields.ecs.yml | 18 ++++++---- experimental/generated/csv/fields.csv | 12 +++---- experimental/generated/ecs/ecs_flat.yml | 18 ++++++---- experimental/generated/ecs/ecs_nested.yml | 21 +++++++---- .../composable/component/dll.json | 3 +- .../composable/component/file.json | 3 +- .../composable/component/process.json | 6 ++-- .../composable/component/threat.json | 6 ++-- .../elasticsearch/legacy/template.json | 18 ++++++---- generated/beats/fields.ecs.yml | 18 ++++++---- generated/csv/fields.csv | 12 +++---- generated/ecs/ecs_flat.yml | 18 ++++++---- generated/ecs/ecs_nested.yml | 21 +++++++---- .../composable/component/dll.json | 3 +- .../composable/component/file.json | 3 +- .../composable/component/process.json | 6 ++-- .../composable/component/threat.json | 6 ++-- generated/elasticsearch/legacy/template.json | 18 ++++++---- rfcs/text/0044/code_signature.yml | 4 +-- schemas/code_signature.yml | 2 +- scripts/tests/test_ecs_spec.py | 35 +++++++++++++++++++ 22 files changed, 175 insertions(+), 78 deletions(-) diff --git a/docs/fields/field-details.asciidoc b/docs/fields/field-details.asciidoc index f2259fb87e..23ae02e99a 100644 --- a/docs/fields/field-details.asciidoc +++ b/docs/fields/field-details.asciidoc @@ -873,7 +873,7 @@ a| beta:[ This field is beta and subject to change. ] The flags used to sign the process. -type: string +type: keyword diff --git a/experimental/generated/beats/fields.ecs.yml b/experimental/generated/beats/fields.ecs.yml index 625206235f..ee0ecb5e3b 100644 --- a/experimental/generated/beats/fields.ecs.yml +++ b/experimental/generated/beats/fields.ecs.yml @@ -1273,7 +1273,8 @@ default_field: false - name: code_signature.flags level: extended - type: string + type: keyword + ignore_above: 1024 description: The flags used to sign the process. example: 570522385 default_field: false @@ -2439,7 +2440,8 @@ default_field: false - name: code_signature.flags level: extended - type: string + type: keyword + ignore_above: 1024 description: The flags used to sign the process. example: 570522385 default_field: false @@ -4793,7 +4795,8 @@ default_field: false - name: code_signature.flags level: extended - type: string + type: keyword + ignore_above: 1024 description: The flags used to sign the process. example: 570522385 default_field: false @@ -6117,7 +6120,8 @@ default_field: false - name: parent.code_signature.flags level: extended - type: string + type: keyword + ignore_above: 1024 description: The flags used to sign the process. example: 570522385 default_field: false @@ -9177,7 +9181,8 @@ default_field: false - name: enrichments.indicator.file.code_signature.flags level: extended - type: string + type: keyword + ignore_above: 1024 description: The flags used to sign the process. example: 570522385 default_field: false @@ -10798,7 +10803,8 @@ default_field: false - name: indicator.file.code_signature.flags level: extended - type: string + type: keyword + ignore_above: 1024 description: The flags used to sign the process. example: 570522385 default_field: false diff --git a/experimental/generated/csv/fields.csv b/experimental/generated/csv/fields.csv index 85f24dce13..be5ee33461 100644 --- a/experimental/generated/csv/fields.csv +++ b/experimental/generated/csv/fields.csv @@ -149,7 +149,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 8.12.0-dev+exp,true,device,device.serial_number,keyword,core,,DJGAQS4CW5,Serial Number of the device 8.12.0-dev+exp,true,dll,dll.code_signature.digest_algorithm,keyword,extended,,sha256,Hashing algorithm used to sign the process. 8.12.0-dev+exp,true,dll,dll.code_signature.exists,boolean,core,,true,Boolean to capture if a signature is present. -8.12.0-dev+exp,true,dll,dll.code_signature.flags,string,extended,,570522385,Code signing flags of the process +8.12.0-dev+exp,true,dll,dll.code_signature.flags,keyword,extended,,570522385,Code signing flags of the process 8.12.0-dev+exp,true,dll,dll.code_signature.signing_id,keyword,extended,,com.apple.xpc.proxy,The identifier used to sign the process. 8.12.0-dev+exp,true,dll,dll.code_signature.status,keyword,extended,,ERROR_UNTRUSTED_ROOT,Additional information about the certificate status. 8.12.0-dev+exp,true,dll,dll.code_signature.subject_name,keyword,core,,Microsoft Corporation,Subject name of the code signer @@ -280,7 +280,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 8.12.0-dev+exp,true,file,file.attributes,keyword,extended,array,"[""readonly"", ""system""]",Array of file attributes. 8.12.0-dev+exp,true,file,file.code_signature.digest_algorithm,keyword,extended,,sha256,Hashing algorithm used to sign the process. 8.12.0-dev+exp,true,file,file.code_signature.exists,boolean,core,,true,Boolean to capture if a signature is present. -8.12.0-dev+exp,true,file,file.code_signature.flags,string,extended,,570522385,Code signing flags of the process +8.12.0-dev+exp,true,file,file.code_signature.flags,keyword,extended,,570522385,Code signing flags of the process 8.12.0-dev+exp,true,file,file.code_signature.signing_id,keyword,extended,,com.apple.xpc.proxy,The identifier used to sign the process. 8.12.0-dev+exp,true,file,file.code_signature.status,keyword,extended,,ERROR_UNTRUSTED_ROOT,Additional information about the certificate status. 8.12.0-dev+exp,true,file,file.code_signature.subject_name,keyword,core,,Microsoft Corporation,Subject name of the code signer @@ -593,7 +593,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 8.12.0-dev+exp,true,process,process.args_count,long,extended,,4,Length of the process.args array. 8.12.0-dev+exp,true,process,process.code_signature.digest_algorithm,keyword,extended,,sha256,Hashing algorithm used to sign the process. 8.12.0-dev+exp,true,process,process.code_signature.exists,boolean,core,,true,Boolean to capture if a signature is present. -8.12.0-dev+exp,true,process,process.code_signature.flags,string,extended,,570522385,Code signing flags of the process +8.12.0-dev+exp,true,process,process.code_signature.flags,keyword,extended,,570522385,Code signing flags of the process 8.12.0-dev+exp,true,process,process.code_signature.signing_id,keyword,extended,,com.apple.xpc.proxy,The identifier used to sign the process. 8.12.0-dev+exp,true,process,process.code_signature.status,keyword,extended,,ERROR_UNTRUSTED_ROOT,Additional information about the certificate status. 8.12.0-dev+exp,true,process,process.code_signature.subject_name,keyword,core,,Microsoft Corporation,Subject name of the code signer @@ -775,7 +775,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 8.12.0-dev+exp,true,process,process.parent.args_count,long,extended,,4,Length of the process.args array. 8.12.0-dev+exp,true,process,process.parent.code_signature.digest_algorithm,keyword,extended,,sha256,Hashing algorithm used to sign the process. 8.12.0-dev+exp,true,process,process.parent.code_signature.exists,boolean,core,,true,Boolean to capture if a signature is present. -8.12.0-dev+exp,true,process,process.parent.code_signature.flags,string,extended,,570522385,Code signing flags of the process +8.12.0-dev+exp,true,process,process.parent.code_signature.flags,keyword,extended,,570522385,Code signing flags of the process 8.12.0-dev+exp,true,process,process.parent.code_signature.signing_id,keyword,extended,,com.apple.xpc.proxy,The identifier used to sign the process. 8.12.0-dev+exp,true,process,process.parent.code_signature.status,keyword,extended,,ERROR_UNTRUSTED_ROOT,Additional information about the certificate status. 8.12.0-dev+exp,true,process,process.parent.code_signature.subject_name,keyword,core,,Microsoft Corporation,Subject name of the code signer @@ -1162,7 +1162,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 8.12.0-dev+exp,true,threat,threat.enrichments.indicator.file.attributes,keyword,extended,array,"[""readonly"", ""system""]",Array of file attributes. 8.12.0-dev+exp,true,threat,threat.enrichments.indicator.file.code_signature.digest_algorithm,keyword,extended,,sha256,Hashing algorithm used to sign the process. 8.12.0-dev+exp,true,threat,threat.enrichments.indicator.file.code_signature.exists,boolean,core,,true,Boolean to capture if a signature is present. -8.12.0-dev+exp,true,threat,threat.enrichments.indicator.file.code_signature.flags,string,extended,,570522385,Code signing flags of the process +8.12.0-dev+exp,true,threat,threat.enrichments.indicator.file.code_signature.flags,keyword,extended,,570522385,Code signing flags of the process 8.12.0-dev+exp,true,threat,threat.enrichments.indicator.file.code_signature.signing_id,keyword,extended,,com.apple.xpc.proxy,The identifier used to sign the process. 8.12.0-dev+exp,true,threat,threat.enrichments.indicator.file.code_signature.status,keyword,extended,,ERROR_UNTRUSTED_ROOT,Additional information about the certificate status. 8.12.0-dev+exp,true,threat,threat.enrichments.indicator.file.code_signature.subject_name,keyword,core,,Microsoft Corporation,Subject name of the code signer @@ -1381,7 +1381,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 8.12.0-dev+exp,true,threat,threat.indicator.file.attributes,keyword,extended,array,"[""readonly"", ""system""]",Array of file attributes. 8.12.0-dev+exp,true,threat,threat.indicator.file.code_signature.digest_algorithm,keyword,extended,,sha256,Hashing algorithm used to sign the process. 8.12.0-dev+exp,true,threat,threat.indicator.file.code_signature.exists,boolean,core,,true,Boolean to capture if a signature is present. -8.12.0-dev+exp,true,threat,threat.indicator.file.code_signature.flags,string,extended,,570522385,Code signing flags of the process +8.12.0-dev+exp,true,threat,threat.indicator.file.code_signature.flags,keyword,extended,,570522385,Code signing flags of the process 8.12.0-dev+exp,true,threat,threat.indicator.file.code_signature.signing_id,keyword,extended,,com.apple.xpc.proxy,The identifier used to sign the process. 8.12.0-dev+exp,true,threat,threat.indicator.file.code_signature.status,keyword,extended,,ERROR_UNTRUSTED_ROOT,Additional information about the certificate status. 8.12.0-dev+exp,true,threat,threat.indicator.file.code_signature.subject_name,keyword,core,,Microsoft Corporation,Subject name of the code signer diff --git a/experimental/generated/ecs/ecs_flat.yml b/experimental/generated/ecs/ecs_flat.yml index 6e09b7f52f..e529df5f93 100644 --- a/experimental/generated/ecs/ecs_flat.yml +++ b/experimental/generated/ecs/ecs_flat.yml @@ -1806,12 +1806,13 @@ dll.code_signature.flags: description: The flags used to sign the process. example: 570522385 flat_name: dll.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword dll.code_signature.signing_id: dashed_name: dll-code-signature-signing-id description: 'The identifier used to sign the process. @@ -3957,12 +3958,13 @@ file.code_signature.flags: description: The flags used to sign the process. example: 570522385 flat_name: file.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword file.code_signature.signing_id: dashed_name: file-code-signature-signing-id description: 'The identifier used to sign the process. @@ -7787,12 +7789,13 @@ process.code_signature.flags: description: The flags used to sign the process. example: 570522385 flat_name: process.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword process.code_signature.signing_id: dashed_name: process-code-signature-signing-id description: 'The identifier used to sign the process. @@ -9956,12 +9959,13 @@ process.parent.code_signature.flags: description: The flags used to sign the process. example: 570522385 flat_name: process.parent.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword process.parent.code_signature.signing_id: dashed_name: process-parent-code-signature-signing-id description: 'The identifier used to sign the process. @@ -14782,12 +14786,13 @@ threat.enrichments.indicator.file.code_signature.flags: description: The flags used to sign the process. example: 570522385 flat_name: threat.enrichments.indicator.file.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword threat.enrichments.indicator.file.code_signature.signing_id: dashed_name: threat-enrichments-indicator-file-code-signature-signing-id description: 'The identifier used to sign the process. @@ -17518,12 +17523,13 @@ threat.indicator.file.code_signature.flags: description: The flags used to sign the process. example: 570522385 flat_name: threat.indicator.file.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword threat.indicator.file.code_signature.signing_id: dashed_name: threat-indicator-file-code-signature-signing-id description: 'The identifier used to sign the process. diff --git a/experimental/generated/ecs/ecs_nested.yml b/experimental/generated/ecs/ecs_nested.yml index 1f7f9648b7..f4a2844515 100644 --- a/experimental/generated/ecs/ecs_nested.yml +++ b/experimental/generated/ecs/ecs_nested.yml @@ -1326,11 +1326,12 @@ code_signature: description: The flags used to sign the process. example: 570522385 flat_name: code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] short: Code signing flags of the process - type: string + type: keyword code_signature.signing_id: dashed_name: code-signature-signing-id description: 'The identifier used to sign the process. @@ -2290,12 +2291,13 @@ dll: description: The flags used to sign the process. example: 570522385 flat_name: dll.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword dll.code_signature.signing_id: dashed_name: dll-code-signature-signing-id description: 'The identifier used to sign the process. @@ -5001,12 +5003,13 @@ file: description: The flags used to sign the process. example: 570522385 flat_name: file.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword file.code_signature.signing_id: dashed_name: file-code-signature-signing-id description: 'The identifier used to sign the process. @@ -10020,12 +10023,13 @@ process: description: The flags used to sign the process. example: 570522385 flat_name: process.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword process.code_signature.signing_id: dashed_name: process-code-signature-signing-id description: 'The identifier used to sign the process. @@ -12194,12 +12198,13 @@ process: description: The flags used to sign the process. example: 570522385 flat_name: process.parent.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword process.parent.code_signature.signing_id: dashed_name: process-parent-code-signature-signing-id description: 'The identifier used to sign the process. @@ -17482,12 +17487,13 @@ threat: description: The flags used to sign the process. example: 570522385 flat_name: threat.enrichments.indicator.file.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword threat.enrichments.indicator.file.code_signature.signing_id: dashed_name: threat-enrichments-indicator-file-code-signature-signing-id description: 'The identifier used to sign the process. @@ -20224,12 +20230,13 @@ threat: description: The flags used to sign the process. example: 570522385 flat_name: threat.indicator.file.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword threat.indicator.file.code_signature.signing_id: dashed_name: threat-indicator-file-code-signature-signing-id description: 'The identifier used to sign the process. diff --git a/experimental/generated/elasticsearch/composable/component/dll.json b/experimental/generated/elasticsearch/composable/component/dll.json index 55e2246263..e59687764d 100644 --- a/experimental/generated/elasticsearch/composable/component/dll.json +++ b/experimental/generated/elasticsearch/composable/component/dll.json @@ -18,7 +18,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, diff --git a/experimental/generated/elasticsearch/composable/component/file.json b/experimental/generated/elasticsearch/composable/component/file.json index adb9d1d8ec..175a0cbab7 100644 --- a/experimental/generated/elasticsearch/composable/component/file.json +++ b/experimental/generated/elasticsearch/composable/component/file.json @@ -25,7 +25,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, diff --git a/experimental/generated/elasticsearch/composable/component/process.json b/experimental/generated/elasticsearch/composable/component/process.json index de0be5f249..76b8983a3b 100644 --- a/experimental/generated/elasticsearch/composable/component/process.json +++ b/experimental/generated/elasticsearch/composable/component/process.json @@ -25,7 +25,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, @@ -832,7 +833,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, diff --git a/experimental/generated/elasticsearch/composable/component/threat.json b/experimental/generated/elasticsearch/composable/component/threat.json index cdcbbd7ae6..32056d1507 100644 --- a/experimental/generated/elasticsearch/composable/component/threat.json +++ b/experimental/generated/elasticsearch/composable/component/threat.json @@ -67,7 +67,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, @@ -995,7 +996,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, diff --git a/experimental/generated/elasticsearch/legacy/template.json b/experimental/generated/elasticsearch/legacy/template.json index 6b9172fe34..bc7f446065 100644 --- a/experimental/generated/elasticsearch/legacy/template.json +++ b/experimental/generated/elasticsearch/legacy/template.json @@ -801,7 +801,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, @@ -1376,7 +1377,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, @@ -2768,7 +2770,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, @@ -3575,7 +3578,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, @@ -5320,7 +5324,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, @@ -6248,7 +6253,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, diff --git a/generated/beats/fields.ecs.yml b/generated/beats/fields.ecs.yml index 77f9536d95..3883c5b045 100644 --- a/generated/beats/fields.ecs.yml +++ b/generated/beats/fields.ecs.yml @@ -1223,7 +1223,8 @@ default_field: false - name: code_signature.flags level: extended - type: string + type: keyword + ignore_above: 1024 description: The flags used to sign the process. example: 570522385 default_field: false @@ -2389,7 +2390,8 @@ default_field: false - name: code_signature.flags level: extended - type: string + type: keyword + ignore_above: 1024 description: The flags used to sign the process. example: 570522385 default_field: false @@ -4743,7 +4745,8 @@ default_field: false - name: code_signature.flags level: extended - type: string + type: keyword + ignore_above: 1024 description: The flags used to sign the process. example: 570522385 default_field: false @@ -6067,7 +6070,8 @@ default_field: false - name: parent.code_signature.flags level: extended - type: string + type: keyword + ignore_above: 1024 description: The flags used to sign the process. example: 570522385 default_field: false @@ -9127,7 +9131,8 @@ default_field: false - name: enrichments.indicator.file.code_signature.flags level: extended - type: string + type: keyword + ignore_above: 1024 description: The flags used to sign the process. example: 570522385 default_field: false @@ -10748,7 +10753,8 @@ default_field: false - name: indicator.file.code_signature.flags level: extended - type: string + type: keyword + ignore_above: 1024 description: The flags used to sign the process. example: 570522385 default_field: false diff --git a/generated/csv/fields.csv b/generated/csv/fields.csv index a7210ad73b..8af3fac81a 100644 --- a/generated/csv/fields.csv +++ b/generated/csv/fields.csv @@ -142,7 +142,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 8.12.0-dev,true,device,device.serial_number,keyword,core,,DJGAQS4CW5,Serial Number of the device 8.12.0-dev,true,dll,dll.code_signature.digest_algorithm,keyword,extended,,sha256,Hashing algorithm used to sign the process. 8.12.0-dev,true,dll,dll.code_signature.exists,boolean,core,,true,Boolean to capture if a signature is present. -8.12.0-dev,true,dll,dll.code_signature.flags,string,extended,,570522385,Code signing flags of the process +8.12.0-dev,true,dll,dll.code_signature.flags,keyword,extended,,570522385,Code signing flags of the process 8.12.0-dev,true,dll,dll.code_signature.signing_id,keyword,extended,,com.apple.xpc.proxy,The identifier used to sign the process. 8.12.0-dev,true,dll,dll.code_signature.status,keyword,extended,,ERROR_UNTRUSTED_ROOT,Additional information about the certificate status. 8.12.0-dev,true,dll,dll.code_signature.subject_name,keyword,core,,Microsoft Corporation,Subject name of the code signer @@ -273,7 +273,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 8.12.0-dev,true,file,file.attributes,keyword,extended,array,"[""readonly"", ""system""]",Array of file attributes. 8.12.0-dev,true,file,file.code_signature.digest_algorithm,keyword,extended,,sha256,Hashing algorithm used to sign the process. 8.12.0-dev,true,file,file.code_signature.exists,boolean,core,,true,Boolean to capture if a signature is present. -8.12.0-dev,true,file,file.code_signature.flags,string,extended,,570522385,Code signing flags of the process +8.12.0-dev,true,file,file.code_signature.flags,keyword,extended,,570522385,Code signing flags of the process 8.12.0-dev,true,file,file.code_signature.signing_id,keyword,extended,,com.apple.xpc.proxy,The identifier used to sign the process. 8.12.0-dev,true,file,file.code_signature.status,keyword,extended,,ERROR_UNTRUSTED_ROOT,Additional information about the certificate status. 8.12.0-dev,true,file,file.code_signature.subject_name,keyword,core,,Microsoft Corporation,Subject name of the code signer @@ -586,7 +586,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 8.12.0-dev,true,process,process.args_count,long,extended,,4,Length of the process.args array. 8.12.0-dev,true,process,process.code_signature.digest_algorithm,keyword,extended,,sha256,Hashing algorithm used to sign the process. 8.12.0-dev,true,process,process.code_signature.exists,boolean,core,,true,Boolean to capture if a signature is present. -8.12.0-dev,true,process,process.code_signature.flags,string,extended,,570522385,Code signing flags of the process +8.12.0-dev,true,process,process.code_signature.flags,keyword,extended,,570522385,Code signing flags of the process 8.12.0-dev,true,process,process.code_signature.signing_id,keyword,extended,,com.apple.xpc.proxy,The identifier used to sign the process. 8.12.0-dev,true,process,process.code_signature.status,keyword,extended,,ERROR_UNTRUSTED_ROOT,Additional information about the certificate status. 8.12.0-dev,true,process,process.code_signature.subject_name,keyword,core,,Microsoft Corporation,Subject name of the code signer @@ -768,7 +768,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 8.12.0-dev,true,process,process.parent.args_count,long,extended,,4,Length of the process.args array. 8.12.0-dev,true,process,process.parent.code_signature.digest_algorithm,keyword,extended,,sha256,Hashing algorithm used to sign the process. 8.12.0-dev,true,process,process.parent.code_signature.exists,boolean,core,,true,Boolean to capture if a signature is present. -8.12.0-dev,true,process,process.parent.code_signature.flags,string,extended,,570522385,Code signing flags of the process +8.12.0-dev,true,process,process.parent.code_signature.flags,keyword,extended,,570522385,Code signing flags of the process 8.12.0-dev,true,process,process.parent.code_signature.signing_id,keyword,extended,,com.apple.xpc.proxy,The identifier used to sign the process. 8.12.0-dev,true,process,process.parent.code_signature.status,keyword,extended,,ERROR_UNTRUSTED_ROOT,Additional information about the certificate status. 8.12.0-dev,true,process,process.parent.code_signature.subject_name,keyword,core,,Microsoft Corporation,Subject name of the code signer @@ -1155,7 +1155,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 8.12.0-dev,true,threat,threat.enrichments.indicator.file.attributes,keyword,extended,array,"[""readonly"", ""system""]",Array of file attributes. 8.12.0-dev,true,threat,threat.enrichments.indicator.file.code_signature.digest_algorithm,keyword,extended,,sha256,Hashing algorithm used to sign the process. 8.12.0-dev,true,threat,threat.enrichments.indicator.file.code_signature.exists,boolean,core,,true,Boolean to capture if a signature is present. -8.12.0-dev,true,threat,threat.enrichments.indicator.file.code_signature.flags,string,extended,,570522385,Code signing flags of the process +8.12.0-dev,true,threat,threat.enrichments.indicator.file.code_signature.flags,keyword,extended,,570522385,Code signing flags of the process 8.12.0-dev,true,threat,threat.enrichments.indicator.file.code_signature.signing_id,keyword,extended,,com.apple.xpc.proxy,The identifier used to sign the process. 8.12.0-dev,true,threat,threat.enrichments.indicator.file.code_signature.status,keyword,extended,,ERROR_UNTRUSTED_ROOT,Additional information about the certificate status. 8.12.0-dev,true,threat,threat.enrichments.indicator.file.code_signature.subject_name,keyword,core,,Microsoft Corporation,Subject name of the code signer @@ -1374,7 +1374,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description 8.12.0-dev,true,threat,threat.indicator.file.attributes,keyword,extended,array,"[""readonly"", ""system""]",Array of file attributes. 8.12.0-dev,true,threat,threat.indicator.file.code_signature.digest_algorithm,keyword,extended,,sha256,Hashing algorithm used to sign the process. 8.12.0-dev,true,threat,threat.indicator.file.code_signature.exists,boolean,core,,true,Boolean to capture if a signature is present. -8.12.0-dev,true,threat,threat.indicator.file.code_signature.flags,string,extended,,570522385,Code signing flags of the process +8.12.0-dev,true,threat,threat.indicator.file.code_signature.flags,keyword,extended,,570522385,Code signing flags of the process 8.12.0-dev,true,threat,threat.indicator.file.code_signature.signing_id,keyword,extended,,com.apple.xpc.proxy,The identifier used to sign the process. 8.12.0-dev,true,threat,threat.indicator.file.code_signature.status,keyword,extended,,ERROR_UNTRUSTED_ROOT,Additional information about the certificate status. 8.12.0-dev,true,threat,threat.indicator.file.code_signature.subject_name,keyword,core,,Microsoft Corporation,Subject name of the code signer diff --git a/generated/ecs/ecs_flat.yml b/generated/ecs/ecs_flat.yml index 7e504589db..bad8611fa7 100644 --- a/generated/ecs/ecs_flat.yml +++ b/generated/ecs/ecs_flat.yml @@ -1737,12 +1737,13 @@ dll.code_signature.flags: description: The flags used to sign the process. example: 570522385 flat_name: dll.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword dll.code_signature.signing_id: dashed_name: dll-code-signature-signing-id description: 'The identifier used to sign the process. @@ -3888,12 +3889,13 @@ file.code_signature.flags: description: The flags used to sign the process. example: 570522385 flat_name: file.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword file.code_signature.signing_id: dashed_name: file-code-signature-signing-id description: 'The identifier used to sign the process. @@ -7718,12 +7720,13 @@ process.code_signature.flags: description: The flags used to sign the process. example: 570522385 flat_name: process.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword process.code_signature.signing_id: dashed_name: process-code-signature-signing-id description: 'The identifier used to sign the process. @@ -9887,12 +9890,13 @@ process.parent.code_signature.flags: description: The flags used to sign the process. example: 570522385 flat_name: process.parent.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword process.parent.code_signature.signing_id: dashed_name: process-parent-code-signature-signing-id description: 'The identifier used to sign the process. @@ -14713,12 +14717,13 @@ threat.enrichments.indicator.file.code_signature.flags: description: The flags used to sign the process. example: 570522385 flat_name: threat.enrichments.indicator.file.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword threat.enrichments.indicator.file.code_signature.signing_id: dashed_name: threat-enrichments-indicator-file-code-signature-signing-id description: 'The identifier used to sign the process. @@ -17449,12 +17454,13 @@ threat.indicator.file.code_signature.flags: description: The flags used to sign the process. example: 570522385 flat_name: threat.indicator.file.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword threat.indicator.file.code_signature.signing_id: dashed_name: threat-indicator-file-code-signature-signing-id description: 'The identifier used to sign the process. diff --git a/generated/ecs/ecs_nested.yml b/generated/ecs/ecs_nested.yml index b08955b69b..a401fa7b0a 100644 --- a/generated/ecs/ecs_nested.yml +++ b/generated/ecs/ecs_nested.yml @@ -1246,11 +1246,12 @@ code_signature: description: The flags used to sign the process. example: 570522385 flat_name: code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] short: Code signing flags of the process - type: string + type: keyword code_signature.signing_id: dashed_name: code-signature-signing-id description: 'The identifier used to sign the process. @@ -2210,12 +2211,13 @@ dll: description: The flags used to sign the process. example: 570522385 flat_name: dll.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword dll.code_signature.signing_id: dashed_name: dll-code-signature-signing-id description: 'The identifier used to sign the process. @@ -4921,12 +4923,13 @@ file: description: The flags used to sign the process. example: 570522385 flat_name: file.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword file.code_signature.signing_id: dashed_name: file-code-signature-signing-id description: 'The identifier used to sign the process. @@ -9940,12 +9943,13 @@ process: description: The flags used to sign the process. example: 570522385 flat_name: process.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword process.code_signature.signing_id: dashed_name: process-code-signature-signing-id description: 'The identifier used to sign the process. @@ -12114,12 +12118,13 @@ process: description: The flags used to sign the process. example: 570522385 flat_name: process.parent.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword process.parent.code_signature.signing_id: dashed_name: process-parent-code-signature-signing-id description: 'The identifier used to sign the process. @@ -17402,12 +17407,13 @@ threat: description: The flags used to sign the process. example: 570522385 flat_name: threat.enrichments.indicator.file.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword threat.enrichments.indicator.file.code_signature.signing_id: dashed_name: threat-enrichments-indicator-file-code-signature-signing-id description: 'The identifier used to sign the process. @@ -20144,12 +20150,13 @@ threat: description: The flags used to sign the process. example: 570522385 flat_name: threat.indicator.file.code_signature.flags + ignore_above: 1024 level: extended name: flags normalize: [] original_fieldset: code_signature short: Code signing flags of the process - type: string + type: keyword threat.indicator.file.code_signature.signing_id: dashed_name: threat-indicator-file-code-signature-signing-id description: 'The identifier used to sign the process. diff --git a/generated/elasticsearch/composable/component/dll.json b/generated/elasticsearch/composable/component/dll.json index b5f52995c5..7c76d1ed0d 100644 --- a/generated/elasticsearch/composable/component/dll.json +++ b/generated/elasticsearch/composable/component/dll.json @@ -18,7 +18,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, diff --git a/generated/elasticsearch/composable/component/file.json b/generated/elasticsearch/composable/component/file.json index cc12f10be1..c032c0a53c 100644 --- a/generated/elasticsearch/composable/component/file.json +++ b/generated/elasticsearch/composable/component/file.json @@ -25,7 +25,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, diff --git a/generated/elasticsearch/composable/component/process.json b/generated/elasticsearch/composable/component/process.json index 610ff74848..d48a4eddab 100644 --- a/generated/elasticsearch/composable/component/process.json +++ b/generated/elasticsearch/composable/component/process.json @@ -25,7 +25,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, @@ -832,7 +833,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, diff --git a/generated/elasticsearch/composable/component/threat.json b/generated/elasticsearch/composable/component/threat.json index c9030c4167..40f98ec195 100644 --- a/generated/elasticsearch/composable/component/threat.json +++ b/generated/elasticsearch/composable/component/threat.json @@ -67,7 +67,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, @@ -995,7 +996,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, diff --git a/generated/elasticsearch/legacy/template.json b/generated/elasticsearch/legacy/template.json index 6725cae44d..66b302cebd 100644 --- a/generated/elasticsearch/legacy/template.json +++ b/generated/elasticsearch/legacy/template.json @@ -759,7 +759,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, @@ -1334,7 +1335,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, @@ -2726,7 +2728,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, @@ -3533,7 +3536,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, @@ -5278,7 +5282,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, @@ -6206,7 +6211,8 @@ "type": "boolean" }, "flags": { - "type": "string" + "ignore_above": 1024, + "type": "keyword" }, "signing_id": { "ignore_above": 1024, diff --git a/rfcs/text/0044/code_signature.yml b/rfcs/text/0044/code_signature.yml index 091339048d..d16b64b6e4 100644 --- a/rfcs/text/0044/code_signature.yml +++ b/rfcs/text/0044/code_signature.yml @@ -3,8 +3,8 @@ fields: - name: flags level: extended - type: string + type: keyword short: Code signing flags of the process description: > The flags used to sign the process. - example: 570522385 \ No newline at end of file + example: 570522385 diff --git a/schemas/code_signature.yml b/schemas/code_signature.yml index 67e6a5da4a..e5808e6e3d 100644 --- a/schemas/code_signature.yml +++ b/schemas/code_signature.yml @@ -37,7 +37,7 @@ - name: flags level: extended - type: string + type: keyword short: Code signing flags of the process description: > The flags used to sign the process. diff --git a/scripts/tests/test_ecs_spec.py b/scripts/tests/test_ecs_spec.py index edb0b8420c..a55d7bbb68 100644 --- a/scripts/tests/test_ecs_spec.py +++ b/scripts/tests/test_ecs_spec.py @@ -130,6 +130,41 @@ def test_normalize_always_array(self): for (field_name, field) in self.ecs_fields.items(): self.assertIsInstance(field.get('normalize'), list, field_name) + def test_valid_type(self): + valid_types = ['binary', + 'boolean', + 'keyword', + 'constant_keyword', + 'wildcard', + 'long', + 'integer', + 'short', + 'byte', + 'double', + 'float', + 'half_float', + 'scaled_float', + 'unsigned_long', + 'date', + 'date_nanos', + 'alias', + 'object', + 'flattened', + 'nested', + 'join', + 'long_range', + 'double_range', + 'date_range', + 'ip', + 'text', + 'match_only_text', + 'geo_point', + 'geo_shape', + 'point', + 'shape'] + for (field_name, field) in self.ecs_fields.items(): + self.assertIn(field.get('type'), valid_types, field_name) + if __name__ == '__main__': unittest.main()