[O11y][AWS] Rally benchmark aws.cloudtrail
#9448
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
29 tasks
… into aws_benchmark_cloudtrail
shmsr
approved these changes
Apr 3, 2024
| - name: userIdentity_sessionContext_sessionIssuer_arn | ||
| value: "arn:aws:iam::123456789012:role/RoleToBeAssumed" | ||
| - name: userIdentity_sessionContext_sessionIssuer_accountId | ||
| value: "123456789044444412" |
There was a problem hiding this comment.
This may be correct but shouldn't this be in the range of userIdentity_accountId?
… into aws_benchmark_cloudtrail
|
/test |
… into aws_benchmark_cloudtrail
🚀 Benchmarks reportTo see the full report comment with |
|
💚 Build Succeeded
History
cc @aliabbas-elastic |
milan-elastic
added a commit
to milan-elastic/integrations
that referenced
this pull request
May 1, 2024
commit e2a688fbb1c8712ba0cad243713146867ac2f986
Author: milan-elastic <milan.Parmar@elastic.co>
Date: Wed May 1 15:43:52 2024 +0530
Squashed commit of the following:
commit a17de73aa84608f67a1baca4c094819b562e42e0
Author: milan-elastic <“milan.parmar@elastic.co”>
Date: Wed May 1 15:29:41 2024 +0530
Squashed commit of the following:
commit fccdb1f83f0048b07df6ee82fbd91ca432c799b9
Author: milan-elastic <milan.parmar@elastic.co>
Date: Wed May 1 14:58:41 2024 +0530
add global filter on dashboard level for hadoop
commit 686e49be78dc980b2f12d365580cb800fd7cf330
Merge: 024d864b4 01201a7
Author: “milan-elastic” <“milan.parmar@elastic.co”>
Date: Wed May 1 11:38:59 2024 +0530
Merge branch 'main' of github.com:milan-elastic/integrations into mongodb-atlas-database-logs
commit 01201a7
Author: Eric Forte <119343520+eric-forte-elastic@users.noreply.github.com>
Date: Tue Apr 30 10:46:55 2024 -0400
[Security Rules] Update security rules package to v8.13.5 (elastic#9762)
* [Security Rules] Update security rules package to v8.13.5
* Add changelog entry for 8.13.5
---------
Co-authored-by: protectionsmachine <72879786+protectionsmachine@users.noreply.github.com>
commit c9d1f1b
Author: Eric Forte <119343520+eric-forte-elastic@users.noreply.github.com>
Date: Tue Apr 30 09:30:30 2024 -0400
[Security Rules] Update security rules package to v8.13.5-beta.1 (elastic#9758)
* [Security Rules] Update security rules package to v8.13.5-beta.1
* Add changelog entry for 8.13.5-beta.1
---------
Co-authored-by: protectionsmachine <72879786+protectionsmachine@users.noreply.github.com>
commit a79f813
Author: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>
Date: Tue Apr 30 11:32:37 2024 +0200
[kubernetes] Remove deprecated fields, add missing status.last_terminated_reason metric (elastic#9736)
* remove deprecated fields
Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>
* Update changelog.yml
* add missing metric: last_terminated_reason; update description of the status.reason field
Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>
---------
Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>
commit b1627a3
Author: ShourieG <105607378+ShourieG@users.noreply.github.com>
Date: Tue Apr 30 13:03:29 2024 +0530
[integrations][http_endpoint] - Converted HTTP Endpoint Integration to input type (elastic#9732)
* converted http_endpoint to input package type
* updated changelog
* updated original event in sample event
commit 3a9b508
Author: Lalit Satapathy <69236064+lalit-satapathy@users.noreply.github.com>
Date: Tue Apr 30 11:49:09 2024 +0530
Remove separate codeowners for system package kibana paths. (elastic#9731)
commit c90e817
Author: Krishna Chaitanya Reddy Burri <krishnachaitanyareddy.burri@elastic.co>
Date: Tue Apr 30 11:32:17 2024 +0530
[Crowdstrike,Azure] Fix flaky tests with ECS fields (elastic#9738)
* Fix flaky pipeline tests.
* `azure.graphactivitylogs`: Add missing ECS field definitions.
* `crowdstrike.falcon`: Update `geoip` processor to `destination` instead of `source`.
commit ace8fb4
Author: Aliabbas Attarwala <124054599+aliabbas-elastic@users.noreply.github.com>
Date: Mon Apr 29 16:37:23 2024 +0530
[O11y][AWS] Rally benchmark `aws.cloudtrail` (elastic#9448)
commit d4e4aa4
Author: niraj-elastic <124254029+niraj-elastic@users.noreply.github.com>
Date: Mon Apr 29 14:45:46 2024 +0530
[Apache] Update grok pattern for accepting user-identity (elastic#9632)
* update grok pattern
* update changelog
* address review comments
* address review comments
Co-authored-by: muthu-mps <101238137+muthu-mps@users.noreply.github.com>
* address review comments
* address review comment
---------
Co-authored-by: muthu-mps <101238137+muthu-mps@users.noreply.github.com>
commit dce5699
Author: Mario Rodriguez Molins <mario.rodriguez@elastic.co>
Date: Mon Apr 29 10:33:19 2024 +0200
Enable publishing packages from integrations-publish pipeline (elastic#9712)
Enable publishing packages from integrations-publish pipeline,
and remove corresponding step from the main pipeline.
commit c7bc530
Author: Chema Martínez <chema.martinez@elastic.co>
Date: Sat Apr 27 08:57:55 2024 +0200
[zscaler_zia] Fix mapping of source.ip and source.nat.ip (elastic#9727)
* Fix mapping of source.ip and source.nat.ip
* Update changelog
* updated web datastream pipeline tests
---------
Co-authored-by: Shourie Ganguly <shourie.ganguly@elastic.co>
commit 4750ea8
Author: Mario Rodriguez Molins <mario.rodriguez@elastic.co>
Date: Fri Apr 26 13:09:53 2024 +0200
[nginx] Update nginx config to listen in ipv6 too (elastic#9720)
commit 25b0988
Author: Mario Rodriguez Molins <mario.rodriguez@elastic.co>
Date: Fri Apr 26 10:45:03 2024 +0200
[Buildkite] Update filter to use api source (elastic#9717)
commit 45327cf
Author: Mario Rodriguez Molins <mario.rodriguez@elastic.co>
Date: Fri Apr 26 10:13:22 2024 +0200
[Buildkite] Update filter condition to allow just from webhook source (elastic#9714)
commit 024d864b49f1dd333529f96e06de6dec15aac703
Author: milan-elastic <milan.parmar@elastic.co>
Date: Fri Apr 26 13:00:47 2024 +0530
add dashboard level filter for apache tomcat
commit 1cb5fad
Author: Dan Kortschak <dan.kortschak@elastic.co>
Date: Fri Apr 26 16:23:35 2024 +0930
entityanalytics_ad: new package for Active Directory user collection (elastic#9485)
commit 37c598f
Author: CarsonHrusovsky <95260807+CarsonHrusovsky@users.noreply.github.com>
Date: Thu Apr 25 18:13:26 2024 -0500
[BBOT] New integration for Black Lantern Security scanner (elastic#9651)
commit d13e474
Author: Mario Rodriguez Molins <mario.rodriguez@elastic.co>
Date: Thu Apr 25 11:55:39 2024 +0200
[Buildkite] Skip install package command in serverless builds for some packages (elastic#9686)
commit 0c2198b
Author: Mario Rodriguez Molins <mario.rodriguez@elastic.co>
Date: Thu Apr 25 11:41:42 2024 +0200
[Buildkite] Add retry suffix for logs (elastic#9703)
commit d932e79
Author: Simon Kötting <145989254+SimonKoetting@users.noreply.github.com>
Date: Thu Apr 25 07:35:45 2024 +0200
[Exchange Server] GA of Integration, Add Dashbord Panel Titles & System Tests (elastic#9560)
* Add Dashboard Titles
* Add Dashboard Titles
* Change Version to GA
* adjust PR in Changelog
* Add System Tests to all datstreams
* fix imap system test config
* remove Folder structure out of system tests sample logs
* Fix mapping
* Add convert for inode field
* specify numeric_keyword_fields in system tests
commit dba2901
Author: Dan Kortschak <dan.kortschak@elastic.co>
Date: Thu Apr 25 10:21:30 2024 +0930
rapid7_insightvm: canonicalize host.name to lower case and map subdomain to host.hostname (elastic#9665)
commit 4284262
Author: Panos Koutsovasilis <panos.koutsovasilis@elastic.co>
Date: Wed Apr 24 20:34:13 2024 +0300
fix(fim): add auto option for backend and make it the default one (elastic#9702)
commit c563bb3
Author: Panos Koutsovasilis <panos.koutsovasilis@elastic.co>
Date: Wed Apr 24 19:40:04 2024 +0300
[juniper_netscreen]: include log.file.device_id and log.file.inode in base-fields (elastic#9658)
* fix(juniper_netscreen): include log.file.device_id and log.file.inode in base-fields.yml
* fix(juniper_netscreen): update README.md
commit f187d0d
Author: Panos Koutsovasilis <panos.koutsovasilis@elastic.co>
Date: Wed Apr 24 19:11:28 2024 +0300
[juniper_junos]: include log.file.device_id and log.file.inode in base-fields (elastic#9657)
* fix(juniper_junos): include log.file.device_id and log.file.inode in base-fields.yml
* fix(juniper_junos): update README.md
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed commit message
cloudtraildata stream ofAWSSample Response
sample_event.json
{ "s3": { "bucket": { "name": "elastic-package-aws-bucket-63461", "arn": "arn:aws:s3:::elastic-package-aws-bucket-63461" }, "object": { "key": "wolverine-powerboar.log" } }, "agent": { "name": "aws-scale-123456", "id": "de42127b-4db8-4471-824e-a7b14f478663", "ephemeral_id": "22ed892c-43bd-408a-9121-65e2f5b6a56e", "type": "filebeat", "version": "8.8.0" }, "offset": 0, "benchmark_metadata": { "info": { "run_id": "9afb8980-43c0-4c06-9fcf-55f54e9e58dd", "benchmark": "cloudtrail-benchmark" } }, "elastic_agent": { "id": "de42127b-4db8-4471-824e-a7b14f478663", "version": "8.8.0", "snapshot": false }, "source": { "address": "127.0.0.1", "ip": "127.0.0.1" }, "tags": [ "preserve_original_event", "forwarded", "aws-cloudtrail" ], "cloud": { "region": "us-east-1", "account": { "id": "0123456789012" } }, "input": { "type": "aws-s3" }, "@timestamp": "2020-01-10T16:06:40.000Z", "ecs": { "version": "8.0.0" }, "related": { "user": [ "Alice" ] }, "data_stream": { "namespace": "ep", "type": "logs", "dataset": "aws.cloudtrail" }, "event": { "agent_id_status": "auth_metadata_missing", "ingested": "2024-04-02T13:02:42Z", "original": "{ \"eventVersion\": \"1.05\", \"userIdentity\": { \"type\": \"IAMUser\", \"principalId\": \"EXAMPLE_ID\", \"arn\": \"arn:aws:iam::0123456789012:user/Alice\", \"accountId\": \"0123456789012\", \"accessKeyId\": \"EXAMPLE_KEY\", \"userName\": \"Alice\", \"sessionContext\": { \"attributes\": { \"mfaAuthenticated\": \"true\", \"creationDate\": \"2020-01-10T14:38:30Z\" }, \"sessionIssuer\": { \"accountId\": \"111111111111\", \"arn\": \"arn:aws:iam::111111111111:role/JohnRole1\", \"principalId\": \"AROAIN5ATK5U7KEXAMPLE\", \"type\": \"Role\" } }, \"invokedBy\": \"signin.amazonaws.com\" }, \"eventTime\": \"2020-01-10T16:06:40Z\", \"eventSource\": \"iam.amazonaws.com\", \"eventName\": \"UploadSSHPublicKey\", \"awsRegion\": \"us-east-1\", \"sourceIPAddress\": \"127.0.0.1\", \"userAgent\": \"signin.amazonaws.com\", \"requestParameters\": { \"sSHPublicKeyBody\": \"ssh-rsa AAAAdeadcodedeadcode Alice@localhost.domain\", \"userName\": \"Alice\" }, \"responseElements\": { \"sSHPublicKey\": { \"fingerprint\": \"de:ad:c0:de:de:ad:c0:de:de:ad:c0:de:de:ad:c0:de\", \"status\": \"Active\", \"uploadDate\": \"Jan 10, 2020 4:06:40 PM\", \"userName\": \"Alice\", \"sSHPublicKeyId\": \"EXAMPLE_KEY_ID\", \"sSHPublicKeyBody\": \"ssh-rsa AAAAdeadcodedeadcode Alice@localhost.domain\" } }, \"requestID\": \"EXAMPLE-44b9-41cd-90f2-EXAMPLE\", \"eventID\": \"EXAMPLE-9a9d-4da4-9998-EXAMPLE\", \"eventType\": \"AwsApiCall\", \"recipientAccountId\": \"0123456789012\" }", "provider": "iam.amazonaws.com", "created": "2024-04-02T18:32:37.612+05:30", "kind": "event", "action": "UploadSSHPublicKey", "id": "EXAMPLE-9a9d-4da4-9998-EXAMPLE", "type": [ "info" ], "dataset": "aws.cloudtrail", "outcome": "success" }, "aws": { "cloudtrail": { "event_version": "1.05", "flattened": { "request_parameters": { "sSHPublicKeyBody": "ssh-rsa AAAAdeadcodedeadcode Alice@localhost.domain", "userName": "Alice" }, "response_elements": { "sSHPublicKey": { "sSHPublicKeyBody": "ssh-rsa AAAAdeadcodedeadcode Alice@localhost.domain", "sSHPublicKeyId": "EXAMPLE_KEY_ID", "uploadDate": "Jan 10, 2020 4:06:40 PM", "fingerprint": "de:ad:c0:de:de:ad:c0:de:de:ad:c0:de:de:ad:c0:de", "userName": "Alice", "status": "Active" } } }, "event_type": "AwsApiCall", "user_identity": { "access_key_id": "EXAMPLE_KEY", "invoked_by": "signin.amazonaws.com", "session_context": { "session_issuer": { "account_id": "111111111111", "type": "Role", "arn": "arn:aws:iam::111111111111:role/JohnRole1", "principal_id": "AROAIN5ATK5U7KEXAMPLE" }, "mfa_authenticated": "true", "creation_date": "2020-01-10T14:38:30.000Z" }, "type": "IAMUser", "arn": "arn:aws:iam::0123456789012:user/Alice" }, "recipient_account_id": "0123456789012", "request_parameters": "{sSHPublicKeyBody=ssh-rsa AAAAdeadcodedeadcode Alice@localhost.domain, userName=Alice}", "request_id": "EXAMPLE-44b9-41cd-90f2-EXAMPLE", "response_elements": "{sSHPublicKey={sSHPublicKeyBody=ssh-rsa AAAAdeadcodedeadcode Alice@localhost.domain, sSHPublicKeyId=EXAMPLE_KEY_ID, uploadDate=Jan 10, 2020 4:06:40 PM, fingerprint=de:ad:c0:de:de:ad:c0:de:de:ad:c0:de:de:ad:c0:de, userName=Alice, status=Active}}" } }, "user": { "name": "Alice", "id": "EXAMPLE_ID", "target": { "name": "Alice" } }, "user_agent": { "original": "signin.amazonaws.com", "name": "Other", "device": { "name": "Other" } } }Checklist
How to test this PR locally
Run this command from package root
elastic-package benchmark rally --benchmark cloudtrail-benchmark -velastic-package benchmark stream --benchmark cloudtrail-benchmark -vRelated issues
Screenshots