Logs from S3 bucket in another Account

[adiospeds]

Hello All,

I have logs going into an s3 bucket in an AWS account X. And CLO is installed in AWS Account Y. If I goto CLO > Application log > Create Pipeline > S3 .. it only allow me to select s3 bucket from same account where CLO is installed (account Y) only even thought Account X is enrolled. Is there a known way I can achieve sending of those logs from S3 account to opensearch via CLO ?

Regions(us-east-1) is same for all S3, CLO and Opensearch. (While I have tried this, the Marketplace App doesn't allow me to write those logs to another account Y so we cant go that route as of now) Please requesting some guidance here if I am missing something here wrt CLO on how this can be achieved. ?

[jangidms]

Hi @adiospeds

Currently, CLO doesn't natively support cross-account ingestion for Application logs with S3 as source.
Below is one of the workaround you can try -

1. Create an Application Log pipeline with any bucket from CLO deployed account as source
2. Modify EventBridge rule:
   • Locate the EventBridge rule in the pipeline stack
   • Update the rule pattern to reference the bucket from your source account
3. Update IAM permissions:
   • Add sts:AssumeRole permission to the LogProcessor lambda role for Member Account Role (found in SubAccount stack output as MemberAccountRoleARN)
4. Update LogProcessor environment variables:
   • Set LOG_SOURCE_ACCOUNT_ASSUME_ROLE to the MemberAccountRoleARN value
   • Set LOG_SOURCE_ACCOUNT_ID to your source account ID
5. Modify LogProcessor Lambda code:.
   • In event/event_parser.py file, change: if log_type in ("Lambda", "RDS") or IS_APP_PIPELINE: to if log_type in ("Lambda", "RDS"):

After implementing these changes, the pipeline will ingest logs from your source account bucket.

[adiospeds]

Thanks @jangidms - This is helpful. I will try this out and post here.