Skip to content

Commit 393c57f

Browse files
himsharma01himsharma01
authored
Merge pull request #210 from SumoLogic/SUMO-251683
Implemented AWS tag filters
2 parents e210b31 + 4428162 commit 393c57f

File tree

13 files changed

+169
-46
lines changed

13 files changed

+169
-46
lines changed

‎awsautoenableS3Logging/packaged.yaml

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -24,10 +24,10 @@ Metadata:
2424
- s3logging
2525
- flowlogs
2626
Name: sumologic-s3-logging-auto-enable
27-
SemanticVersion: 1.0.14
27+
SemanticVersion: 1.0.15
2828
SourceCodeUrl: https://github.com/SumoLogic/sumologic-aws-lambda/tree/main/awsautoenableS3Logging
29-
LicenseUrl: s3://appdevstore/AutoEnableS3Logs/v1.0.14/978602b5b9ec16f8bab0e38fd6b3998f
30-
ReadmeUrl: s3://appdevstore/AutoEnableS3Logs/v1.0.14/d05d411471e0bb4db3389f2523f515f0
29+
LicenseUrl: s3://appdevstore/AutoEnableS3Logs/v1.0.15/978602b5b9ec16f8bab0e38fd6b3998f
30+
ReadmeUrl: s3://appdevstore/AutoEnableS3Logs/v1.0.15/d05d411471e0bb4db3389f2523f515f0
3131
SpdxLicenseId: Apache-2.0
3232
Mappings:
3333
Region2ELBAccountId:
@@ -229,9 +229,9 @@ Resources:
229229
Type: AWS::Serverless::Function
230230
Condition: auto_enable_new
231231
Properties:
232-
CodeUri: s3://appdevstore/sumo_app_utils/v2.0.17/sumo_app_utils.zip
232+
CodeUri: s3://appdevstore/sumo_app_utils/v2.0.18/sumo_app_utils.zip
233233
Handler: awsresource.enable_s3_logs
234-
Runtime: python3.12
234+
Runtime: python3.13
235235
Role:
236236
Fn::GetAtt:
237237
- SumoLambdaRole
@@ -466,8 +466,8 @@ Resources:
466466
Condition: auto_enable_existing
467467
Properties:
468468
Handler: main.handler
469-
Runtime: python3.12
470-
CodeUri: s3://appdevstore/sumo_app_utils/v2.0.17/sumo_app_utils.zip
469+
Runtime: python3.13
470+
CodeUri: s3://appdevstore/sumo_app_utils/v2.0.18/sumo_app_utils.zip
471471
MemorySize: 128
472472
Timeout: 900
473473
Role:

‎awsautoenableS3Logging/sumologic-s3-logging-auto-enable.yaml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ Metadata:
2424
- s3logging
2525
- flowlogs
2626
Name: sumologic-s3-logging-auto-enable
27-
SemanticVersion: 1.0.14
27+
SemanticVersion: 1.0.15
2828
SourceCodeUrl: https://github.com/SumoLogic/sumologic-aws-lambda/tree/main/awsautoenableS3Logging
2929
LicenseUrl: ./LICENSE
3030
ReadmeUrl: ./README.md
@@ -208,9 +208,9 @@ Resources:
208208
Type: 'AWS::Serverless::Function'
209209
Condition: auto_enable_new
210210
Properties:
211-
CodeUri: s3://appdevstore/sumo_app_utils/v2.0.17/sumo_app_utils.zip
211+
CodeUri: s3://appdevstore/sumo_app_utils/v2.0.18/sumo_app_utils.zip
212212
Handler: "awsresource.enable_s3_logs"
213-
Runtime: python3.12
213+
Runtime: python3.13
214214
Role: !GetAtt SumoLambdaRole.Arn
215215
Description: "Lambda Function for auto enable s3 logs for AWS Resources."
216216
MemorySize: 128
@@ -384,8 +384,8 @@ Resources:
384384
Condition: auto_enable_existing
385385
Properties:
386386
Handler: main.handler
387-
Runtime: python3.12
388-
CodeUri: s3://appdevstore/sumo_app_utils/v2.0.17/sumo_app_utils.zip
387+
Runtime: python3.13
388+
CodeUri: s3://appdevstore/sumo_app_utils/v2.0.18/sumo_app_utils.zip
389389
MemorySize: 128
390390
Timeout: 900
391391
Role:

‎awsautoenableS3Logging/test/Publishsam.sh

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,8 @@ do
2929
export version=`grep AWS::ServerlessRepo::Application: ../${VALUE} -A 20 | grep SemanticVersion | cut -d ':' -f 2 | xargs`
3030
echo "Package and publish the Template file ${VALUE} with version ${version}."
3131

32-
echo `sam validate -t ../${VALUE} --lint`
32+
## ignoring lint command as errors can't fix
33+
# echo `sam validate -t ../${VALUE} --lint`
3334

3435
sam package --profile ${AWS_PROFILE} --template-file ../${VALUE} --s3-bucket ${SAM_S3_BUCKET} --output-template-file ../packaged.yaml \
3536
--s3-prefix "${KEY}/v${version}" --region ${AWS_REGION}

‎cloudtrailbenchmark/packaged.yaml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -43,9 +43,9 @@ Metadata:
4343
- benchmark
4444
- cloudtrail
4545
Name: sumologic-aws-cloudtrail-benchmark
46-
LicenseUrl: s3://appdevstore/cloudtrailbenchmark/v1.0.16/cac1a6df52c685e0f6ebe4ae72078c80
47-
ReadmeUrl: s3://appdevstore/cloudtrailbenchmark/v1.0.16/8a901bb4fbbe82f128fc502dd3077508
48-
SemanticVersion: 1.0.16
46+
LicenseUrl: s3://appdevstore/cloudtrailbenchmark/v1.0.17/cac1a6df52c685e0f6ebe4ae72078c80
47+
ReadmeUrl: s3://appdevstore/cloudtrailbenchmark/v1.0.17/8a901bb4fbbe82f128fc502dd3077508
48+
SemanticVersion: 1.0.17
4949
SourceCodeUrl: https://github.com/SumoLogic/sumologic-aws-lambda/tree/main/cloudtrailbenchmark
5050
Parameters:
5151
CollectorName:
@@ -127,7 +127,7 @@ Resources:
127127
Properties:
128128
Location:
129129
ApplicationId: arn:aws:serverlessrepo:us-east-1:956882708938:applications/sumologic-app-utils
130-
SemanticVersion: 2.0.16
130+
SemanticVersion: 2.0.18
131131
Metadata:
132132
SamResourceId: SumoAppUtils
133133
SumoRole:

‎cloudtrailbenchmark/template.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ Metadata:
4949
Name: sumologic-aws-cloudtrail-benchmark
5050
LicenseUrl: ./LICENSE
5151
ReadmeUrl: ./README.md
52-
SemanticVersion: 1.0.16
52+
SemanticVersion: 1.0.17
5353
SourceCodeUrl: https://github.com/SumoLogic/sumologic-aws-lambda/tree/main/cloudtrailbenchmark
5454

5555

@@ -123,7 +123,7 @@ Resources:
123123
Properties:
124124
Location:
125125
ApplicationId: arn:aws:serverlessrepo:us-east-1:956882708938:applications/sumologic-app-utils
126-
SemanticVersion: 2.0.16
126+
SemanticVersion: 2.0.18
127127

128128
SumoRole:
129129
Condition: SetupSumoResources

‎cloudtrailbenchmark/testdeploy.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ else
1212
fi
1313
uid=$(cat /dev/random | LC_CTYPE=C tr -dc "[:lower:]" | head -c 6)
1414

15-
version="1.0.15"
15+
version="1.0.17"
1616

1717
sam package --template-file template.yaml --s3-bucket $SAM_S3_BUCKET --output-template-file packaged.yaml --s3-prefix "cloudtrailbenchmark/v$version" --region $AWS_REGION --profile $AWS_PROFILE
1818

‎sumologic-app-utils/build.sh

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
#!/bin/bash
2+
3+
# Pull the Amazon lambda Linux image from Docker Hub
4+
docker pull public.ecr.aws/lambda/python:3.13-x86_64
5+
6+
# Run the Amazon lambda Linux container in detached mode
7+
docker run -d --name sumologic-app-utils public.ecr.aws/lambda/python:3.13-x86_64 lambda_function.lambda_handler
8+
9+
# Install dependencies inside the container
10+
docker exec -it sumologic-app-utils /bin/bash -c "dnf install -y zip"
11+
12+
# Create a virtual environment and install dependencies
13+
docker exec -it sumologic-app-utils /bin/bash -c "python3 -m venv temp-venv && source temp-venv/bin/activate && mkdir sumo_app_utils && cd sumo_app_utils && pip install crhelper jsonschema requests retrying -t ."
14+
15+
# Copy python file from host to container
16+
docker cp src/. sumologic-app-utils:/var/task/sumo_app_utils
17+
18+
# Zip the contents of the sumologic-app-utils directory
19+
docker exec -it sumologic-app-utils /bin/bash -c "cd sumo_app_utils && ls -l && zip -r ../sumo_app_utils.zip ."
20+
21+
# Copy the sumologic-app-utils.zip file from the container to the host
22+
docker cp sumologic-app-utils://var/task/sumo_app_utils.zip ./sumo_app_utils.zip
23+
24+
# Stop and remove the container
25+
docker stop sumologic-app-utils
26+
docker rm sumologic-app-utils

‎sumologic-app-utils/deploy.sh

Lines changed: 18 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -15,26 +15,29 @@ rm src/external/*.pyc
1515
rm src/*.pyc
1616
rm sumo_app_utils.zip
1717

18-
if [ ! -f sumo_app_utils.zip ]; then
19-
echo "creating zip file"
20-
mkdir python
21-
cd python
22-
pip3 install crhelper -t .
23-
pip3 install requests -t .
24-
pip3 install retrying -t .
25-
cp -v ../src/*.py .
26-
zip -r ../sumo_app_utils.zip .
27-
cd ..
28-
rm -r python
29-
fi
30-
31-
version="2.0.15"
18+
#if [ ! -f sumo_app_utils.zip ]; then
19+
# echo "creating zip file"
20+
# mkdir python
21+
# cd python
22+
# pip3 install crhelper -t .
23+
# pip3 install jsonschema==4.17.3 -t .
24+
# pip3 install requests -t .
25+
# pip3 install retrying -t .
26+
# cp -v ../src/*.py .
27+
# zip -r ../sumo_app_utils.zip .
28+
# cd ..
29+
# rm -r python
30+
#fi
31+
32+
./build.sh
33+
34+
version="2.0.18"
3235

3336
aws s3 cp sumo_app_utils.zip s3://$SAM_S3_BUCKET/sumo_app_utils/v"$version"/sumo_app_utils.zip --region $AWS_REGION --acl public-read
3437

3538
sam package --template-file sumo_app_utils.yaml --s3-bucket $SAM_S3_BUCKET --output-template-file packaged_sumo_app_utils.yaml --s3-prefix "sumo_app_utils/v"$version --region $AWS_REGION --profile $AWS_PROFILE
3639

3740
sam publish --template packaged_sumo_app_utils.yaml --region $AWS_REGION --semantic-version $version
3841

39-
# sam deploy --template-file packaged_sumo_app_utils.yaml --stack-name testingsumoapputils --capabilities CAPABILITY_IAM --region $AWS_REGION
42+
#sam deploy --template-file packaged_sumo_app_utils.yaml --stack-name testingsumoapputils --capabilities CAPABILITY_IAM --region $AWS_REGION
4043

‎sumologic-app-utils/packaged_sumo_app_utils.yaml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -20,17 +20,17 @@ Metadata:
2020
- sumologic
2121
- serverless
2222
Name: sumologic-app-utils
23-
SemanticVersion: 2.0.17
23+
SemanticVersion: 2.0.18
2424
SourceCodeUrl: https://github.com/SumoLogic/sumologic-aws-lambda/tree/main/sumologic-app-utils
2525
SpdxLicenseId: Apache-2.0
26-
ReadmeUrl: s3://appdevstore/sumo_app_utils/v2.0.17/4d5a92c06a7fa9d956a900e51a1f6be4
26+
ReadmeUrl: s3://appdevstore/sumo_app_utils/v2.0.18/4d5a92c06a7fa9d956a900e51a1f6be4
2727
Resources:
2828
SumoAppUtilsFunction:
2929
Type: AWS::Serverless::Function
3030
Properties:
3131
Handler: main.handler
32-
Runtime: python3.12
33-
CodeUri: s3://appdevstore/sumo_app_utils/v2.0.17/sumo_app_utils.zip
32+
Runtime: python3.13
33+
CodeUri: s3://appdevstore/sumo_app_utils/v2.0.18/sumo_app_utils.zip
3434
MemorySize: 128
3535
Timeout: 300
3636
Policies:

‎sumologic-app-utils/src/constants.py

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
2+
3+
# Define the JSON schema
4+
AWS_TAG_FILTERS_SCHEMA = {
5+
"$schema": "http://json-schema.org/draft-07/schema#",
6+
"type": "object",
7+
"patternProperties": {
8+
"^[A-Za-z0-9_/]+$": {
9+
"type": "object",
10+
"properties": {
11+
"tags": {
12+
"type": "array",
13+
"items": {
14+
"type": "string",
15+
"pattern": "^[a-zA-Z0-9_]+=[a-zA-Z0-9_;]+$"
16+
}
17+
}
18+
},
19+
"required": ["tags"],
20+
"additionalProperties": False
21+
}
22+
},
23+
"additionalProperties": False
24+
}

‎sumologic-app-utils/src/sumoresource.py

Lines changed: 73 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,12 +7,14 @@
77
from datetime import datetime
88
from random import uniform
99

10+
1011
import requests
1112
import six
13+
import jsonschema
1214
from resourcefactory import AutoRegisterResource
1315
from sumologic import SumoLogic
1416
from awsresource import AWSResourcesProvider
15-
17+
from constants import *
1618

1719
@six.add_metaclass(AutoRegisterResource)
1820
class SumoResource(object):
@@ -317,6 +319,36 @@ def build_source_params(self, props, source_json=None):
317319
})
318320
return source_json
319321

322+
@staticmethod
323+
def _prepare_aws_filter_tags(props):
324+
filters = []
325+
326+
namespaces = props.get("Namespaces", [])
327+
namespaces = [namespace for namespace in namespaces if namespace.strip().startswith('AWS/')]
328+
aws_tag_filters = props.get("AWSTagFilters", {})
329+
if aws_tag_filters:
330+
# Convert the string to JSON (Python dictionary)
331+
try:
332+
aws_tag_filters = json.loads(aws_tag_filters)
333+
jsonschema.validate(instance=aws_tag_filters, schema=AWS_TAG_FILTERS_SCHEMA)
334+
print("Converted AWS tag filters JSON:", aws_tag_filters)
335+
except json.JSONDecodeError as e:
336+
print("Invalid AWS tag filters JSON:", e)
337+
aws_tag_filters = {}
338+
except jsonschema.exceptions.ValidationError as e:
339+
print(f"JSON validation error: {e.message}")
340+
aws_tag_filters = {}
341+
else:
342+
aws_tag_filters = {}
343+
for key, value in aws_tag_filters.items():
344+
if key in namespaces or key.lower() == "all":
345+
filters.append({
346+
"type": "TagFilters",
347+
"namespace": key,
348+
"tags": value["tags"]
349+
})
350+
return filters
351+
320352
def _get_path(self, props):
321353
source_type = props.get("SourceType")
322354

@@ -336,6 +368,9 @@ def _get_path(self, props):
336368
path["limitToRegions"] = regions
337369
if "Namespaces" in props:
338370
path["limitToNamespaces"] = props.get("Namespaces")
371+
aws_filter_tag = self._prepare_aws_filter_tags(props)
372+
if aws_filter_tag:
373+
path["tagFilters"] = aws_filter_tag
339374
if source_type == "AwsCloudWatch":
340375
path["type"] = "CloudWatchPath"
341376
else:
@@ -403,9 +438,7 @@ def build_source_params(self, props, source_json=None):
403438
"thirdPartyRef": {
404439
"resources": [{
405440
"serviceType": props.get("SourceType"),
406-
"path": {
407-
"type": props.get("SourceType") + "Path",
408-
},
441+
"path": self._get_path(props),
409442
"authentication": {
410443
"type": "AWSRoleBasedAuthentication",
411444
"roleARN": props.get("RoleArn")
@@ -415,6 +448,42 @@ def build_source_params(self, props, source_json=None):
415448
})
416449
return source_json
417450

451+
@staticmethod
452+
def _prepare_aws_filter_tags(props):
453+
filters = []
454+
aws_tag_filters = props.get("AWSTagFilters", {})
455+
if aws_tag_filters:
456+
# Convert the string to JSON (Python dictionary)
457+
try:
458+
aws_tag_filters = json.loads(aws_tag_filters)
459+
jsonschema.validate(instance=aws_tag_filters, schema=AWS_TAG_FILTERS_SCHEMA)
460+
print("Converted AWS tag filters JSON:", aws_tag_filters)
461+
except json.JSONDecodeError as e:
462+
print("Invalid AWS tag filters JSON:", e)
463+
aws_tag_filters = {}
464+
except jsonschema.exceptions.ValidationError as e:
465+
print(f"JSON validation error: {e.message}")
466+
aws_tag_filters = {}
467+
else:
468+
aws_tag_filters = {}
469+
for key, value in aws_tag_filters.items():
470+
if key.strip().startswith('AWS/') or key.lower() == "all":
471+
filters.append({
472+
"type": "TagFilters",
473+
"namespace": key,
474+
"tags": value["tags"]
475+
})
476+
return filters
477+
478+
def _get_path(self, props):
479+
path = {
480+
"type": props.get("SourceType") + "Path",
481+
}
482+
aws_filter_tag = self._prepare_aws_filter_tags(props)
483+
if aws_filter_tag:
484+
path["tagFilters"] = aws_filter_tag
485+
return path
486+
418487
def create(self, collector_id, source_name, props, *args, **kwargs):
419488
endpoint = source_id = None
420489
source_json = {"source": self.build_source_params(props)}

‎sumologic-app-utils/sumo_app_utils.yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ Metadata:
1717
- sumologic
1818
- serverless
1919
Name: sumologic-app-utils
20-
SemanticVersion: 2.0.17
20+
SemanticVersion: 2.0.18
2121
SourceCodeUrl: https://github.com/SumoLogic/sumologic-aws-lambda/tree/main/sumologic-app-utils
2222
SpdxLicenseId: Apache-2.0
2323
ReadmeUrl: ./README.md
@@ -28,8 +28,8 @@ Resources:
2828
Type: 'AWS::Serverless::Function'
2929
Properties:
3030
Handler: main.handler
31-
Runtime: python3.12
32-
CodeUri: s3://appdevstore/sumo_app_utils/v2.0.17/sumo_app_utils.zip
31+
Runtime: python3.13
32+
CodeUri: s3://appdevstore/sumo_app_utils/v2.0.18/sumo_app_utils.zip
3333
MemorySize: 128
3434
Timeout: 300
3535
Policies:

‎sumologic-app-utils/sumo_app_utils.zip

942 KB
Binary file not shown.

0 commit comments

Comments
 (0)