Skip to content

Commit 4287835

Browse files
authored
Fix ec2 daemon service cpu & memory values (#96)
Manually testing the cgroup cpu and memory limits in an EC2 instance revealed a problem with the values we set in the `.service` config file. As it turns out, we can't place comments on the same line as the value because the comments will be interpreted as part of the value. Running `journalctl -fu postman-insights-agent.service` in the ec2 instance revealed the following logs: ``` Mar 21 00:51:45 ip-172-31-76-65.ec2.internal systemd[1]: /usr/lib/systemd/system/postman-insights-agent.service:14: Invalid CPU weight, ignoring: 90 #comment Mar 21 00:51:45 ip-172-31-76-65.ec2.internal systemd[1]: /usr/lib/systemd/system/postman-insights-agent.service:15: Invalid CPU quota '50% #comment', ignoring. Mar 21 00:51:45 ip-172-31-76-65.ec2.internal systemd[1]: /usr/lib/systemd/system/postman-insights-agent.service:18: Invalid memory limit '750M #comment', ignoring: Invalid argument Mar 21 00:51:45 ip-172-31-76-65.ec2.internal systemd[1]: /usr/lib/systemd/system/postman-insights-agent.service:19: Invalid memory limit '1G #comment', ignoring: Invalid argument ``` Additionally, when I check the cgroup settings set under `/sys/fs/cgroup/system.slice/postman-insights-agent.service`, I see that: - `cpu.weight` has value `100` - `cpu.max` has value `max 100000` - `memory.high` has value `max` - `memory.low` has value `0` - `memory.max` has value `max` - `memory.min` has value `0` Which are the default values for all of these settings according to this [page](https://www.freedesktop.org/software/systemd/man/latest/systemd.resource-control.html)! Removing the comments and running: ``` $ sudo systemctl daemon-reload $ sudo systemctl restart postman-insights-agent.service $ journalctl -fu postman-insights-agent.service ``` Got rid of the errors from `systemd[1]` and checking the cgroup settings again I see that: - `cpu.weight` has value `90` - `cpu.max` has value `50000 100000` - `memory.high` has value `786432000` - `memory.low` has value `0` - `memory.max` has value `1073741824` - `memory.min` has value `0` Which is what we would expect based on the values we configured.
1 parent b507329 commit 4287835

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed

cmd/internal/ec2/postman-insights-agent.service.tmpl

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -11,12 +11,12 @@ EnvironmentFile=/etc/default/postman-insights-agent
1111
ExecStart={{.AgentInstallPath}} apidump --project "${PROJECT_ID}" --interfaces "${INTERFACES}" --filter "${FILTER}" {{.ExtraApidumpArgs}} $EXTRA_APIDUMP_ARGS
1212

1313
# CPU resource controls
14-
CPUWeight=90 # lower relative CPU priority; default is 100. In case of contention main process will be prioritized.
15-
CPUQuota=50% # aboslute CPU time limit (85% of CPU core). Process won't be able to take advantage of multi core.
14+
CPUWeight=90
15+
CPUQuota=50%
1616

1717
# Memory resource controls
18-
MemoryHigh=750M # requested limit
19-
MemoryMax=1G # hard limit that triggers OOM kill if exceeded
18+
MemoryHigh=750M
19+
MemoryMax=1G
2020

2121
# Restart on OOMs
2222
Restart=on-failure

0 commit comments

Comments
 (0)