measurements: SSL certificate expired for 17 hours

[hellais]

Impact: https://measurements.ooni.torproject.org/ was giving SSL certificate errors to users trying to access it due to an expired certificate.

Timeline UTC:
4 Jul 13:39: [FIRING] Instance https://measurements.ooni.torproject.org/api/v1/files down
4 Jul 16:34: On IRC @hellais: @darkk are you doing something with measurements API?
4 Jul 16:36: On IRC @darkk no
5 Jul 07:05: @hellais restarts https-portal docker service and SSL certificate is renewed
5 Jul 07:19: [RESOLVED] Instance https://measurements.ooni.torproject.org/api/v1/files down

What went well:

• Alerting was timely and relevant.
• Seems, alerting checks certificate validity time (?)

What went wrong:

• Alerting to IRC was unavailable due to regression slack-irc: support notificatons from other slack bots #119
• It was not clear who was responsible for bringing the service back up
• The root cause of the issue was not identified

What could be done to prevent relapse and decrease impact:

• https-service should be debugged to ensure that it does not miss renewal in the future (this has happened before with other services)

[darkk]

I added Alerting to IRC was unavailable due to regression #119 as it was a bit unclear to me that the incident is actually going on as I was busy with #116.

[hellais]

ooni/backend#343