`@sentry/serverless` does not send Error events on crashes (GCP Function)

[alexbchr]

• Review the documentation: https://docs.sentry.io/
• Search for existing issues: https://github.com/getsentry/sentry-javascript/issues
• Use the latest release: https://github.com/getsentry/sentry-javascript/releases
• Provide a link to the affected event from your Sentry account

Package + Version

Using:

• @sentry/serverless@6.2.5
• @sentry/tracing@6.2.5

Description

When using @sentry/serverless, uncaught exception are not sent to Sentry when hosted on GCP Functions (or when executed locally using @google-cloud/functions-framework package). Here is the basic code I have for this integration:

import * as Sentry from '@sentry/serverless'
import * as SentryTracing from '@sentry/tracing'
import { HttpFunction } from '@google-cloud/functions-framework/build/src/functions'

// Necessary so Sentry doesn't crash on error (because of Webpack tree-shaking)
SentryTracing.addExtensionMethods()

Sentry.GCPFunction.init({
  dsn: 'SENTRY_DSN_HERE',
  tracesSampleRate: 1.0,
  environment: 'development',
  debug: true,
})

const myHandler: HttpFunction = async (req, res) => {
  if (req.method === 'POST') {
    throw new Error('Dummy function error')
  }

  // Rest of code...
}

export const myFunc = Sentry.GCPFunction.wrapHttpFunction(myHandler)

When running this locally, here is the output I get when calling this with a manual POST request (it doesn't work either when deployed on GCP Functions):

Sentry Logger [Log]: Integration installed: InboundFilters
Sentry Logger [Log]: Integration installed: FunctionToString
Sentry Logger [Log]: Integration installed: Console
Sentry Logger [Log]: Integration installed: Http
Sentry Logger [Log]: Integration installed: OnUncaughtException
Sentry Logger [Log]: Integration installed: OnUnhandledRejection
Sentry Logger [Log]: Integration installed: LinkedErrors
Sentry Logger [Log]: Integration installed: GoogleCloudHttp
Sentry Logger [Log]: Integration installed: GoogleCloudGrpc
Serving function...
Function: contactUs
URL: http://localhost:8081/
Sentry Logger [Log]: [Tracing] starting gcp.function.http transaction - POST /
This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason:
Error: Dummy function error
    at /Users/username/Projects/projectName/packages/functions-contact-us/dist/index.js:2:560827
    at /Users/username/Projects/projectName/packages/functions-contact-us/dist/index.js:2:561650
    at /Users/username/Projects/projectName/packages/functions-contact-us/dist/index.js:2:60124
    at /Users/username/Projects/projectName/packages/functions-contact-us/dist/index.js:2:54696
    at Object.apply (/Users/username/Projects/projectName/packages/functions-contact-us/dist/index.js:2:54787)
    at /Users/username/Projects/projectName/node_modules/@google-cloud/functions-framework/build/src/invoker.js:100:17
    at processTicksAndRejections (internal/process/task_queues.js:79:11)
Unhandled rejection
Error: Dummy function error
    at /Users/username/Projects/projectName/packages/functions-contact-us/dist/index.js:2:560827
    at /Users/username/Projects/projectName/packages/functions-contact-us/dist/index.js:2:561650
    at /Users/username/Projects/projectName/packages/functions-contact-us/dist/index.js:2:60124
    at /Users/username/Projects/projectName/packages/functions-contact-us/dist/index.js:2:54696
    at Object.apply (/Users/username/Projects/projectName/packages/functions-contact-us/dist/index.js:2:54787)
    at /Users/username/Projects/projectName/node_modules/@google-cloud/functions-framework/build/src/invoker.js:100:17
    at processTicksAndRejections (internal/process/task_queues.js:79:11)
Error: Process exited with code 16
    at process.<anonymous> (/Users/username/Projects/projectName/node_modules/@google-cloud/functions-framework/build/src/invoker.js:275:22)
    at process.emit (events.js:314:20)
    at process.EventEmitter.emit (domain.js:483:12)
    at process.exit (internal/process/per_thread.js:168:15)
    at Object.sendCrashResponse (/Users/username/Projects/projectName/node_modules/@google-cloud/functions-framework/build/src/logger.js:37:9)
    at process.<anonymous> (/Users/username/Projects/projectName/node_modules/@google-cloud/functions-framework/build/src/invoker.js:271:22)
    at process.emit (events.js:326:22)
    at process.EventEmitter.emit (domain.js:483:12)
    at processPromiseRejections (internal/process/promises.js:209:33)
    at processTicksAndRejections (internal/process/task_queues.js:98:32)

If I manually use Sentry.captureException(error), errors are sent correctly to Sentry, but just instrumenting the code as in the Sentry GCP documentation just doesn't track the errors at all. Also, I don't know if it is normal that I get the same error twice in the console logs.

I honestly don't understand what is going on and feel like I will need to manually annotate my code with try/catch and Sentry.captureException(error) everywhere in order for the Crash reporting to work... However, this should all be done automatically by @sentry/serverless.

[pravinba9495]

@alexbchr Hi Alex, I believe the one of the error log is because of the OnUncaughtException extension method and the other one is because of the OnUnhandledRejection extension method (from SentryTracing.addExtensionMethods()). Looking closely at the errors, I noticed that the OnUnhandledRejection is causing the process to crash. I have encountered the error-like warnings from Node for missing try/catch with async/await, but the process never exited with an error code. I hope this comment may help with the further investigation of this issue.

[alexbchr]

@pravinba9495 Thanks Praveen for the direction!

However, the OnUncaughtException and OnUnhandledRejection are meant to be there, as it is those Sentry Integrations that make sure that crashes on "Uncaught Exceptions" and on "Unhandled Promise Rejections" get sent to Sentry.

I believe you're kind of right though about the issue coming from OnUnhandledRejection, as from what I understood of this issue when opening it, it happens mainly because of the way the @google-cloud/functions-framework package handle errors. Basically, the error throws, Sentry intercepts it and wants to send it using Sentry.flush(), which is an async function. However, this function never has the time to complete before @google-cloud/functions-framework ends the process.

This exact issue is described in this issue in the @google-cloud/functions-framework repo : GoogleCloudPlatform/functions-framework-nodejs#215

I honestly don't understand how the referenced issue was never fix but @sentry/serverless GCP support is supposed to be working.

[pravinba9495]

@alexbchr I tested a similar piece of code. But this time, sentry was able to receive the events. First I thought, may be it was a hit-or-miss scenario. I invoked the http function many times, and all the times, sentry was able to receive them (process still crashes though). The only change I did this time is to re-wrap the Sentry.GCPFunction.wrapHttpFunction inside functions.https.onRequest. I am not sure if that fixes it.

import * as Sentry from '@sentry/serverless'
import * as SentryTracing from '@sentry/tracing'
import * as functions from 'firebase-functions';

// Necessary so Sentry doesn't crash on error (because of Webpack tree-shaking)
SentryTracing.addExtensionMethods()

Sentry.GCPFunction.init({
    dsn: 'some_dsn_from_sentry',
    tracesSampleRate: 1.0,
    environment: 'development',
    debug: true,
})

const helloHttp = Sentry.GCPFunction.wrapHttpFunction((req, res) => {
    let message = req.query.message || req.body.message || 'Hello World!';
    throw new Error('oh, hello there!');
    res.status(200).send(message);
});

export const httpHandler = functions.https.onRequest(helloHttp);

As you can see, I am not explicitly catching exceptions/rejections, just like in your example.

Also check callbackWaitsForEmptyEventLoop & WrapperOptions from here.

The code was tested directly from GCP (deployed via firebase-cli) and not hosted locally.

[alexbchr]

@pravinba9495 Thanks for the attempt at it. However, we are using GCP Functions directly (not via Firebase) with the @google-cloud/functions-framework package. With the GCP Functions package, the serverless function is exported directly and not wrapped by something like functions.https.onRequest.

[github-actions]

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you label it Status: Backlog or Status: In Progress, I will leave it alone ... forever!

---

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀