[Category] Webhooks

[robotdan]

General Webhook enhancements

Problem

As we look forward to enable FusionAuth as a SCIM client we want to enhance webhooks to have some additional capability that will make it easier to support SCIM.

Solution

This is a general issues to enhance webhooks. Some specific features we'd like to deliver:

1. Persist a historical log of all webhooks. Options for view, resend, maybe a delete option to prune events?
2. Allow a retry from a webhook event that was not successfully received
   • Resend / Retry a single event to one or more webhooks
   • (SCIM) Optionally send all events beginning at a specific sequence to reset or catch up from a point time to "now".
3. Additional configuration for retry logic. Back off logic and then finally move to a failed state to allow for a future manual retry. Or optionally try forever.
4. A webhook event should record which webhooks it was sent to, the status code of the webhook returned a response, etc. i.e. which ones succeeded and which ones failed in the event viewer , webhook event log.
5. Possibly additional configuration for queuing, and ordering.
6. Remove application join table, this is confusing and is not useful.
7. Add a tenant join table, allow webhooks to be enabled for one-to-many tenants, or all
   • Delivered via Move Webhoks to a Tenant config instead of Application #1812
8. Error code mapping. For example, what is success for a particular webhook, currently must be 200.
   • Optionally map retry logic to particular status codes.
9. Upsert options. For example, a Update event is sent and the webhook returns 404, we could convert this to a Create event. This is sort of a SCIM advanced feature.
10. Allow the webhook to return a message that can be displayed to an end user or an API response.
    • Webhook Status Code Definitions #814 (comment)
11. Better certificate management using Key Master
12. Disable a webhook to temporarily stop events without deleting it or modifying any existing event configuration.
    • When "disabled" or "paused" config to say still consider for events, meaning instead of sending, just queue pending and wait for it to come back online or for paused to be moved to "play".
13. Health checks for webhooks, health checks for the number of failed or pending events, queued, etc.
    • DropWizard health checks that show up in /api/status or /api/prometheus/metrics
14. Add a config per webhook or per event for a max duration to wait for the webhook to respond.
15. Possibly add a config to a webhook to indicate if this webhook should be considered in the TX configuration. For example, a single webhook is a "fire and forget" style (3rd party) and you don't want it to cause a TX failure, even though the other two Webhooks are "yours" and they are critical.
16. (SCIM) Bulk Event sender options
17. Review all current events to see if any more of them should be considered non-transactional. For example, should UserLoginSuccessEvent and UserLoginSuspiciousEvent be allowed to be configured as transactional? Or can we just fire and forget.
18. Allow a lambda to be assigned to handle webhook events in addition to, or instead of a webhook.

Related

• https://github.com/FusionAuth/fusionauth-cloud/issues/18
• Queue for faulty webhooks #391
• Can a webhook supply user uuid as parameter to external API? #443
• Support JWT authentication for Webhooks #569
• Webhook Status Code Definitions #814
• Modify webhook transaction failure status code #1250
• Add a webhook event log #1314
• Add ability to have static ip for FusionAuth cloud instances #1393
• Add MFA Abandoned and MFA Failed webhooks #1431
• [Feature] New Webhook Event: Rate limiting event #1605
• Add ability to pass value from login form to webhook #1660
• [Feature] New Webhook Event: Rate limiting event #1605
• Add ability to pass value from login form to webhook #1660
• Allow webhooks to return data to be merged and persisted with FusionAuth data #1696
• Determine if a webhook was triggered by an API client #1698
• Consider making the IdP Link and Unlink webhooks transactional #1710
• Enable Lambdas to process events #1781
• Update login failure event to indicate if it was first factor or second factor that failed #1888
• User Profile Update Lambda Trigger #1914
• Add UX notice if a user has webhooks but hasn't configured them for any tenant #2016
• Event for when a refresh token is issued #2091
• Provide authentication type details on Event when a User registers #2148
• Add the ability to propagate context between API and WebHooks #2263
• WebHooks may throw exceptions under high load #2330
• Add a webhook when cloud instance starts/stops #2356
• Implement SCIM Client functionality #2377

Completed

• [Feature] SCIM support #106
• Application webhooks not firing #169
• Add Webhook for Group Membership Changes #633
• Better error message when test webhook run from the admin UI fails #793
• Webhook test json is reset each time you test from the admin UI #797
• Have webhook certificates managed by keymaster #883
• Have login.success webhook only fired when user and registration are complete objects #1130
• Webhooks user.delete and user.create don't fire if set to specific application  #1354
• Add thread pool metrics for Webhooks to the health check / status endpoint #1499
• Event/Webhook on User IDP link creation #1525
• Add Tooltip for All Applications Configuration in Webhooks #1542
• Add A Webhook Event to listen for Idp Link and Unlink on the user #1589
• Tenant webhook settings display non-tenant webhooks #1593
• User Action Does Not Fire Webhook #1654
• Webhooks for group create and group delete events #1803
• Move Webhoks to a Tenant config instead of Application #1812
• Add config for Webhook Signature #1859
• Webhook Event user.registration.update.complete not sending updated roles #1898
• Invalid error when webhook fails at MFA-input #1955
• metadata.device.description not always sent in a webhook #2100

Community guidelines

All issues filed in this repository must abide by the FusionAuth community guidelines.

How to vote

Please give us a thumbs up or thumbs down as a reaction to help us prioritize this feature. Feel free to comment if you have a particular need or comment on how this feature should work.

[hughevans]

Add a tenant join table, allow webhooks to be enabled for one-to-many tenants

This would cover the use case we have where in our development instance we have multiple tenants and each tenant needs to have unique webhook URLs. Another solve would be for webhooks to belong to tenants.

[glen-84]

Add a tenant join table, allow webhooks to be enabled for one-to-many tenants

It's really surprising to me that webhooks are not scoped to tenants (and/or applications). It means that creating a user (for example) will fire a hook across n tenants, resulting in:

1. Exposure of user data across multiple tenants that should otherwise be isolated. (not an issue for us, but could affect others)
2. Reduced performance having to wait for all hooks to succeed (we can't just wait for one). This is really bad if some tenants are staging environments with far less performance. This will affect us.
3. Transaction failure if one of the hooks fails in an unrelated tenant. This means that a failing request sent to a staging URL would mean that a production user could not sign up (!). This will affect us.
4. The requirement to filter by tenant or application in the webhook.
5. A lot of confusion (as seen in Application webhooks not firing #169).

Is this issue being prioritised yet? It seems quite serious. Creating another deployment just for a different environment can be costly.

(We're looking into the option of using a separate deployment, though this was an unexpected cost.)

[robotdan]

@robfusion can you open a separate GH issue that is linked to this one to track the tenant change we are making so that we can track it to close and leave this one open as the larger project task? The tenant work you are doing will essentially be delivering item 6 and 7 from the above list.

[glen-84]

#1660 is listed twice.