Webhooks not dispatched to Atlassian Connect apps for content deleted during permanent space deletion
Incident
Report for Atlassian Developer
Resolved
Apps built on the Atlassian Connect platform can subscribe to webhooks notifying them when content is deleted in Confluence. Historically, these webhooks are sent individually when a user deletes a single piece of content, or in rapid succession for all content in a space if an administrator permanently deletes that space (as described in https://support.atlassian.com/confluence-cloud/docs/delete-a-space/#Permanently-delete-a-space).
Historically, permanent deletion has been slow and/or unreliable for spaces containing large amounts of content, resulting in frequent customer issues and support requests. In early March we rolled out some changes designed to improve the stability and performance of Confluence’s permanent space deletion functionality. These updates included shifting much of the deletion logic from the application layer into optimised bulk deletion SQL queries.
Unfortunately these changes had the unintended side-effect of preventing content deletion webhooks from being dispatched during permanent space deletions. While permanent deletions are infrequent, this change in webhook behaviour may have had an adverse impact on apps that are subscribed to them. Because of this, we have rolled back the change and have recommenced sending webhooks for content deletions that occur when a space is permanently deleted.
The problematic changes were progressively rolled out for all customers between March 7th at 20:10 UTC and March 11th at 18:20 UTC. The changes were rolled back on April 3rd at 06:27 UTC. Apps built on the Atlassian Connect framework will not have received the following webhooks for content deleted during a permanent space deletion on a site with the changes enabled:
- page_removed
- blog_removed
- attachment_removed
- comment_removed
- content_removed
Note that webhooks were delivered as normal for any content deletions that occurred outside of the context of a permanent space deletion.
We have reached out to all app developers whom we believe were impacted by this incident. If you believe you are impacted and have not been contacted via email or an ADDON issue, please submit a "Bugs" ticket at https://ecosystem.atlassian.net/servicedesk/customer/portal/34/group/106/create/567 and mention “confluence-webhooks-2024” in the ticket subject - we will endeavour to support you.
Our sincerest apologies for any inconvenience this has caused you or our mutual customers.
Historically, permanent deletion has been slow and/or unreliable for spaces containing large amounts of content, resulting in frequent customer issues and support requests. In early March we rolled out some changes designed to improve the stability and performance of Confluence’s permanent space deletion functionality. These updates included shifting much of the deletion logic from the application layer into optimised bulk deletion SQL queries.
Unfortunately these changes had the unintended side-effect of preventing content deletion webhooks from being dispatched during permanent space deletions. While permanent deletions are infrequent, this change in webhook behaviour may have had an adverse impact on apps that are subscribed to them. Because of this, we have rolled back the change and have recommenced sending webhooks for content deletions that occur when a space is permanently deleted.
The problematic changes were progressively rolled out for all customers between March 7th at 20:10 UTC and March 11th at 18:20 UTC. The changes were rolled back on April 3rd at 06:27 UTC. Apps built on the Atlassian Connect framework will not have received the following webhooks for content deleted during a permanent space deletion on a site with the changes enabled:
- page_removed
- blog_removed
- attachment_removed
- comment_removed
- content_removed
Note that webhooks were delivered as normal for any content deletions that occurred outside of the context of a permanent space deletion.
We have reached out to all app developers whom we believe were impacted by this incident. If you believe you are impacted and have not been contacted via email or an ADDON issue, please submit a "Bugs" ticket at https://ecosystem.atlassian.net/servicedesk/customer/portal/34/group/106/create/567 and mention “confluence-webhooks-2024” in the ticket subject - we will endeavour to support you.
Our sincerest apologies for any inconvenience this has caused you or our mutual customers.
This incident affected: APIs (Confluence Cloud APIs).