Resolved
squareweb.app Root Domain Block Incident
Today, our squareweb.app domain experienced a significant service interruption. The incident was not the result of a direct external attack, but was caused by a human error within our internal network operations team. Our team received a high volume of legitimate phishing reports from Cloudflare, which were correctly escalated to our network team for mitigation.
Our standard procedure is to block only the specific subdomains responsible for the abuse. However, while processing this large and sudden volume of reports, a manual error was made, and the block was incorrectly applied to the entire squareweb.app root domain. This action immediately cut off service for all clients and unfortunately triggered several cascading failures across our dependent infrastructure.
The error was identified by our engineering team after the cascading alerts began, and the incorrect block was immediately removed. We have since restored service, though we are still resolving the subsequent issues from the cascading failure. We are conducting a full post-mortem to understand the breakdown in our process and are implementing new technical safeguards and procedural checks to prevent a single point of failure like this from ever happening again. We sincerely apologize for the disruption this internal error caused.
Resolved
·
Maintenance