Yahoo Finance Roblox comes back online after three-day outage
The company didn't pinpoint the cause until just hours earlier.
Roblox is finally returning to normal after a nearly three-day outage. The gaming platform's developer said it was "incrementally" bringing regions back to service after having pinpointed the cause roughly three hours earlier. The company had a possible candidate on October 30th, but didn't narrow it down until a day later.
The company didn't detail the cause, but had previously ruled out particular "experiences or partnerships." Some had blamed the outage on a Chipotle promo that launched half an hour before the failure took place on the evening of October 28th.
Whatever the reason for the outage, it may have had a lasting effect. Roblox has over 40 million daily users, and has been home to major concerts in recent months. That could leave more than a few frustrated kids, not to mention parents and creators wondering about the long-term reliability of the platform.
Quick update as we work to get things back to normal. We are incrementally bringing regions back online.
— Roblox (@Roblox) October 31, 2021
Update 01/11/21 12:15PM ET: The developer of Roblox has shared more information on what caused this weekend's three-day outage. In a blog post published over the weekend, the studio attributed the incident to a combination of factors that involved a hard-to-identify bug.
This was an especially difficult outage in that it involved a combination of several factors. A core system in our infrastructure became overwhelmed, prompted by a subtle bug in our backend service communications while under heavy load. This was not due to any peak in external traffic or any particular experience. Rather the failure was caused by the growth in the number of servers in our datacenters. The result was that most services at Roblox were unable to effectively communicate and deploy.
The difficulty the company had in identifying the glitch was what took so long for a fix to come through. Once it's done a full analysis of what happened, it plans to publish a post-mortem that will include measures it plans to take to avoid similar outages in the future. "We are sorry for the length of time it took us to restore service, and are deeply grateful for the patience and support of Roblox’s player, developer, and partner community," a spokesperson for the company told Engadget.
