Petlibro founder and CEO York Wu provided more information about an outage that left users unable to control their smart pet feeders through the companion app and, in some reported cases, left them with hungry pets.
Petlibro, a company owned by Shenzhen, China-headquartered firm Shenzhen Libro Technology Co., first acknowledged the outage on Tuesday at 5 am PT. At 7 pm PT on Wednesday, Petlibro said the outage was resolved. Notably, some people online reported different information at the time, and some claimed to still have problems as of August 13.
In a blog post on Saturday that included an apology and acknowledgement of the “frustration and concern” the outage caused, Wu confirmed that problems with Petlibro’s cloud server caused the outage:
The outage was due to cloud server issues that caused a backlog of data requests, overloading the system memory and bringing down device service in the process. After successfully restarting the cloud service, the backlog reoccurred and the system went down again. At that point, the team conducted systematic troubleshooting of the cloud service system, identified the underlying issue, patched a solution, and then closely monitored the slow upscale of traffic towards stable recovery.
Because of this outage, the app was no longer able to connect to devices and as a result device communication was not possible. Services related to the app, including on-demand actions, operation history, records, notifications, and videos, were unavailable.
Wu noted that Petlibro has “increased demand capacity” and is “preparing an audit protocol to search our backend and work to resolve any similar inconsistencies.”
Outage timeline
Wu said that at 5:20 am PT on Tuesday, August 11, a “critical error occurred affecting app login and control.” At 6:20 am PT, “app functionality was restored, and the system began to process a large backlog of data requests resulting from the outage.” But 23 minutes later, “a flood of incoming requests overwhelmed connection capacity, and the system failed again,” per Wu.
At 8:04 am PT “basic functionality was restored,” and Petlibro started restoring other services, Wu said. About an hour later, “the app’s loading time slowed,” so Petlibro decided to “scale back functionality,” the exec said.