Two weeks ago, Bungie brought Destiny 2 down for emergency maintenance and did its first-ever rollback of character data because of a bug that deleted players' materials and currency. Yesterday, with the release of an update for Destiny 2's Valentine's Day event, the bug reemerged and forced another emergency shutdown. Now, in the wake of a second rollback, Bungie's offering a thorough and candid explanation as to what went wrong.
The previous emergency downtime period, which happened on Destiny 2's last weekly reset in January, lasted for about 9 hours. Once resolved, player's inventories were restored back to a state logged a few hours before the hotfix was pushed. When a handful players logged in yesterday to discover that their Glimmer and other materials had once again vanished, Bungie quickly moved to bring down the servers and had Destiny 2 back up 6 hours later.
In a new post credited to the Destiny Dev Team, Bungie confirms that the same bug from two weeks ago made a comeback. "Since both of these incidents are identical in cause and the effect on our players, and because both incidents happened within a close window of time, we wanted to give you a picture of what went wrong, how we fixed it, and how we're planning on making sure this doesn't happen again in the future," Bungie says.
The currency and material deletion bug, Bungie explains, was an unintended result of a fix for Destiny 2's quest log that slipped through testing. With the downtime and rollback two weeks ago, Bungie identified and patched the bug in question. Unfortunately, though, a small percentage of Destiny 2's servers crashed ahead of yesterday's update and, when rebooted, started without the previous bugfix in place.
If one of Bungie's tester accounts had been routed through one of the few servers in question, the bug's return would've been caught ahead of time. In this instance, Bungie wasn't so fortunate: "Because we have hundreds of servers in our retail environment, every manual test we performed was (un)lucky enough to hit the 'good' servers, and all of them missed the small percentage of servers that were in a bad state."
The unusually detailed response has gone over fairly well in the Destiny community. On Reddit's r/DestinyTheGame, the top upvoted comments on Bungie's explanation are mostly appreciative of Bungie's transparency. "As a sys admin, this is by far one of the most well written, professional community posts I've ever seen," writes one player. "Well done. I can only imagine what the team(s) involved with this have gone and are going through."
In addition to changing its testing and backup plans for the future, Bungie says it has already fixed the issue that allowed servers to start with a previous version of Destiny 2 and promises that a permanent fix for the missing currency and materials issue (as well as the infinite Dawnblade glitch wreaking havoc in PvP) will arrive with the next update. Currently, players can log in to Destiny 2 to enjoy the limited-time Crimson Days event—and, assuming they never played in the two brief windows that Bungie needed to rollback progress for, it should feel as if the bug never happened.