Any news on Curse (the website)?

[Invisusira]

The entire Curse network (aka almost every WoW, Minecraft, etc website because they friggin own everything) has been down for a while now, as I'm sure most of you are aware. Does anyone actually know what's going on? At this point I'm guessing they got hacked?

[KysenMurrin]

There's been talk in the Frustrations thread about the FBI seizing hardware at the hosting company, I think. I'm not sure what that's all about.

[RedAces]

CurseForge is down for emergency maintenance, we apologize for the inconvenience.

Update 18:30 UTC / 1:30 pm CST: Faulty hardware has been replaced, we are in the process of verifying data and expect to be back online in the next 60 minutes.
Update 19:40 UTC / 2:30 pm CST: The data verification is taking longer than expected. We have no new ETA at this time.

Update 23:45 UTC / 6:45 pm CST: Unfortunately, the data verification and restoration process is taking much longer than initially expected. Our current estimate is 6 hours to complete the process.

Update 06:30 UTC / 1:30 am CST: Data verification is 55% complete. As we cannot accurately estimate the time it will take to complete, we cannot give any further time estimates until this process is complete.

[lythac]

Not sure if it has any relevance -

Forum Software Update - We updated the site to the latest version of vBulletin and you might have noticed that the text editor changed. If you want to revert back to the old editor go at the bottom of your profile settings and select the Basic or Standard editor.

[Zalaria]

There's been talk in the Frustrations thread about the FBI seizing hardware at the hosting company, I think. I'm not sure what that's all about.

I think MMO-C was still up after that.

[halabar]

Not sure if it has any relevance -

Forum Software Update - We updated the site to the latest version of vBulletin and you might have noticed that the text editor changed. If you want to revert back to the old editor go at the bottom of your profile settings and select the Basic or Standard editor.

Naah.. it was up after that, and wouldn't have taken down the rest of their network.

If they weren't part of the FBI raid, then they must have been using some type of private cloud for their database hosting, without a redundant backup system in place. That's the only reason their whole network would be down and having to be verified/restored.

Either they were sloppy or simply couldn't afford that level of redundancy. It's not cheap.

[sahiel]

Boub mentioned that 'a SAN controller exploded' on his facebook, seems like they're still doing data verification atm.

[fuzzygeek]

Ouch. This is why I keep my data where I can physically reach it (with appropriate backups locally and in the cloud) instead of using the cloud as my primary means of access.

I'm working on an application that'll be hosted on Amazon's AWS platform, which seems like it'll have plenty of redundant options (multiple images working on multiple backend images), but they were still plagued by an outage a month or so ago due to a faulty fallover config ...

So, are things getting better or worse?

I'd probably still argue better, but with new, exciting problems ...

[halabar]

If a SAN controller was the root cause of their failure, then then have been skating on thin ice for a while. The databases should have a fallback clone that could be stood up if something blows up (physical or cloud works, you just need those multiple images) and the same thing goes for the servers.

I hope the admin has that "get out of jail" email in his pocket where he tells management they really should have another layer of redundancy.

[sahiel]

Apparently they were due to replace it in two weeks, and the backup also 'exploded' whatever that actually means in this case, physical or software. Last Boub posted they had multiple people at the server place working to fix it but still no estimated time for completion.

[RedAces]

Update 20:45 UTC / 3:45 pm CDT: Data verification is still underway, but progressing very slowly. We will post another update once we get close to the end of this step of the process.

[Klaudandus]

Next thing you know, someone is gonna mess up with their database =P

[RedAces]

Absolutely not a hack or a virus, just the SAN controller going herp derp and the backup controller doing the same.

Source: http://twitter.com/#!/Boubouille_MMO

[sahiel]

It's mostly back up now, main site is there just missing the last 8 days forums and such, which they'll restore as soon as they get a final software fix from HP I think it was.