Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
Sign in to follow this  
Manc

MancuNET go boom, will be fixed within 48-72 hours

Recommended Posts

Primary hard disk failure on MancuNET. They're going to replace it within 24 hours, but things may be unavailable for the next few days as I will have to go through and set everything up again. Thankfully there's no charge for this (except for my time and frustration) and ServerBeach are quick and professional (as opposed to dreamhost who are a joke and terrible).

Will keep this thread updated with any new information. Yes data are backed up.

[UPDATE 07/08/09 3:11PM]
The drive is being replaced during a 4 hour window of 8pm-11:59pm CDT (so starting within about 5 hours). Starting tomorrow I will be going through the process of setting back up all the crap. Thankfully I got this thing with a spare drive in it I was not yet using, so I partitioned, formatted and now I have a spare drive to copy things to. Makes backing up much faster!

[UPDATE 07/08/09 5:35PM]
Attempting to do mysqldump does not fully dump before throwing itself into read-only mode (from warm boot). I've backed up the actual database files and dirs themselves, which should restore fine. I used the packaged version (lenny) of mysql 5 and I will be reinstalling the same version most likely. Nothing too critical will be lost with these dbs but I am doing my best to ensure they're still alive. ServerBeach continues to be fantastic in their support.

I will not be near a computer until tomorrow morning. When I return to a computer I should have waiting for me a freshly provisioned server. Thankfully lessons were learned from the errors I was having before in my initial setup so things should go more smoothly this time. The biggest issue tends to be permissions related and besides lessons learned, everything has been backed up with that information intact.

[UPDATE 07/09/09 10:08AM]
The server is back online, but it only has the new drive, not my second backup drive. They were testing to see if I could log in before putting in my other drive. However, their language seemed to indicate that they'd be putting in my original disk back in so I could copy off of it. EDIT: Spoke with them on the phone, they did indeed mean putting back in my other healthy drive. It's happening as I type this.

[UPDATE 07/09/09 10:55AM]
All drives are in, fresh /dev/sda, backup /dev/sdb and fartknocker brokendrive on /dev/sdc. Will be doing software rebuild over the course of the afternoon/evening starting in a few hours. Most if not all services will be restored in the next 12-24 hours. Thank you everyone for your continued patience and support.

[UPDATE 07/10/09 8:26AM]
Most everything is back to normal. Email is not set up yet but that is a separate process. All accounts should be restored, passwords are the same, all subversion repositories are restored and websites are restored. Please note: things may still be a little intermittent as I stop/restart different services during the remainder of the setup process.

If you are finding that something is missing, I will still be checking these threads regularly. You can also send an email to my offsite email address (fuzztooth@gmail.com).

Share this post


Link to post

Don't most large scale servers have hot-swappable drive arrays with redundancy? Or are you just using them as a colocation facility for your own server?

Share this post


Link to post

Aha! That would explain why I've not been able to access it. Thanks for letting people know.

Share this post


Link to post

Wait, what? I just bought AD2:UB last night...

I hope this doesn't impact the arrival of my order of pure awesome in a can of a box sewn and stitched by blind elderly orphans who also love awesome!

Share this post


Link to post

I hope that drive failure wasn't the result of having too many fish avatars on the one page.

Share this post


Link to post

I saw it start happening around 6:35AM CST when I tried to approve members of my board, then KAPOW sessions table corrupt, and BAM can't go into webadmin, and DOOSH 404 forever.

Share this post


Link to post
exp(x) said:

Don't most large scale servers have hot-swappable drive arrays with redundancy? Or are you just using them as a colocation facility for your own server?

Yes and if I had a large scale server setup I might have that. But that costs money, and my setup is one machine just for me. I have two drives but they're not set up for any redundancy at this time.

Share this post


Link to post

All drives are in, fresh /dev/sda, backup /dev/sdb and fartknocker brokendrive on /dev/sdc. Will be doing software rebuild over the course of the afternoon/evening starting in a few hours. Most if not all services will be restored in the next 12-24 hours. Thank you everyone for your continued patience and support.

Share this post


Link to post

Most everything is back to normal. Email is not set up yet but that is a separate process. All accounts should be restored, passwords are the same, all subversion repositories are restored and websites are restored. Please note: things may still be a little intermittent as I stop/restart different services during the remainder of the setup process.

If you are finding that something is missing, I will still be checking these threads regularly. You can also send an email to my offsite email address (fuzztooth@gmail.com).

GreyGhost said:

I hope that drive failure wasn't the result of having too many fish avatars on the one page.

More like there's not enough! Now that it's back up I can finally update the gallery!

Scuba Steve said:

DOWN TIME IS LOST URBAN BRAWL PROFITS!

There's profits??

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  
×