NT17 status

[JoshK]

NT17 was not performing well, I have taken it off-line to assess why it is unstable. I hope to get it up and operational before the start of business Monday morning. The data drive is intact so do not be alarmed, sites and data have NOT been lost they are just temporarily off-line while I check the machine out. I will post an update when I find out the root cause of the instability.

[zubin]

Josh, it IS business hours for some of us!! I'v said this before, why can't such maintenaces be done on sat / sunday?

[Brangwyn]

9 hours on, is there any update on the server?

[JoshK]

update:

This wasn't maintenance. I couldn't keep the machine up for very long at all. It kept locking up and restarting. I didn't want to risk further damage and possible data loss or even more extended down time. If you look back in the past, we have had machines fail and be out of service for considerable lengths. I'm trying to catch them before such failures. I believe a few hours, even if it becomes 12, is better than 2 days. A set of fans died in it, overheating the machine, this was no minor problem I could put off. Swapped in practically all new hardware, data (web content) was fully intact. Edwin and I also upgraded it to 2003 in the process and installed the new asp files into it. So when this is done, it will support asp 2.0, ajax, and 3.5. I know sometimes this happens at inconvenient times, but I do try to get things up as fast as possible. It will take a little time for the IIS settings to come back online, so sites will become available again over the next few hours. SSL's unfortunately will have to be manually re-installed as it was not possible to recover the needed information from the old system drive to accurately re-install them. If you have an SSL on that server, you may need to contact support.

[jmbeach]

Originally Posted by JoshK

update:
I believe a few hours, even if it becomes 12, is better than 2 days.

Surely looks like it's coming up on that 2 days, doesn't it? Luckily I don't have any sites on NT17, but as an old-timer at Vortech, I remember back when in the SLA there was a suggestion that a backup server was always available, and with a simple swap of a hard drive a dead server could be back up within minutes... funny, I think this was long ago taken down because as we see that is a lofty goal, but sometimes not too realistic.

Once this is all over, I think it would be a fair request of resellers that Vortech break down for us how long certain steps took.

For example, how long did it take to build a new server? How long to reinstall Windows and all the other software needed? How long to restore user data? How long to set permissions? How long to restore any HSphere data?

In the end it would seem to me that Building a server and installing all requisite software should be done as a preventative item. You should never be building a server from scratch only after one goes down... now, I don't know if that's what happened here, but I think it's fair to ask. Restoring data and settings takes as long as it takes, but it's been what... 40 hours already? It sounds like it's still not fully recovered. That's just not acceptable on any level. At least not in my opinion.

But I digress. Let's have an open discussion into what happened and how it was handled, and let's for once and all find a solution to minimize these in the future. Or am I just being naive?

[Brangwyn]

Thanks for the update Josh.

[rheide]

Can we get another update? I just got an email that said the server was still being worked on so I'm not sure if Josh's comment is still valid or not.

Thanks!

[Brangwyn]

That probably refers to the fact the server is still rebuilding IIS Metabase/Permissions etc, this typically takes severals hours to complete.

[rheide]

6 hours since the last update ... anyone have any input they can share?

update: here is the update I received from Dan:

Right now we are recreating IIS site configurations.

[rheide]

Any time estimates?

[3WHosts]

Please can we have an update. An eta would be really useful please.

[rheide]

This is crazy. Why is it soo difficult to keep us posted as to what is going on?

[JoshK]

Sorry for the lack of updates everyone, unfortunately there is not much I can give you for ETA's. It does take quite a while for our scripts to restore all of the permissions and IIS settings. There is no logical easy to follow order where I can easily say (we're up to users starting with "N") like there is with file content backups which are done alphabetically. We are making good progress, and hope to have it fully functional tonight. I cannot promise when, But I will keep you posted if there are any changes. If you have any domains with SSL 's on that server, it will help speed things up if you hunt down your copy of them now so that when the time comes you're ready to restore them, as they will have to be manually entered (we will do what we can to help as needed in that process too)

[zubin]

any update?

[Brangwyn]

Last I heard from Josh it was still doing it's thing no eta available but said he would post here the minute it was finished.