Nt38

[Edwin]

As some of you may already be aware, the server went down sometime in the afternoon/evening of December 6th. I came in at 7pm EST and have been tasked with continuing where David left off. Unfortnately I did not get the details as to exactly what happened, as I was more focused on getting it to the datacenter and getting the restoration process started. At this time, the new hardware is already in place and I've been working on restoring all of the accounts and content. This is very lengthy, mostly-automated process and to be honest I do not have an ETA at this time.

Lastly, I want to apologize for not having a forum topic up sooner; this was just brought to my attention as I was under the assumption that a topic had already been created. Sorry!

Update - 12-08-2007 @ 6:35am EST - Hey guys. Thank you so much for being patient with us! I know, the restoration took a lot longer than everyone, including ourselves, had hoped -- Robocopy.exe decided that it would take its time to copy all the content over and we were pretty much stuck waiting for that to finish before being able to continue with the rest of the procedure. David checked regularly throughout the day and night to make sure everything was still running smoothly, and when it and the recreation script were finally finished I was able to run the permissions restoration script and import the SSL certificates. The good news is that as of this update everything should be back to normal, with one key difference -- nt38 is now running Windows 2003 Server.

If any of you experience any problems, such as with permissions, missing a domain or anything that seems odd, do not hesitate to put in a support ticket. Again, thanks for being patient with us, guys!

[soky]

Please keep us posted... I'm getting slammed.

[kattouf]

I know its all automated but i never remember it taking so long on other servers. It's worries me how slow the restore is.

[soky]

okay... now I'm getting nervous.



gulp

[soky]

FYI... I submitted a ticket on the issue and received a reply in less than 3 minutes. I believe they're on it.

Reply:

Hello,

We are finishing up transferring the content over to the new server. We hope to have it fully functional by this afternoon. I apologize for the inconvenience.

[Silverbug]

any update on this?
some of our sites still bring up 'NT38' on a white background

[soky]

Looks like it's back up... but it lost all of my modifications made 24 hours prior. I guess I would rather that then to have to rebuild the entire site.

[kattouf]

Yupp they seem to be back up now. But seriously over 24hrs for a restore is rediculous anyway you look at it. I know it's robocopy's fault but there must be another way. Looking on the bright side... i COULD get a free month of hosting out of this ;-)

[soky]

Originally Posted by kattouf

Yupp they seem to be back up now. But seriously over 24hrs for a restore is rediculous anyway you look at it. I know it's robocopy's fault but there must be another way. Looking on the bright side... i COULD get a free month of hosting out of this ;-)

Yeah, I thought of that too but I always feel a little guilty when 40 some odd other sites were just fine.

[DigitalSkyline]

24+ hours to recover a server... yeah thats about the order of the day around here... should be used to it by now!

(I'm not trying to pile on, it's simply the truth)

[Brangwyn]

So what was the official word on total downtime for the box? I presume like others here we'll be considering requesting credits and need to know what Vortech believes is the downtime in total before all domains were working again.

[kattouf]

I'll forgive vortech if they double my diskspace this christmas ;-)

[MasCon]

what caused the outage ?

we know it was recovered but what caused and what is the preventative measures to prevent it from happening again?