A real backup system

[powvex]

As far as you all know, nt33 has been dead twise yesterday with over 18 hours downtime.
this incident has been hit many here like me, and this is not the first time here in the past years. I lost some customers each time, as I'm sure everyone will.
We know hdd fail, software issues is not related to anybody or anytime, and they happen time to time so we cant prevent them. but what I think we can prevent from happening, is downtime, and at least increasing the downtime to less than an hour, using more efficient backup systems.
We at our hosting company did this by try and error on almost 10 backup system on our dedi. servers in past few years, and what we have came across to be the best way is something like Norton Ghost. it completely backup our 40GB ide hdd in 1.5 Hour, and its not depend on file count. and normally restore all the things was on server in very much faster than normal operations. We use ghost in backup drive on system and it also can compress data backups too.
I dont say that vortech should use ghost, all I'm telling is that vortech should use a more efficient way of backup and restore to prevent these kind of heavy downtimes.
I hope everyone could suggest a way, and it help to find a better way here in vortech.

[Dean]

We already run LiveState, which is essentially ghosting a live drive. Running standard ghost isn't practical as the server has to be taken offline. The downside to LiveState is that its resource consumption during the backups tends to be very large.

[admin]

powvex, way ahead of you man. This has been in place for about 2 months now, this is why we took all the windows servers down one at a time was to add the ghost drives.

We tried to use the ghost image yesterday but when we did the drive died on us, so we switched to our old backup system. The drive with the ghost image was fine it was the drive in the system that died the D drive. We did not want to chance another ghost image so we switched to our old system.

We have 2 systems in place and here is how it works.

1. image drive to the local Z drive every 24 hours.
2. Backup systems copy image to the there loacl raid 5 setup.
3. Once a week we run our old system with robocopy to copy all files as a fall back to the new system using ghost. ( this is what saved us ysterday but is a bit slower )


The problems with ghost is that it is many 2GB files and hogs a lot of CPU time, we are working to find ways to get around the CPU issues. The real problem is if the box gets rebooted when it's doing a mirror because it's using to much CPU is we don't have a mirror then, it will not send us a warning or anything that it was unable to finsh. This we are working to fix as well with some audit stuff we are trying to do on the backend.

Trust me we are doing everything we can to better our backup system(s) we also just added another 5TB of backup space to our already 5 TB so now we are right around 10 TB total backup space.

[powvex]

Brad; thanks for explaining, yes ghost use huge cpu timings and should be done at low traffic times. but its really fast that backup without compression by rate of 1GB/minute.
drive die is not software issue and is not something that caused by ghost.
all I'm looking is to reduce downtime, as I think 6 hours or more is not an acceptable downtime for any server in industry. please correct me if i'm wrong.

[admin]

Your not wrong 6 hours plus sucks.. We are doing everything we can to speed it up, one of the reasons we started using ghost problem is we have not had a system die where we could fully use ghost yet.

[Dean]

Actually, there was one, but no one noticed it because the downtime was so small...14 minutes if I recall correctly.

[powvex]

so it works as i mentioned Dean, but why hdds will die like these incidents recently? do you use one brand for all servers? did you specified any same problem with all the hdd's that died/crashed?

[bobi]

What would Vortech do, to it's resellers who have lost their repution for the down time!!!

Why can't you give us an estimate when the Host is down!

[joshcombs]

because if they gave everyone ETA's then everyone would complain about them no being on time. So to cover their own they do not offer ETA's. Just know that they are working as hard as they can to fix the isssue.