Network issues

[chungmike]

....that's what I tell my customers. Although they're not happy with the downtime, they do understand.

Anyway, Vortech gets things fixed quickly and is good about giving us the status on the forum. I didn't get that with my previous host.

Just tell your customer it's being fixed and will be back on quickly. It looks like it's back up now, so your customers will think you're a hero and hopefully forget about the downtime.

Michael

[Infinitation]

Most users ever online was 702, 02-26-2006 at 06:01 PM.

I think the forum can hold its own. So what was the cause of the traffic gain???? A gigabit of traffic?? Was this an attack how about a little more detail. I have been hung up on, and redirected, and nothing has been answered. We have a right to be upset as this seems to be a recurring way of how these issuse are handeled. For me personally this is the 4th in 2 years and each has been handled the same.

Your customers are our customers and if you know the problem, WE need to know it. Were getting hammered with just as many calls, and its wasting our days as now I have to provide answers I don't have. If you are getting 26,000 phone calls then lets work together as a community to solve these issues. There is no reason for everyone to call if there is a proper procedure to handle these situations.

Soky - Your on the right role with the offline forum and lets continue to get this done!!!

[Craig_Smith]

Originally Posted by harikarum

If that is the case (and it very well may be), why did you not post anything about the outage until about 24 minutes ago?

Because I wasn't here to be honest. I had just arrived when I posted on the forum. In all of the phone calls and confusion noone had thought to post until I arrived, so I did.

[Dean]

ok, I'm currently on the phone with the owners of the IP that were flooding us. They were sending around a gb/s at us and it was saturating our connection and driving the load on the router to 100%. We tried access-listing the offending IP on our end, but the incoming packets/s was simply too high for the router to keep up. I had mail2 (the target of the attack) null routed at cogent to get the rest of the network up. I'm trying to get the company responsible for the IP to stop the flood. Unfortunately, they are apparently one of Quebec's largest residential ISPs and have the worlds smallest abuse department. I'll have mail2 back up asap.

[Infinitation]

Craig - Support, Forums, Mail were all down, ping tested as I tried several times for tickets, posting, and e-mail. The router was a 0 status

[DustinStream]

Craig, I appreciate your efforts to resolve the problem, and your creating this forum to keep us updated. For the amazing value your company provides, I'll happily take your rates, a few headaches and a 99.8% uptime rather than thousands of dollars a month for 99.9%.

Keep up the good work, and I'll leave you alone to do your jobs!

-Dustin

[joshnolan22]

if you where to take a look at the internet traffic report, you will see that a few of the routers in the USA took some major hits this morning. Some in Texas, Florida, Washington and a few other areas. Europe also took some major its. Wonder what it was...

[Infinitation]

Dean - Thanks for the description as thats what I needed to hear! Keep up the good work and looking forward to seeing mail2.

[chungmike]

Originally Posted by Dean

ok, I'm currently on the phone with the owners of the IP that were flooding us. They were sending around a gb/s at us and it was saturating our connection and driving the load on the router to 100%. We tried access-listing the offending IP on our end, but the incoming packets/s was simply too high for the router to keep up. I had mail2 (the target of the attack) null routed at cogent to get the rest of the network up. I'm trying to get the company responsible for the IP to stop the flood. Unfortunately, they are apparently one of Quebec's largest residential ISPs and have the worlds smallest abuse department. I'll have mail2 back up asap.

Dean, thanks for the detailed update. Sounds like spam. Keep up the good work.

[generic]

I am so tired of my $10 a month clients calling me with "MY GOD MY SITE IS DOWN, IM LOSING MONEY"

just tell them as I do, I have bullet proof hosting, but it runs well over $100 a month. Would you like me to sign u up?

If not we are working on our shared server issue and it will be up as soon as possible, I will provide you with a full explanation once its completed.

NO, I cant give you an eta because we are still pinpointing the issue.

[ekrubonline]

Originally Posted by gene0

Guys SERVERS and NETWORKS go down I too am havin a bad day the reason the phone are redirecting you here is so techs can do there job and not answer the phones. but and idea of when thing will be back a 100% would be nice.

I like to be reasonable. I've done support for large networks as well. My reason for dissatisfaction is nto being able to give any of my clients an answer, because everything sends me to a website, and the network is down.

My recommendation is to have a backup recording for the phone system that says that all personnel are being used to deal with a network outage. I mean, really.... what's the sense of having all these things - server status monitor; status thread in the forum; phone system, when all it takes is a VERY perceivable system (network outage) to destroy all of that planning?

I was taught in school that no plan A is complete without a Plan B. I just thought that there'd be a plan B for a network outage like this. Especally because we all know its possible, and they "SERVERS and NETWORKS go down..."

[Craig_Smith]

Originally Posted by Infinitation

Craig - Support, Forums, Mail were all down, ping tested as I tried several times for tickets, posting, and e-mail. The router was a 0 status

Sorry thats not true. I don't want to argue about it, but your wrong. I was answering emails to my wife, tickets and forum during the outage. I even had her logon from home to check the forums. It was up.

[garygpg]

Originally Posted by joshnolan22

if you where to take a look at the internet traffic report, you will see that a few of the routers in the USA took some major hits this morning. Some in Texas, Florida, Washington and a few other areas. Europe also took some major its. Wonder what it was...

Hey Josh - can you post links to some of the sites folks can look at to see that? Would be great reference for all of us to give our customers whatever info we can.

[Dean]

The router had to be reset several times. The resulting route dampening may have caused your provider to temporarily remove the route to our network.

[DVHost]

Whether things appeared up or not depends on where you were connecting from. I have a client in Miami that couldn't do email (prolly mail2) but could do FTP and reach her site, however, her colleagues in Argentina could not reach anything related to her.

However, I could FTP and reach her website but not send email from Kentucky.