01/20/2004: Emergency Maintanence - Router

[joshcombs]

Originally Posted by thecomputerpro

Still have my vote....

And will not be asking for a credit!

TCP

Same here. I will not be asking for a credit. Because it is one of those things. That are not the fault of proper maintance. This was a freak accident....


Thanks again Vortech!

Josh Combs

[admin]

Originally Posted by zennix

Hi, while I really like OpenBSD you are not going to get much use from a dual Xeon with it. OpenBSD does not support dual processor systems. It will work, but the extra cpu is going to be wasted. You can check their support page here:

http://www.openbsd.org/i386.html

Yea we tossed out the other xeon.. But still a 2.4GHz with 533Mhz bus and 1Gig of memroy just like the web servers but with out the 2nd CPU more less. SCSI HD 15k RPM drive and all.. It should make a great router/firewall/QOS it will give us more to work with then what we had with the cisco put it that one plus we will have the backup system and be in a much better shape then with the cisco stuff..

[soky]

Yea we tossed out the other xeon..

I'LL TAKE IT

(SoKy scrounges in the dumpster)

[zennix]

Hmm, care to toss that Xeon my way? ;-) Sounds like a nice little box, it should scream as a router. As a firewall, they are unstoppable. I have replaced a very high end Cisco PIX with Open BSD before. I think there is some work on failover solutions too, you might wanna check around for that info. Which routing software are you going with? Good luck!

[joshcombs]

Originally Posted by soky

I'LL TAKE IT

(SoKy scrounges in the dumpster)

or that could be the next giveaway??

[admin]

Originally Posted by joshcombs

or that could be the next giveaway??

Hmm might be an idea there..

[joshcombs]

Originally Posted by admin

Hmm might be an idea there..

I am trying to post here more so I am eligible lol

[admin]

Originally Posted by zennix

Hmm, care to toss that Xeon my way? ;-) Sounds like a nice little box, it should scream as a router. As a firewall, they are unstoppable. I have replaced a very high end Cisco PIX with Open BSD before. I think there is some work on failover solutions too, you might wanna check around for that info. Which routing software are you going with? Good luck!

PF is doing the routing for just inside our network. The juniper we are using right now does the routing to the "out side world". The router that took a dive today was the core router for the inside network work.. Here is how the network is kind of setup. I will see if i can draw a map with words here..


EPIK/Level(3)/Cogent/Qwest -----> Juniper ---------> cisco switch ------> (Old Cisco 7120) now will be the OpenBSD box ---------------> CoreSwitch cisco 3500 this is the switch that splits up all the racks --------- Patch panel ----------> Cisco 2900's in each rack.


Thats how every thing gets in and has to pass to get to the servers... Juniper full backed by anther junpier for internet routing to providers. Cisco switch we can replace if needed in less then 10 min. The 7120 was backed by the cisco 3500 and should of been able to do every thing the 7120 could. That might of been true a few months a go, but as we found out today don't try and push 20,000+ Ips and 50MB of bandwith in a cisco 3500 switch..

I hope that helps, if any one here knows networking and I think a few of you do I am always open to ideas and will run with them.. I think this openBSD setup will also help a lot of things besides having a real good back plan. We will also now be able to watch bandwidth more better then we could before. Will also be looking at getting snort on the network as well to help us also filter out some bad stuff as well. That will only speed the network up even more.


Thanks every one and again very sorry about the down time and doing everything I can make sure that does not happen again ( at least at this point of the network with out a good backup plan in place. )

[Brangwyn]

Originally Posted by admin

It has not been down for 8 hours stright so just to be nice I just went ahead and oked 15 days. Thing to remember is it must be 8 hours stright. Not 1 hour down, up 4 hours, down 4 hours, up 2 hours, down 6 hours. That would be a total of 11 hours but not stight. I thought that was clear in the SLA maybe not. But in any case it has not been down 8 hours ether. I think it went down around 4:30PM and on and off up till around 8:30PM or 9:30PM can't remember the time off the top of my head. But that would be 4 hours or so. So I am really giving a little more then I should to try and make up or help in this case..

Cheers Brad, that does make it clearer .. though the SLA probably does need adresssing becuase nowhere does it mention any outage must be "continuous".

As it reads currently, if you had 8 single one hour outages then that should be eligible for a months refund, it actually doesn't even say it has to be the same outage either but that will be remedied as well if you simply change the SLA from

CREDITS

In the event of an outage where Vortech, Inc. does not meet 99.9% uptime, Vortech, Inc. will issue credit to the reseller's account only for the monthly service as calculated below. This will be calculated by a 24 hour day in a calendar month, with the maximum credit not to exceed the monthly service charge for the affected month. Credits are only issued to accounts in good financial standing with Vortech Inc.

CREDIT CALCULATION METHOD

One(1) hour is equal to One(1) day of service. For outages of 8 hours or more a maximum credit of one full month of service will be credited to the reseller's account.

To something along the lines of

CREDITS

In the event of any outage where Vortech, Inc. does not meet 99.9% uptime, the reseller will be issued a service credit to their account. This credit will be based upon their monthly Vortech Inc. service charge and will be calculated as follows:

If the continuous duration of any one outage exceeds 8 hours, the service credit will be the equivalent in value to the corresponding calendar months Vortech Inc. service charge to the reseller.

If the continuous duration of any one outage does not exceed 8 hours, the service credit will be the equivalent in value to one days Vortech Inc. service charge for the corresponding month to the reseller for each continuous hour of the outage (refer examples below).

At no time will the service credits paid to any reseller during any one calendar month exceed the total service charges paid to Vortech Inc. by the reseller during the corresponding calendar month.

Credits will only be issued to accounts in good financial standing.

Credits will be applied directly to the resellers Vortech Inc. account(s)

No other form of reimbursement will be considered by Vortech Inc.

That should cover most things I think, examples can stay as they are though might be worth changing the heading in the examples table that says "downtime" to read "continuous downtime"

[Brangwyn]

Actually, I'd just like clarification of one other thing, if you have a server down, and bring it up to see if whatever you've done has worked and it stays up for 5 minutes and then falls over again, that would still be considered continuous wouldn't it ? ... a good example was mssql2 last week, for brief periods it was working but the problem was not resolved becuase it "continued" to fall over leaving users with some rather sporadic periods of availability .... and realistically today you had sporadic periods of availability too which to me would constitute still being part of the same outage and therefore continuous in nature.

Yeah you don't need to say it .. everyone here already knows I'm pedantic

[Brangwyn]

This last little outage, just a 'wee blip' on the radar I hope ?

[robrobinm]

Is someone over there looking at all the issues lately and addressing them. Over the last month I have had the servers my customers are on, reboot in the middle of the day at least 7 times. Now this slow network performance is killing things for me and my business. Just like the other guy said, my customers switched to me because of reliable backbone and servers; Which I have not been able to deliver lately.

Is this just the result of growth for Vortech? Am I the only one starting to get frustrated with this kind of thing?

[gavrielh]

All of our sites on NT26 have slowed to a trickle, some are not working at all as of 11:24PM 1/20/2004 PST. We are wondering what is happening, is this another outage, maintenance, or something else? Will this issue be resolved by morning?

---------------------------------------------------------------------
I posted earlier in regards to the downtime and realize that you are doing the best you can to resolve these issues. We are very happy with your service and your responsiveness in general, but when downtime does occur we get calls from our clients and have to start offering refunds to try and keep them on board - therefore from a business perspective we need to recoup some expenses related to this by requesting a credit. We definately don't want to impose any sort of burden upon Matrix especially during times like this, but unfortunately we are also placed in a situation that requires us to take care of our clients and offer them some sort of relief. We do want to thank your team and support department for all the wonderful work they do and tip our hats off to them!
---------------------------------------------------------------------

[generic]

all my sites, both unix and nt are slow or no go...whats up now...

[tkraffty]

Hi... It's after 1:00AM here (PST)... and going down as far as NT15, web service is still *extremely* slow: Too slow to view any sites and FTP can't move more than one file before timing out.

Yes, I too understand that things break, and I also assume the issues are being worked on by your IT team, but I'll save all my own gushing "matrix rocks" hoo-ha until *after* my sites and clients are running and happy -- which might be tough, considering they've been too slow to access for around 12 hours now...

Frickin' frackin' gripe grumble bitch bitch bitch...