cf.hsphere.cc

[tkraffty]

correct, I just meant "yourusername_<whatever you enter>" in my example, where "yourusername_" is prepended by the control panel to the name you enter automatically. I have no idea if dashes or other characters are allowed AFTER that - I was just referring to the prefix and underscore added by the control panel

[Ezdno]

The username_ bit is prepended by the control panel though isn't it ? so what your entering into the screen is the bit that will follow that in the final DSN name, quite likely why it won't allow an underscore in it.

Yes, I was trying to add a custom MX DSN via the Domain Info settings, which would not let me install the _underscore.

Whats missing in that list of items in creating a DSN is that fact that I needed to add the mySql driver to bring me to the creation page. I have started working on the recreation of the DSN's but so far nothing is working.
I agree this has been a frustrating day our business has taken a major hit, hopefully we will recover. We have had to field calls from 8am to present to answer why our customers websites are not functioning. We don't have 30 datasources yet but we have so many sites that run off the ones we do have it has nearly crippled us.
I do understand you folks have been working hard to correct the problem, but that doesnt change the fact that its going to cost our company clients and ultimately $$.

[jgouger]

Any new updates on the server?

[nabsUK]

Any updates on this? We have a few CF sites that are refusing connections completely.

[Ram_Argid]

Originally Posted by Two Roads Media

Our clients alone represent more than 30 datasources, which is bad enough, but because the naming conventions on them are NOW set by the control panel (i.e. username_dbname) - something we cannot control - many of our sites had significantly broken code due to the changed datasources.

Thus the need for standards and quality written code. We too have over 30 datasources, and over 30 sites. Some with a few pages, and some with thousands. We go in and set the DSN name with one line of code in the application.cfm file. It's called: Planning. This type of planning is very much in our control. Never, for even the smallest site, would we have to go in and change a DSN or any global variable for that matter, in more than one place.

Why would someone that does more than 80% of their development in CF code not practice something as fundamental as this?

[nsdesign]

The couple of client sites that we have that use the Cold Fusion server are now giving DNS errors - so I cannot even connect to the websites to see how badly they are affected!!..

[Dean]

We are continuing to work on this server. At this point, some sites are working but most are not. We will let you know as soon as everything is up.

[Clutch]

Originally Posted by Ram_Argid

Thus the need for standards and quality written code. We too have over 30 datasources, and over 30 sites. Some with a few pages, and some with thousands. We go in and set the DSN name with one line of code in the application.cfm file. It's called: Planning. This type of planning is very much in our control. Never, for even the smallest site, would we have to go in and change a DSN or any global variable for that matter, in more than one place.

Why would someone that does more than 80% of their development in CF code not practice something as fundamental as this?

LOL, "planning". Of course we have global variables set for datasources, Ben Forta, but there shouldn't have been any need to touch any live code or change datasources at all last night.

Explain to me why a server that Vortech is responsible for goes down, and none of the data is backed up? That it is now the responsibility of the paying users of that box to recreate their previously existing data from scratch? I don't understand it. It doesn't even seem to be the policy of Vortech, except with regards to ColdFusion.

Since I've been with Vortech, it has been requested of me to send in all of my ColdFusion datasource requests to support. They take care of them in timely fashion. Last night, the email said to update them in H-Sphere.... well, of course they never were in H-Sphere. Furthermore, H-Sphere has a strict underscore naming convention that essentially meant none of our datasource names (with dashes) could be kept. This meant gather all datasource information, re-create all datasources with different names, change all global datasource variables in dev/staging/production and test/confirm that the new datasources, created in H-Sphere, were working. For some clients who have multiple datasources, we can't confirm the datasource without taking an order all the way to fruition.

Of course, the sites were down after the 9:30pm CF reboot, so there was no way of knowing if the sites were communicating properly with the databases or not.

So yeah, 3 hours were spent last night on this mess, but no Einstein... it wasn't because we changed a datasource name on every cfquery. And the sites were all down -- whether by broken DSNs or server/CF problems -- while it was our responsibility to make these fixes.

I'd like to add that this isn't the first time that Vortech has treated the ColdFusion customer like a third-rate user. Tops still has to be when the "ColdFusion box" idea came about and Vortech wanted all resellers to be responsible for re-creating all their sites and emails. Lovely.

In closing, let me say I'm speaking out here already knowing that Vortech will treat this like some rare exception, or some lazy customer. Last night the response to me was "ColdFusion sucks" and this should take me just "10 to 20 minutes" to fix. NEVER should it be our responsibility to get our customers back up because of a Vortech server screwup. If I had been out of town, all of our customers would be sitting there with dead sites, wiped out datasources and Vortech would be happily collecting money from me thinking everything was just fine. Downtime is a problem, but it isn't the problem ... not taking responsibility is the problem.

[nabsUK]

Well said.

[tkraffty]

Is there a reboot schedule for today? It turns out that my changes to one EXISTING data-source from the control panel (which was there before all this happened), haven't taken hold... and I'm getting a weird CF error from it, which encapsulates the username/IP in HTML apostrophes...

eg., &apos;USERNAME_admin@<SOME IP ADDRESS>&apos;

So I deleted that previous datasource and have just created a new one to replace it...

EDITED 11:52am (ET): Ah! Looks like my last datasource changes took hold and we're back in business again! yay!

Also, as for the frustration in some of the comments here, all I can say is the same thing I generally tell my customers. This sort of outage almost never happens here, but they can happen *anywhere*. I can only recall two other times in as many years here that we had outages like this (one was a Windows box, and the other had to with the network router gear). In both cases, the issue was resolved as quickly as possible (and unfortunately, sometimes it does take a day or more to fix something) and the data-center made sure to update their internal processes where necessary to help prevent the issue from happening a second time. In this case, I'm guessing the techs expected the software repair to take far less time than it did, and they chose the repair option over rebuilding the machine from backups most likely to avoid losing atleast a day of customer site updates. By the time they realized the repair approach wasn't quite working, the end result would have been the same: The sites either lose a day's updates or lose a day's traffic... Either way - it's been my experience that these systems still manage to maintain a better uptime record for me and my customers than even Amazon or eBay can boast... although, admittedly, this doesn't make anyone feel any better when it's *their* site. I might lose my customer on this box, but they risk the same issues anywhere else. So, lightning's already struck this spot, and I'm thinking it safer to stand right here, than go find another tree...

TK

[Ezdno]

I could not have said it better! When the President of our company gets back into town. I can already see the mad dash to find another host!

I will add that after hours of work, our sites are back up and running, but I hate it when I have to work from home to fix something that I didn't break!

[Ram_Argid]

Originally Posted by Clutch

Explain to me why a server that Vortech is responsible for goes down, and none of the data is backed up? That it is now the responsibility of the paying users of that box to recreate their previously existing data from scratch? I don't understand it. It doesn't even seem to be the policy of Vortech, except with regards to ColdFusion.

Since I've been with Vortech, it has been requested of me to send in all of my ColdFusion datasource requests to support. They take care of them in timely fashion. Last night, the email said to update them in H-Sphere.... well, of course they never were in H-Sphere. Furthermore, H-Sphere has a strict underscore naming convention that essentially meant none of our datasource names (with dashes) could be kept. This meant gather all datasource information, re-create all datasources with different names, change all global datasource variables in dev/staging/production and test/confirm that the new datasources, created in H-Sphere, were working. For some clients who have multiple datasources, we can't confirm the datasource without taking an order all the way to fruition.

Well without resorting to the typically juvenile sport of name calling:

Apparently you are associated with Two Roads Media. I wasn't commenting on any of the issues that you brought up, I was commenting on this: many of our sites had significantly broken code due to the changed datasources

Significantly broken code tells me that you've got problems with hard coded DSN's. If that's not the case, then explain it like it is and don't make it sound even worse, apparently for dramatic effect.

We all have been impacted by this, not just you. Have you ever administered a CF box? If you have, then you know that CFMX is nowhere near as straight forward as CF5. If not, then perhaps you should. There are plenty of solutions out there where you can control your own box. Try it for a while.

This hosting company is still the cheapest with the most features out there, especially when it comes to CF hosting. If you know of any better, why are you hosting here?

The thing I don't like is having to wait for two hours between attempts. That got old 10 hours ago. Is the server even rebooting on schedule anymore? I think you need to up the reboot schedule to at least 1/2 hour. 2 hours is too long for this type of trouble-shooting.

[Ezdno]

Your right Tkrafty...and I do appreciate the efforts that each person at vortech put in, I know the pressure had to be magnified by the number of customers they have...

On behalf of me....Thank you

[admin]

Clutch,

Let me see if I can lay this out for you. It seems you have never been on the server side of things with CF. I know for a fact you had problems managing a simple server before. So I will try and put this in simple terms as to what happened.

The data source file CF uses has issues from time to time in CF MX, it's a well known issue and there is ONLY 2 fixes and they don't always work. 1. replace the file with the .bak file 2. reinstall CF. Thats the 2 options there is no other option given by macromedia or any one else for that matter. Search google if you don't trust what I am saying. This has happend to a LOT of people.

Before macromedia got ahold of CF it used the ODBC built right in to Windows and worked GREAT, never a issue. Once they changed that in CF MX there started to be issues, they use one file and everything is on one line. VERY VERY hard to work with and read. It's in XML and the file was over 1MB in size. Since it was so big it had no formating and there was no simple way to edit this file. We could not import it to an XML editor and change it because when it saved the file it formated it all nice and not all on one line and CF did not like that and would not work. Macromedia had nothing to give us to edit this file.

So we went out on a search to find this after downloading over 30 XML editers from xml.com we found one and only one that worked. This was at 12:30am last night. By this time we had sent out the email telling customers to turn off CF DSN's and turn them back on. While this was great and would have worked we ran in to another issue. Everyone that had not made this change had no working DSN, so everyone coming to there sites were making errors on the server causing CF/java to kill its self.

So we took the old file that would not work if we just placed it in the lib dir for CF to use and took the first 5 DSN's and put them in a blank neo-quary.xml file and they should up and seemed to work. So we did 10 more, then 10 more and kept going untill we found the one causing the issue. It was right in the middle of the file. What makes this worse is it add's them one after the other one and this one looked to have been in there for over 3 months. Macromedia also could not answer this question of how it worked and then it just stopped and something in the middle of the file made it stop working.

Now that we know how to fix it "next" time if the same thing happens again then we would at least have an idea where to start. Macromedia and google were no help other than the things I gave above and they did not work in our case.

As much as I like selling Coldfusion as it's a great service add on, its almost has too many problems in CF MX. We are having a meeting tomorrow and are looking at 3 options:

1. Get rid of CF all together. There is just too many issues with the MX version.
2. Downgrade back to CF 5, that was supper stable and never had issues.
3. Keep going with CF and fight it when it has issues.

I really don't like offering a service like CF that has some many issues and support from the maker of the program is so lacking that they don't even have an easy way to edit there own files, because they put a 1MB file all on one line, thats over 60,000 lines of code to look over and we needed to do it fast. Give that a try and let me know how easy that is.

Let me know what you guys think. I am sorry about the downtime and doing everything we can to get your sites back up ASAP. Most are up now as we are not seeing many errors in the logs now.

[jkoch]

I understand there are issues to the coldfusion, but downgrading at this point or killing the service would be a huge problem. FOr me I have modifiied my code to work with CF MX and changing back to CF 5 would be a real pain. Of couse ending CF support would be even worst and force me to look for another provider. CF has been pretty reliable from my perspective until now, so I am hoping to at least keep what we have.