You are here: Journal
   
Get syndicated feeds from our Journal!  Add to Technorati Favorites

Vertical Banner 1

   Minimize

Vertical Banner 2

   Minimize

Enter Title

   Minimize

The Dead Hand Journal

Journal

   Minimize
23

Okay, let's be fair: shit happens.

Some time this morning, a nameless registrar disabled the domain name mysite4now.com. As it happens, this is the domain used by mega-host WebHost4Life to abstract new sites awaiting domain assignment as well as—oh, so much worse—all of their database IPs.

Instantly, every data-driven site hosted by WebHost4Life went down hard, including this one. The immediate fix was easy—alter all your config files to reference the database server by IP rather than FQDN—but first you had to get that info, and you can imagine their call volume was rather high.

Meanwhile, though, TDH and related sites were already dragging. Page loads were pushing into the minutes, and none of the usual suspects—cache settings and the like—seemed to be making a difference. We're already spending a mint on RAM allocation, so it wasn't obvious what the problem was.

Right about the time I got the database back online, though, I found this little nugget. It turns out—for no good reason that I can think of—that too many event log entries make DNN run slow. I mean, really slow. The answer is one little line of T-SQL: delete from eventlog.

This is where the user error comes in. See, in a shared hosting environment—and, despite the fact that my web server is a Virtual Private Server, my database server is absolutely shared—actions that eat up too many resources set off all kinds of alarms. And nothing eats up resources like issuing a command to delete about 32 thousand table rows.

My host yanked my database offline before I could even catch my breath.

What was the right answer? Easy, in hindsight:

  1. Script the table as CREATE.
  2. DROP the table.
  3. Execute the CREATE script.

That's it: in & out in 30 seconds. As with most things, it's always easy once you already know how.

And once a database is offline, it takes an act of G-d (or at least a System Administrator) to get it back online again. Now, understand that I'd already been through the wringer by this point, what with my slow-as-hell site dropping completely off the map for several hours. So I felt a little testy. Still... you could insert the following support chat transcript into a Saturday Night Live script and nobody would know the difference:

Powered by SightMax
Welcome to sightmax2.webhost4life.com.

support3
will be right with you.

support3
Hello how can I assist you?

Jason
Your company is killing me today.

Jason
My database just tripped offline.

Jason
Trouble ticket 304392

support3
One moment please let me check

Jason
Between this and the routing issues from this morning, I may lose half my clients.

support3
Dear customer, it seems that there is a routing issue on the network that make the connection for your site can't connect to the database. Our technician are working on the issue now. Sorry for any inconveniences caused. If you want a quick fix, please change the connection string to IP first. If you would like to know your database IP, please do "ping sqlxxx.mysite4now.com"(for MSSQL) or "ping serverxxx.mysite4now.com"(For MySQL) to obtain it. Thank you.

Jason
Is this what I pay over $200 a month for... crappy uptime and a phone that nobody answers?

Jason
NO, damn it!!!

Jason
Read what I'm saying.

Jason
I know about the routing issues.

Jason
My database is O F F L I N E

support3
ok

Jason
It is database "hotquant"

Jason
Please place it ONline before my clients jump out the ^%*&^*( window!!!

support3
one moment please let me check your ticket no 304392

support3
I have checked your ticket no 304392 is update our admins can you please update our admins

Jason
What do you mean?

support3
our admins will update your issue very soon

Jason
Very soon as in how many seconds?

support3
i have checked your ticket no 304392

Jason
Do you understand what you people have done to me today?

support3
can you please place a support ticket for our admins

support3
because of your issue is required the server access

support3
i am not having any server access

Jason
OK, what do you need... the IP address, the password, what?

support3
can you please call on this no so

Jason
???

support3
(Tel) 888.306.3780 (Tel) 323.980.8281

Jason
How long will I be on hold?

Jason
LOOK... I SPENT 30 MINUTES ON HOLD ON THIS DAMNED PHONE LINE!!!

support3
can you please try after some time

Jason
MY SITES ARE DOWN!!!

Jason
Are you telling me I have to wait?

support3
please call after some time

Jason
There is already a trouble ticket in the system, ID 304392. My DB user name & password are on the ticket. WHAT OTHER INFO DO YOU NEED???

Jason
This is URGENT, do you understand???

Jason
Hello???

Jason
I am going to e-mail the transcript of this chat to your CEO via the link at the help desk.

support3
we are really sorry for that please call after some time or place a support ticket for our admins

Jason
THERE IS ALREADY A DAMNED SUPPORT TICKET!!!

Jason
Look... what I need you to do is to bring that support ticket to the attention of whomever is in charge.

Jason
Bringing a DB online is a 30-sec fix, but I can't do it remotely.

support3
can you please check ticket update given by our admins

Jason
I have responded.

support3
ok

Jason
Would you please alert them dso they can check the response ASAP?

support3
Our admins will update you very soon

support3
please check after some time

Jason
Look, SUPPORT3... I pay well over $200/month for your Advance VPS service. I don't WANT to "check after some time." I want YOU to stay with me, pick up the phone, and EXPEDITE this process. Do you understand me?

Jason
If you can't do that, then have somebody who can either get on this chat line or call me direct at 773-xxx-xxxx.

Jason
You can explain to him that your company is on the very edge of losing a client and winning some VERY bad press.

Jason
Have I made myself clear?

support3
ok

support3
Is there any thing else I can assist you with

Jason
The ONLY thing you can assist me with is this issue. I want you to stay with me until it is complete. I don't want to have to wait for some sysadmin to check his e-mail... I want YOU to expedite this process. Okay?

support3
sorry you have to check after 30 to 40 mins

Jason
Do you have a supervisor?

support3
can you please call on this no >>

support3
(Tel) 888.306.3780 (Tel) 323.980.8281

Jason
SUPPORT3, that line is completely tied up. YOU are trying to make me go away. I won't do that.

Jason
Now answer my question: do you have a SUPERVISOR?

support3
no

Jason
Great. Do you actually think that you won't be identifiable by the time code and your support ID?

Jason
This transcript is going to your company's CEO. Perhaps you should think about that.

Jason
Now... do you have a supervisor?

Jason
Is there ANYBODY there who will pick up a phone and CALL me?

support3
Yes please try after some time

support3
the phone support dept will ans your call

Jason
Look... you obviously are able to communicate with the admins, right?

support3
and the supervisor are not available right now

Jason
It's the supervisor's JOB to be available!!!

support3
no they are not available

support3
can you please call us

support3
supervisor are not available on chats

Jason
Look, damn it... I'VE BEEN ON HOLD EVER SINCE WE STARTED THIS CONVERSATION!!!

Jason
The phone is not working.

support3
Can you please turn the caps off

Jason
Meanwhile, I am talking with you.

Jason
Now, would you please be good enough to confact the admins and find out WHAT the status is, WHAT they are doing for me, and WHEN they will be finished?

support3
supervisor are not available on chats you can mail then at ceo@webhost4life.com

support3
And they will reply you on mail

support3
so please contact them

support3
at mail

Jason
Do you realize that, all this time you've been trying to get rid of me, you could have been helping me?

Jason
Why won't you contact the sysadmins just like you did before and exedite this for me?

support3
i have update your ticket no 304392 to our admins they will update you as soon as your issue solved

Jason
I just looked at the ticket and it shows no update.

Jason
Did you TALK to an admin?

Jason
What is the status?

support3
i have update your ticket no 304392 to our admins they will update you as soon as your issue solved

Jason
What does that mean? Does it mean you have moved me higher in the queue?

support3
yes

Jason
So when will it be picked up?

support3
it will be done with in 30 - 45 minutes

Jason
That's exactly what you told me 20 minutes ago.

Jason
You haven't really done anything for me, have you?

Jason
Have you?

support3
we are sorry for that please check after some time

Jason
Fine. I have somebody on the phone now. Good luck riding this one out.

You know what happens next. Here's the form letter I received 10 minutes after I e-mailed the transcript to the company's CEO:

From:      ceo@webhost4life.com

To:          Jason G. Williscroft

Cc:

Subject:   Re: My database is offline. Here's how it was handled.

 

Sorry,

 

We were overwhelm by the domain name Mysite4now.com problem this morning and couldn't provide the best support we can.  Our doman name registrar made a mistake and disabled that domain that, we have over hundreds of calls and live chats and tickets this morning within the 2hr that was down.  We've already fixed the problem with the registrar and they will make sure this doesn't happen again.

 

Thanks,

 

Erwin Yu
Webhost4life.com CEO

OK. I think it's safe to say no human being will ever read my complaint.

Meanwhile, the fellow I finally got on the phone was very nice, and they moved my DB to another server where—after about 4 hours of having my entire portal network completely tits-up—I was finally able to see my site.

Of course, I don't appear to have direct access to my database. And, after apologies and assurance to the contrary, I still don't. Perhaps it was a mistake to escalate the ticket.

<sigh>

After all this, you'd think I would be warning everybody away from WebHost4Life. I'm not, though... fact is, I'm getting more for my money here than I could find anyplace else.

But today has seriously calibrated my expectations.

Post Rating

Comments

Anonymous
# Anonymous
Friday, February 23, 2007 6:44 AM
In my world, one "aw shit"wipes out ten "attaboys." Time for GoDaddy. No hold time; immediate, expert assistance; supervisors always available; 24/7.
Michael
# Michael
Friday, July 23, 2010 9:53 AM
I have located the address for WebHost4Life and plan on making a trip to the office to make sure my frustrations are heard by Mr Erwin Yu, CEO of WebHost4Life. I will be sure to take this entire transcript with me and make sure a human being reads this. I do not think the company has any idea how one mistake on their part can possibly put your or my business out of business.

I will keep you informed.

Post Comment

Only registered users may post comments.

Most Popular Articles

   Minimize
Cool Windows Resource Kit Utility: cleanspl.exe by Jason Williscroft (Tuesday, February 6, 2007)
v: 43493 | c: 5 Article Rating
They say things are big in Texas, but... by Robert Williscroft (Wednesday, March 7, 2007)
v: 27158 | c: 1 Article Rating
Sweet vindication – It really is climate cooling! by Robert Williscroft (Thursday, January 3, 2008)
v: 21904 | c: 11 Article Rating
E-Bomb: The Ultimate Terrorist Weapon by Robert Williscroft (Thursday, December 28, 2006)
v: 20491 | c: 5 Article Rating
Global Warming Deniers – Part 1 – Statistics needed by Robert Williscroft (Wednesday, February 7, 2007)
v: 16496 | c: 9 Article Rating

CLA

   Minimize
The Chicken Little Agenda: Debunking "Experts’" Lies

Block 1

   Minimize

Block 2

   Minimize