*********************************

UPDATE

The connectivity issue which interrupted service to our East Coast Data Center has been resolved at this time. We suspect the issue was due to hardware failure that one of our bandwidth providers had to address. Please let us know if any connectivity issues persist.

*********************************

UPDATE

While our Systems Team is working with our bandwidth provider to resolve the connection issue to our East Coast Data Center, connectivity to the data center may be intermittent.

We sincerely apologize for the inconvenience, and appreciate your patience.

*********************************

Our Systems Administration Team has identified a connectivity issue outside our network hindering many of our customers' ability to reach us and their sites in the East Coast Data Center. Our System Administrators are working with our data center to identify the exact cause of the issue and work with the appropriate provider(s) to effect a resolution as quickly as possible. We sincerely apologize for any inconvenience this may be causing and appreciate your patience.

We will be updating here and on Twitter as soon as more information is available. Please understand that there is a higher than normal call volume at this time, so it may take a little longer than usual for our Support Team to be able to respond.

Did you find this article helpful?

We value your feedback!

Why was this article not helpful? (Check all that apply)
The article is too difficult or too technical to follow.
There is a step or detail missing from the instructions.
The information is incorrect or out-of-date.
It does not resolve the question/problem I have.
How did you find this article?
Please tell us how we can improve our Support Center:
Email Address
Optional, but our team may contact you for more information.
Like this Article?

Comments

Post a comment
2012-04-10 9:47 am
Again, really?
2012-04-10 9:47 am
looks like we are back up
2012-04-10 10:02 am
Down again
2012-04-10 11:14 am
No other provider? The current one needs a few more gerbils and wheels.

Hopefully not a new trend for 2012
2012-04-10 11:41 am
Must admin I'm concerned about this trend.
Two outages in two weeks. I would expect there would be several providers into this datacenter. Will be watching closely over the next month or two to decide if we stay with this provider.
I hope so because they are a pleasure to work with and provide reasonable updates on issues.
2012-04-10 12:50 pm
Why have there been frequent outages at the East Coast data center lately? Up-time almost never used to be a problem with any of my accounts on your system(s), but this has become a reoccurring event over the past few weeks. If it is truly a provider issue as your web announcements seem to indicate, why is your network not peered with other provider(s) for redundancy?

I understand that unpredictable things can happen that are not always within a service provider's control but this is starting to become unacceptable. A data center of any reputable size should never expose itself to the failure of an upstream provider and should always be staffed with WAN/LAN engineers that are capable of designing and maintain large scale network redundancies.

More than anything I need to know if this issue is going to continue on a regular basis so I can plan accordingly.
2012-04-10 1:17 pm
When you say there is an issue outside your data center... you're inferring that its out of your control or not your problem. But as far as my internet hosting goes... it's a black box... I don't care about what's inside the box (things called East Coast Data Centers, etc.) I have the expectation that my website doesn't go down. Now if something breaks inside the box -- the owner of the box needs to have contingency plans so the users of that box don't know something is broken. As far as your customers are concerned... this was not an "issue outside our network" --- it was very much an issue INSIDE your network because you've promised to keep our websites up and running.
Staff
2,760 Points
2012-04-10 1:27 pm
Hello everyone, this is BradM with the InMotion Hosting Support Team. For those of you commenting, I did want to chime in and state that I do personally understand everyone's point of view on this downtime, and I do not plan any rebuttles to try to get off topic. Our System Administration team is still working to get all of the details (as generally things are not 100% clear until all the smoke has cleared). What I have been told from the Systems team is that this issue again has been caused by failed hardware, a router going bad. We are seeking drastic action from the Data Center team in an attempt to get a more permanent resolution.

While downtime and outages can be caused by many different things and could impact any hosting provider (or any server on the web), the bottom line is that it is affecting you (our customers) and we understand the frustration that it causes. We are very sorry, and the entire team is working to tackle this issue from all angles.
2012-04-10 1:47 pm
******]Hello everyone, this is BradM with the InMotion Hosting Support Team. For those of you commenting, I did want to chime in and state that I do personally understand everyone's point of view on this downtime, and I do not plan any rebuttles to try to get off topic. Our System Administration team is still working to get all of the details (as generally things are not 100% clear until all the smoke has cleared). What I have been told from the Systems team is that this issue again has been caused by failed hardware, a router going bad. We are seeking drastic action from the Data Center team in an attempt to get a more permanent resolution.

While downtime and outages can be caused by many different things and could impact any hosting provider (or any server on the web), the bottom line is that it is affecting you (our customers) and we understand the frustration that it causes. We are very sorry, and the entire team is working to tackle this issue from all angles.******


BradM,
Thank you for taking the time to post here and acknowledge our concerns. I believe it speaks volumes when a company is upfront and honest about an issue. If your current information is accurate, I am still concerned because under no circumstances should a single router failure have the capacity to sever connectivity to an entire datacenter. I understand that a previously undocumented fault can probably get a free pass once, but this is at least the second occurrence of this end result in a very short time window. While I have no knowledge of your network design and for security purpose don’t expect to be informed of such, I can only assume a datacenter of your size would be employ something along the lines of a Cisco 6500 or an equivalent class of hardware that offers hot/stand-by operation at a very basic level.

I am not trying to ague anything or even seek a billing credit at this time, but I would like to know that a real solution to issue is being ascertained and not another “quick-fix” assuming these two outages were even remotely related. I would much rather see a notice about a planned non-peak hour outage to correct the issue than run the (probable at this point) risk of another failure.

Again, I appreciate your communication efforts and hope it continues.
Thank you for your time,
2012-04-10 1:49 pm
Hi Brad, I did not finish my networking degree, but why don't they have some redundancy in place. Couldn't there be a second router that could be used for this type of scenario?
2012-04-10 2:12 pm
BradM... I appreciate the time you have taken to acknowledge our concerns. As you can see the impact of this outage was significant. What is most alarming that this has been the second outage in recent weeks and today's outage was a result of a single failed router. It can also clearly be seen IMH does not have an adequate/solid DR plan in place. I hate to be harsh, but that is totally unacceptable and needs to be addressed by IMH ASAP. We as customers also expect a response and acknowledgement from your Management Team on what is going to be done to assure us this will not be a reoccurring issue. InMotion is known as reputable and quality hosting provider, but due to recent events in the minds of many that notion has changed for the worse. I look forward to your and your Management Teams response.
Thank you.
2012-04-11 10:41 am
"Must admin I'm concerned about this trend. Two outages in two weeks. I would expect there would be several providers into this datacenter. Will be watching closely over the next month or two to decide if we stay with this provider. I hope so because they are a pleasure to work with and provide reasonable updates on issues."

Folks, I really need more information on this outage. My executive staff is insisting on answers to the following questions:

1. How/why did a single router failure or provider outage bring down connectivity to the datacenter and specifically to our website?
2. What network redundancies were in place on Tuesday to prevent this outage?
3. How/why did the network redundancies fail to prevent the outage that occurred on Tuesady?
4. What changes have been made to the network infrastructure to prevent a similar failure from causing the same outage?


It's important that I can answer these questions to the satisfaction of my executive staff.
Please help me (and others posting here) to justify our choice to host with InMotion and quiet fears of a similar failure. Thanks for your help.

Post a Comment

Name:
Email Address:
Comment:
Are you a bot?
Submit

Please note: Your name and comment will be displayed, but we will not show your email address.

Write New!
Do you want to publish a tutorial to our support center?

News / Announcements

SSL Certficate Warnings
Updated 2014-04-14 11:34 am EST
Hits: 2015
Heartbleed 0-day OpenSSL security bug
Updated 2014-04-14 04:43 pm EST
Hits: 5303

Related Questions

Here are a few questions related to this article that our customers have asked:
Ooops! It looks like there are no questions about this page.
Would you like to ask a question about this page? If so, click the button below!
Ask a Question

Need more Help?

Search

Ask the Community!

Get help with your questions from our community of like-minded hosting users and InMotion Hosting Staff.

Current Customers

Chat: Click to Chat Now E-mail: support@InMotionHosting.com
Call: 888-321-HOST (4678) Ticket: Submit a Support Ticket

Not a Customer?

Get web hosting from a company that is here to help. Sign up today!