East Coast Data Center Status

Starting at roughly 10:30AM EST on 12/16/2014 our System administration staff noticed a problem with the connectivity to our East Coast data center.

At this time, we currently have multiple teams working on getting connectivity restored as soon as possible. We apologize for any inconvenience to your business or personal website during these connectivity problems, and appreciate your paitence while we work diligently to get everything back online as quickly as possible.

Please bookmark this page as it will be updated with more information regarding this connectivity outage. You can also view the current status of our network on our network status updates. This page will also be updated as soon as we receive news.

Update 12:15 Some connectivity has been restored, but the issue is still being reviewed by senior technicians and staff at the data center. Apologies again for the interruption to your service. We are hoping to see the issue fully resolved shortly.

Update 2:45 PM – Connectivity is considered restored. Investigation of the issue continues, but all websites should be up and running.

Thoughts on “East Coast Data Center Status

    • Hello Ken,

      What would you like an answer to? This happened a year and a half ago.

      Best Regards,
      TJ Edens

  • Ok so now the WestCoast Data Center is out, less than a year after the East Coast issue. This seems quite common. Its already been like 4 hours?!?!?!?!!?!?!? Big surprise my customer is freaking out at me

    • Hi Mac, we apologize for the recent issues at the West Coast Data Center. We had an emergency within the data center outside of our control that caused the issue. At this time, all customers should be operational.

  • That’s it? Connectivity problems. Someone trip over a cord? The chances of our website (and how many others) completely disappearing from the internet for two hours again anytime soon?

     

    • Hello Thomas,

      Sorry again for the problems with the connectivity. The issue is that there was a backbone internet issue -beyond our own network – that affected our data center and other networks. Yes, there was redundancy, but the point of the failure affected the networks in such a way to cause the connectivity loss for a period of time. Review of the response time and with the services involved will be part of the evaluation process for this event. We do strive to make sure that these events are kept to a minimum.

      Regards,
      Arnel C.

    • Hello Guyman,

      Thanks for your concern. The process of explaining the outage issue is complicated because it is part marketing (as you have stated), and also because it has security concerns. We are not an entity like Amazon (who has billions of dollars), but we do value our customers and try to provide some transparency when an event like this occurs. Making a comparison with a company like Amazon is a little different when you’re looking at web hosting companies.

      Our data center is basically a collection of servers and network devices that are attached to the internet at some point. That connection point requires several devices that manage the traffic to and from the servers. At some point in that network, a failure occurred causing a connectivity failure for the data center to the internet. The period of time it took to resolve this issue is what we are closely vetting at this point in time. We are reviewing the processes and services involved in order to make sure that something like that does not happen again.

      We can’t discuss network architecture as it does involve security issues. Suffice it to say, there was a problem at the router level which could have been addressed sooner in order to restore connectivity sooner. The problem did not occur within InMotion’s server hardware, but above it in the scheme of the network. I hope this helps to some explanation for the outage.

      I hope this helps to answer your question, please let us know if you have any further questions or comments.

      Regards,
      Arnel C.

    • Hello Roger P.,

      We do apologize for the connectivity issues. No, it was much more complicated than tripping over a cord (as per my response above). The problem was resolved after the issue with the router was fixed. We are reviewing the issue on all levels to make sure that it does not happen again.

      Kindest regards,
      Arnel C.

  • Really.. – A 4 hour outage, on your entire EC datacenter, and all we’re going to get is – “it was a set of rare issues”..  Epic Fail, in damage control response…

    I suggest you go back to the Todd Robinson, and get him to post some of the details – I’m not looking for makes, models of routers, firmware versions, nor IP internal details around IMH’s setup, that said, give me in some general technical terms on what was the source trigger of the problem, why it took 4 hours to correct, and what have you done to make sure it doesn’t happen again – Your initial explaination, makes it sound like a rare convergence of issues, that’s waiting to happen again.

    FYI, paying customers need some info on what happened to make sense of how we should “vote with our wallets” around continued and future hosting plans.  – We’re having to explain the outage to OUR customers, and your not giving us much to work with.  Even AWS explains major outages – This just wasn’t a server or two, this was half of your company, and I want to understand what your doing to assure this will not re-occur.  –

    While this isn’t as bad as TigerM@te – Silence is not a good thing, in time’s of crisis – So go back and talk to your marketing “damage control” folks, the CTO and CEO, and network folks, and get some info out to your customer’s, beyound “it’s a rare thing, and we’re sorry”. 

    See https://aws.amazon.com/message/65648/ for a good example..

  • I thought that InMotion would have redundant backbone connectivity. Could you please expand on what exactly happened today? We have started to look at alternative hosting solutions in case this happens again.

    • Unfortunately, I am unable to publicly provide exact details in how our hardware is configured, but there were issues in which are very rare and were unfortunately unpredictable.

  • My website automationpractice.com cpanel is NOT loading.

    I tried enough number of times till now.

     

    Can you take a look please?

Was this article helpful? Let us know!