2011.10.17 11:50AM EST
We are aware of a high load on biz100 that is causing intermittent service outages. This has been traced back to an attack on the server. We are currently mitigating this attack in order to relieve the high load and bring all services back to normal.
2011.10.18 11:50AM EST
Biz100 is currently under high load due to a suspected attack against Apache. We are currently working on implementing a patch to address the problem.
2011.10.18 4:45AM EST
Biz100 is still experiencing high load and intermittent issues. Our senior administrators are currently investigating the issue in order to assist with resolving the issue.
2011.10.20 3:11PM EST
While the exact nature of the issue on biz100 has not been completely pinpointed, we believe we have narrowed this down to a hardware issue. In order to best assist our customers and mitigate the issue, our Senior Systems Team is currently working on migrating accounts to new hardware as quickly as possible. Customers affected should expect to receive an email from Systems regarding their account move. We understand how important stability and uptime are for all of our customers and are working to make this process go as smoothly and swiftly as possible.
2011-10-18 12:48 pm EST
We're installing a fix on the server with an estimated time to completion in about one hour. I will follow up with systems again shortly. As soon as I have further details to provide, I will post them here for you. We do apologize for the inconvenience.
2011-10-19 5:13 pm EST
I see firewall updates were made on the 13th and now repeated downtime due to attacks. Any thoughts of backing out the firewall changes? It appears to have exposed a vulnerability. Also, I've received a few spam mails in the past couple of days. They're flagged as spam, but still hitting my inbox. Can't recall the last time I got one of those.
2011-10-20 8:52 am EST
Hello xdi and Oksana,
We do apologize for the inconvenience everyone is experiencing due to the recent issue on this server. We also want to thank you for your patience while our Systems Team works through this.
@xdi - The firewall updates we implemented were made to several servers that are not experiencing the issue that this particular server is. That rules out the updates as being the cause of the issue on this server. I'm sorry you're having trouble with messages marked as SPAM still going to your inbox. I recommend checking the filters in your email client as filters would control what happens with those messages.
@Oksana - I apologize for delay with updates. The proposed fix was implemented on the 18th but did not completely resolve the issue. Our senior systems administration team is working diligently to address this issue and will resolve this as soon as possible.