Tag Archives: disaster recovery

How one NYC data center survived Hurricane Sandy

Because learning from someone else’s experience is the best way:

In the basement, salt water hit Internap’s fuel pumps, taking them out of commission, and a 10,000-gallon fuel tank suffered damage when water penetrated from the outside of the building through a breather pipe. With utility power out and that fuel supply unavailable, Internap turned to its 1,200-gallon header tank on the mezzanine level (where it also stores its generators), providing enough power to keep servers running for about 12 hours.

Internap has 47,000 square feet of office and data center space at 75 Broad, with thousands of servers on the 14th floor. Throughout Sandy and its aftermath, the company had seven or eight people on site. But in a flooded building with fuel floating around, “people were a little bit worried and on the verge of perhaps having to evacuate because of fear of fire,” Orchard said.

Read the full story at How one NYC data center survived Hurricane Sandy | Ars Technica.

IT Outages Cause Businesses $26.5 Billion in Lost Revenue Each Year

Part of a good risk assessment strategy is to estimate what can go wrong, estimate the likelihood of it going wrong, the cost of it going wrong, and the cost of preventing or ameliorating its going wrong.  Part of a good back-up plan is doing restores to make sure the back-ups and restores work.

North American businesses suffer an average of ten hours of IT downtime in a year,collectively costing them $26.5 billion in revenue, according to a study released by CA Technologies on Dec. 9.

In a series of interviews with CIOs, IT directors, and IT managers from 200 companies spanning financial services, manufacturing, retail, and public sectors, researchers calculated the financial losses incurred when businesses cannot quickly recover from service outages.


The researchers wrote that “most of this considerable cost to businesses” can be “avoided” through better data protection strategies.

“IT organizations can’t always prevent service outages, but they can take the right steps to improve the speed of recovery when outages occur,” said Mike Crest, general manager of data management at CA Technologies.

IT departments tend to focus on efficient ways to securely backup critical systems and neglected to consider the speed of recovery, the report said. With a myriad of products offering system and data protection, recovery and availability, organizations can assess their existing disaster recovery plans to minimize the time required to be back up and running after an outage, the researchers wrote.

via IT Outages Cause Businesses $26.5 Billion in Lost Revenue Each Year, Survey – IT Infrastructure – News & Reviews – eWeek.com.

How to Ensure Business Continuity with Cloud Computing

These days, businesses of all sizes are looking to cloud computing as a means to more efficiently deliver IT services to users. Cloud-based solutions offer a cost-effective way to maintain high availability and reliability for user applications, especially if they support mobile workers, telecommuters or field-based teams. Here, Knowledge Center contributor Chris Pyle explains the important disaster recovery and business continuity benefits that cloud computing can deliver to your business.

via How to Ensure Business Continuity with Cloud Computing – Cloud Computing from eWeek.

What does your Disaster Recovery plan cover?

While reading an otherwise unrelated article on Cloud Computing (Does the Fourth Amendment cover ‘the cloud’? | The Wisdom of Clouds – CNET News), I stumbled across this paragraph (emphasis mine):

I’ve reported on this early legal landscape a couple of times, looking at decisions to relax expectations of privacy for e-mail stored online and the decision to allow the FBI to confiscate servers belonging to dozens of companies from a co-location facility whose owners were suspected of fraud.

Like many other companies, my employer has its entire production environment at a co-lo facility.  Our disaster recovery plan covers the possibility of a few of those servers crashing or going down, but does not cover the confiscation of our entire server farm by the FBI.  We back up the servers, but the backup and restore utilities run from one of the devices in that rack. Would they take the firewalls and load balancers, too?

Of course, it doesn’t take a government action to result in the complete loss of a set of servers.

Does your DR plan account for the possibility of the complete disappearance or destruction of a set of servers?