Wednesday, June 17, 2015

Can Inadequate Disaster Recovery Planning be worse than locusts?

Why are US farmers facing a disaster?

Is it due to locusts? No.

It's due to inadequate IT disaster recovery planning.

As reported in the Wall Street Journal, the US Immigration Department is unable to issue visas to temporary workers due to a system failure. Specifically:

"“The system that helps perform necessary security checks has suffered hardware failure,” said Niles Cole, a State Department spokesman. “Until it is repaired, no visas can be issued.” He said technicians are working around the clock to resolve the issue but couldn’t offer a timeline for when the system would be back in action.

Specifically, a central database isn't receiving biometric information from U.S. consulates world-wide, he said. Biometric data, including fingerprints, are used for security screening of applicants."

And the losses are mounting daily. Over 200 workers are sitting at the Mexican-US border waiting to be processed by system so they can get into the US and help harvest the crops. The article reported that farmers are losing between $500,000 to $1,000,000 per day because the fruits are spoiling.

Reading this article I had the following questions

Why isn't there a hot site? 
Given the importance of the technology, why don't they have the ability to swap to a new piece of hardware instantaneously?

Was the security information backed up and why is there no manual work around? 
If it's digital information, why isn't there a manual work around to transmit the information and circumvent the faulty hardware? The data could be manually uploaded to the central database.

Was a proper risk assessment done? When a disaster recovery plan (DRP) is created for a system, the organization must determine the Recovery Time Objective (RTO) that determines how quickly a system will be stored after failure. Google, for example, has an RTO of zero. To determine what the RTO is there needs to be an assessment of the impact of such a failure. In this case when setting the RTO did the risk management professional include the fact that this system was critical in supporting the visa program H2-A for temporary farm workers? It should be noted that the US farmers association had paid into this program and now they are suffering losses of over $500,000. This will also reduce the amount of tourist visas issued potentially resulting in lost tourist dollars to the US.

The lesson we can learn from this is to ensure that we understand what business processes a system supports and understand the impact to those business processes should the system go down.