Staying Afloat During a Major Tech Outage: Make Analog Part of Your Crisis Management Strategy

What can your organization do to continue to operate in a MANUAL environment while you wait for your digital environment to be fully restored?

Reverting to handwritten boarding cards allowed IndiGo passengers to board their flight to Kolkata during the Crowdstrike outage.

Counting the total consequential losses from last week’s CrowdStrike outage is going to take a long time and, without doubt, many organizations will be reassessing their resilience in the event of another service drop like this.  Continuity plans - already in place for ransomware attacks - should have served as a stopgap if they are well designed, but in the meantime looking at your ability to restore a form of service as soon as possible, even if by non-digital means, can reap rapid rewards.

Most modern organizations have developed a business continuity plan.  However, the majority of these concentrate on how to recover from loss of data/equipment due to a catastrophic event or a ransomware attack.  Many companies have a robust backup and recovery plan to ensure the RTO (recovery time objective) can be met.  These plans are often expensive but can be well worth it to ensure your digital environment can return to operations as quickly as possible. But they are only valid if the infrastructure they rely on is available to you.. 

We ask you to look at the situation a little differently

Customers expect resilience from the brands they patronize and have little tolerance for brands that abandon them. What can your organization do to continue to operate in a manual environment while you wait for your digital environment to be fully restored?

What is your risk tolerance on the possible impact to productivity and revenue if you are unable to continue business?  

Conducting a Business Impact Analysis to determine critical business functions and the resources required to deliver them is a good starting point when developing your business continuity plan. 

• Identify potential risks to your business operations as well as the likelihood of occurrence

• Identify the cost of disruption for the different risks

• Identify alternative manual processes if possible

• Perform a risk-reward analysis to determine if the manual process is worth the cost

What is the level of outage and the expected duration?

Have you lost power to all devices that are required to run your operations, is it a situation where one component of your operation is non-functioning?  Do you know the expected time to recovery? (e.g. a few hours until the power is restored) or is it a catastrophic event that may result in days or weeks of outages? Your continuity plan should encompass multiple scenarios and alternative courses of action.

What type of service or function do you provide - and could it be converted to analog? 

For example I once saw a Starbucks operation shut down completely when their cash registers failed because the workers were unable to calculate in their heads the amount due on the transactions.  Coffee operations were not affected, but no one could manually process the sale.  A simple calculator would have sufficed, along with a policy that permitted manual transactions in the event of an outage.  The reverse situation would be more untenable - one where the coffee operations were affected, but the cash registers were not.  

Image generated using Adobe Firefly

Sometimes all you need to do is read the manual! In another example, a hurricane in the DC area grounded all the planes at Reagan International. The power was out and passengers were told that the airport would not be back in service for a few days. So, many people rushed to the car rental area to make their journey by road.

All of the rental car companies, except for one, put up closed signs, because they had no way to process credit cards and actually rent a car. And the famous story is, the minute the power went out, one guy opened a drawer, and behold, there was a company manual that outlined what to do in a situation where the power goes out!

The printed manual gave instructions on how to process receipts by hand; how to initiate the reciprocal agreement with the other car companies; how to go out and take a manual inventory of the cars, and how to log the transaction and rental details. And so they ended up renting out almost every single car at Reagan National because they were the only ones that had an analog plan

How can you encourage your teams to incorporate analog approaches?

At a high level, most business processes are abstract. Yet many companies rely so heavily on digital means to conduct their day-to-day functions, that these business operations often grind to a halt without power or access to computers.  Taking the time to walk through how you “could” stay in operation during a disruption can allow you to continue to function and generate revenue instead of just throwing your hands up in the air and saying you can’t do anything without your computers. 

Front-line employees are your best resource when developing an analog continuity plan.  They are the ones who would have to implement it and would know if alternatives are feasible or not. Your teams may need to feel empowered to use their initiative. Different corporate or national cultures may have different attitudes to this challenge.

If your business functions were disrupted by the CrowdStrike issue, you already know how that affected your operations and revenue stream.  Right now is the time to ask your front-line employees what procedures and policies need to be in place in order to continue to operate in an analog environment versus a digital one.  For example, having a policy that authorizes ‘manager override’ where the person in charge at a location sees the need to invoke an analog process.

Let this be a “teachable moment”.  Take advantage of the fact that your organization just participated in a real-life test of your business continuity plan.  Did you even have one? Now is the time to either update your existing plan or create one that can stand the test of the next CrowdStrike-type event.

How Canary & Shield can help

We live in an uncertain world where business and personal risks are a constant part of our lives. The nature and severity of threats is dynamic, so as a leader you need to regularly review their risk position for every aspect of operations: internal and external. Canary & Shield works with you and your team to ensure your business strategy has threat response built in. We can help you ‘look around corners’, challenge assumptions and develop leadership muscle memory that could help you avert or head off risks, providing you with confidence that supports your strategic goals and your recovery plan.