IT Risk Management 101: Help! We’ve Been Compromised

But, wait we umm.. never finished our Cyber Security Strategy… because you know {insert shiny item here} happened so we lost our focus. What do we do now??

The headline above is your worst nightmare. Believe me, this is an IT Department’s worst nightmare as well. There is nothing in this industry that will floor you or put you into panic mode faster than the realization that your data, your network or your customers’ data has been compromised. In addition, there may be no strategy in place to deal with the security incident. While we all “hope” it doesn’t happen to us, inevitably it will in some form or fashion given enough time. In these instances, it is very difficult to figure out where to start or even how to start. The question often asked is, “What do we do now?”

1. Cooler Heads Prevail

While it is human nature to go into panic mode whilst throwing your paperwork and computer from your desk in anger, often emotions and personnel instability make matters worse. Take a step back. Take a breath. Gather your thoughts and do your best in a terrible situation. Remember, the person next to you feels the same way, so make sure you are a supportive member of the team and not a destructive force instead. Your business and your team members will thank you later.

2. Identify and Assemble Cyber Incident Response team

Some businesses already have a team such as this in place and ready for action when a Cyber Incident occurs. While others may use a different approach, such as 3rd party independent analysts. Many businesses do not have anything defined yet. At Think|Stack, we create a group (I.T. S.W.A.T. Team) very quickly based on the type of incident reported. We choose our team based on expertise and skill set, threat knowledge, client familiarity, and a proven track record of problem solving and teamwork in a stressful situation. A point person is selected to develop strategy, assign tasks, and manage the effort in its entirety. Then we select a person that is responsible for gathering information from the team and customer then communicating it effectively to all involved parties. Lastly, we select our technical team that will perform breach/threat identification, mitigation and recovery. This strategy can work well for any business. Let your leaders lead, let your PR people communicate and let your technical people do what they do best, solve the problem. If you don’t have a team in place or a strategy developed for Cyber Security response, I recommend you create one immediately. Security Incidents are increasing annually and there is no end in sight.

3. Identify the Threat

While the team is being assembled or briefed, more than likely one of your technical team members is diving in to identify the threat based on the initial report. Is it a virus? Is it ransomware and every file you ever created is encrypted? Have we been hacked from the outside? Is it from the inside? It is important to understand how your infrastructure has been breached so you can create a mitigation strategy to prevent further data loss or data access referred to as “containment”. While you may not be able to find ground zero quickly, there are generally clues all over the network. The first person to report the issue, the pc or server that randomly went offline or was impacted in some way, the funny email you received or the link you clicked on, alerts that were sounded from various systems or log analyzers that show anomalies that you didn’t pay attention to at first. As you gather information and follow the breadcrumbs, use these pieces of aggregate data to identify what your team is up against.

4. Assess Impact and Penetration of Breach

Once you have identified the threat, its time to find out what the impact is to the users, network, customers and data as well as the penetration spread. Check your external and internal network mitigation tools. (Oh, you don’t have these yet? Shame on you!) Is one pc infected by a virus? Pull it off the network and wipe it. Simple and done. However, most incidents have far more of an impact and are much more complex than that example. Let’s just say that you have a poor password policy, a publicly accessible RDP session using the default port into a domain controller, and your NT Administrator account is live in your in your active directory environment and you don’t have segmentation or access lists created on network devices and said network devices are using default passwords. With relative ease, you can be compromised. Not only can all accounts be accessed leading to cracked passwords, but all servers and computers can be found on the network in a few minutes. In addition, your databases are now vulnerable and your customer data is exposed. Now the threat that is compromising your network can also be used for identity theft as well as sold on the dark web for quick profit i.e. credit card information or private information for marketing campaigns, research groups or other shadowy figures. Determining the impact and network penetration is critical in formulating the strategy for mitigation and recovery. It will also dictate how far you must go for your customers’ protection as well — Do you need to provide identity theft protection like Target and Home Depot did after their breaches were confirmed?

5. Communicate, Communicate, Communicate

I’m breaking up the process to include one of the most important steps of all. Remember that team you are working with? And that customer you are working for? And the person that is assigned to facilitate communication? Interact with them always. Be transparent about what is going on currently, what you are finding and what steps you are taking as you work through the process. Be sure everyone is up to date on the current tasks and strategy. Ensure your customers are aware of everything you are doing. A lot of processes in IT are done behind the scenes so you need to outline the progress and present that detailed strategy. Great communication builds great relationships with your team and your customers. The more you communicate effectively, the better everyone will feel about the situation as it progresses even if it is dire. Remember to ask questions too. Sometimes the slightest detail someone adds will provide a wealth of insight and in turn may save a lot of time.

6. Tactical Mitigation

Team Assembled — Check. Threat Identified — Check. Assess Impact and Penetration-Check. Engage Tactical Mitigation — Full Power! Remember some of these steps happen in parallel as we are a unit moving as one toward the goal. This is the step where we start plugging holes and mitigating the incident. Pull infected pcs or servers off the network, create access lists, block ports on firewall, disable specific accounts. Run AV/Malware scans. Confirm backups and off-site replication (if you don’t have either of these this, set them up soon) for recovery; This is especially important for ransomware attacks. After the review of firewall/IDS/IPS logs for strange events, block call back requests or any unknown IPs. If you detect IP ranges from other countries, setup Geo IP blocking on your firewall (shouldn’t you have done this a long time ago? You can start here.

It’s also a good time to use secure DNS forwarders such as OpenDNS or Cloud Flare depending on your security incident. Another possibility is password resets for all accounts internal and external. Fully patch all devices (please don’t let this be the cause of your breach). The list goes on and on, but the point is during this stage the steps performed are determined by the threat assessment, impact and penetration spread of your network and data. The work performed and changes made will “contain” the issue and should be a compliment toward the recovery of your network/data.

7. Recovery

Ah, everyone’s favorite part. We have been down for so long we need to get back up immediately! We are losing money! Our employees can’t service our customers! Hold on for just one moment before you make that decision. You need to be absolutely certain that you are ready for recovery and the threat has been mitigated. If you move to recover too soon, you run risk of the incident occurring all over again during or after your recovery procedures. Believe this one fact, you will lose a lot more money and time if this happens.

Double check your network and all logs or analyzers. Double check your security tools. Make sure your mitigation tasks are all completed before proceeding. Perform one task of your recovery plan to see if there are complications or initiated infections. If all goes well, slowly keep turning up services but continue to monitor like an owl in the night. Once confidence is gained go full throttle but make sure you recover using an organized methodology. Some systems need to be available before others do so keep that in mind when recovering your critical systems. One point to remember about a DR plan is that a portion of it can be used in your cyber security response plan to recover your systems in the right order, most critical to least critical. Once you are back up, congratulations! Now, think about everything you have learned and then write a root cause analysis to be used in future planning.

Nearly every business has insurance or a net in place to limit liability. Most businesses have disaster recovery plans and business processes documented for offline operation or fail over redundancy. However, do all businesses have a documented Cyber Security Strategy in place to handle the various incident scenarios? How about the tools or the team to help mitigate those security incidents once a breach has been identified? Being prepared and ready for a cyber security incident should be a part of every business risk assessment and plan. If you don’t have a strategic plan in place, need help picking your tools or if you have general questions about how to get started, don’t wait until it happens to you. Please contact us and we will be happy to help guide you down the path of securing what you value most, your data.