Developing an Incident Response Plan

Sometimes, I am asked to clarify the meaning of Cybersecurity or elaborate on operations handled by the field. Depending on the context of the conversation, we may discuss common examples that easily translate into cybersecurity concepts and then expand on the topics of threat hunting, malware analysis, and network security. However, all aspects of cybersecurity are defined by one constituent element: risk management.

Recently, I had the pleasure of leading the development of a simulated, practical incident response plan (IRP) and deploying it against a ransomware attack at scale on an organization's mission critical infrastructure. Regardless of the in-place architecture, our team was able to successfully assess the organization, plan for potential threats, develop a response strategy, and convey the IRP's importance to stakeholders. As such, understanding the concept of risk was crucial in operationalizing the IRP to protect critical business functions (CBFs).

Now you may be asking, "Why is this relevant to cybersecurity? Why do I need to know incident response? How does it relate to risk?" If this is you, don't worry.  I will touch on all of these points.

In this post I will be explaining what incident response is and why organizations need an IRP to improve their security posture. Then I'll outline the four phases of incident response along with the methods we used for deployment according to NIST SP 800-61 Rev 2, Computer Security Incident Handling Guide:

1. Preparation
2. Detection & Analysis
3. Containment, Eradication, Recovery
4. Post-Incident Activities

What is Incident Response?

Before understanding the incident response process, it is important to differentiate events from incidents. According to NIST, "An event is any observable occurrence in a system or network... [and] a computer security incident is a violation or imminent threat of violation of computer security policies, acceptable use policies, or standard security practices."

While NIST generalizes the definition of an incident, what constitutes as an incident is dependent on the organization. For example, Company A has a policy that prohibits the use of the Brave browser, while Company B encourages the use of the Brave browser. Company A may consider the installation of Brave a security incident that can be handled by a Tier I/II Security Operations Center (SOC) analyst.

In this scenario, the event would be the installation of the Brave browser on a company computer. This would then be classified as an incident once the event is analyzed, which would be triaged to the correct security specialist. In most organizations, this specific event would not be grounds for initiating an incident response plan as IRPs are designed to respond against more impactful adversaries/threats.

Incident response (IR) is the process organizations take to eradicate and limit the impact of an active threat against organizational assets and critical business functions. An incident response plan (IRP) is a document that outlines the procedures of incident response and grants the authority for incident responders to operate.

It may seem trivial for security analysts and system administrators to follow security best-practices. However, this is not the goal of an incident response plan. The mission of an IRP is to respond to incidents that have the potential of harming the organization from a holistic standpoint. An IRP would likely be initiated if an adversary is detected gaining initial access to a network, laterally moving between machines, attempting exploits against critical SQL servers, and exfiltrating credit card information, PII, PHI, or trade secrets. This would obviously impact the organization in several ways including paying federal fines, rebuilding reputational damage, and recovering from competitive disadvantages.

Developing a comprehensive IRP is essential for an organization as there are several factors to consider when responding to an incident. The IRP outlines the mission critical processes and supporting assets of the business with the organization's Business Impact Analysis (BIA), identifies the current threat landscape with risk assessments, assigns roles and responsibilities for the Computer Security Incident Response Team (CSIRT), and documents authorized processes for responding to incidents which are tailored to the business' infrastructure. 

How does this help an organization defend their infrastructure? Picture the response procedure as a linear frame of decision-making. In the mist of an incident, having an IRP pre-defines and authorizes certain decisions in varying situations. Without the proper procedures, the organization is essentially "guessing" what to do at every turn. This paves the way for several mistakes and improper handling of an incident.

On the contrary, having a vetted incident response plan can streamline the decision making with pre-approved response procedures in an orderly fashion. This decreases average response times which translates to a more controlled impact (or in other words, less money is lost as a result in an incident). 

With all of this information, let's now look into how our team developed an effective IRP.

Developing the Incident Response Plan

Below you can find the overall approach our team used during development as part of our presentation to the organization.

Since each phase is dependent on the last, we had taken a hierarchical approach when developing our IRP. We outlined the steps as follows:

1. Identify critical business functions, conduct a risk assessment on supporting assets, assess the level of impact if those operations were to be adversely affected, and calculate recovery time objectives (RTO), recovery point objective (RPO), and Maximum Tolerable Downtime (MTD)
2. Establish roles and responsibilities of the computer security incident response team and integrate necessary departments for efficient response
3. Operationalize the four phases of the incident response lifecycle in relation to resources available to the Cybersecurity Division

Identify CBFs & Supporting Assets

In this phase, we had gathered as much information about the organization as possible and developed a business impact analysis based on our findings and the technological infrastructure. Developing a BIA is out of scope for this post. However, below you can view the impact level for each business function. In this case, the organization was a tire manufacturing company with physical and online retail stores worldwide. 

The above table represents the key operations of the company simply in order to exist. Without these functions there would be no company, so it is important to document and acknowledge them as functions the CSIRT team aims to protect.

Specific assets can be tied to each function. For example, "Processing electronic payments" requires technical intervention as there is an online store that customers can purchase from. There could also be point-of-sale (POS) systems at the physical locations to process credit card information. Any system relating to processing customer payments in a technical format, including the tracking of cash payments, should fall under this CBF. Those assets would then be evaluated through a risk assessment by making connections between applicable threats and vulnerabilities.

Establish Roles & Responsibilities

Defining the roles and responsibilities of the CSIRT team required further understanding of the organization. For instance, below is the organization chart for the organization's information technology department with sub-departments for each specialized division.

As we can see, there is no dedicated division for incident response. By looking at the organizational chart, it is unclear who should be handling response procedures. While the SOC may be an understandable assumption, the question is who in the SOC should be responsible, and who manages the CSIRT?

Below is a brief overview of our CSIRT along with non-technical components that require interaction in the event of a wide spread incident.

CSIRT Manager (CISO):

• Role: Oversees the daily operations, strategic initiatives, and resource utilization of CSIRT
• Responsibilities: Establishing goals, handling financial matters, guiding team activities, coordinating responses to cyber incidents, and representing CSIRT in high-level discussions with upper management, stakeholders, executives, etc.

Technical Team:

• Incident Handlers: Detect, investigate, and respond to cybersecurity breaches. Includes Senior Network and System Administrators and Cybersecurity Engineers with deep institutional knowledge of the organization's computer architecture.
• Analysts: Study security events, industry trends, and monitor threat intelligence feeds to anticipate and mitigate future risks. Analyze security logs and provide reports with actionable intelligence.

Non-Technical Team:

• Legal Advisors: Provide guidance on legal matters related to incident response and compliance. Ensure the organization follows best practice when disclosing information on security breaches. Handles fallout of security incidents including potential lawsuits and proper compliance with federal law and regulations. 
• Communications Coordinator: Facilitates communication with both internal and external stakeholders throughout the incident. Oversees interactions with press, law, and public regarding computer security incidents. Manages public relations during and after incidents. Role assumed by Public Relations Manager.

There is much more than what is outlined here, such as identifying funding, tools, services, and staff availability requirements.

Operationalizing the Incident Response Lifecycle

The main focus of the incident response plan is the operational portion of the plan. Using NIST SP 800-61 Rev 2, there are four main phases of the IRLC:

1. Preparation: The preparation stage defines pre-incident operations to prepare the organization and CSIRT to defend against various threat actors. This includes ongoing risk assessments, continuous updates to the IRP, procuring security appliances, performing tabletop exercises, training of key staff, defining operating facilities, implementing security controls, and ensuring the organization has a clear understanding of CSIRT operations.
2. Detection & Analysis: The detection and analysis stage defines the procedures for detecting threats within the environment, including threat hunting and alerting protocols. Precursors to an attack are documented and monitored, and indicators of compromise (IoC) are thoroughly investigated. 
3. Containment, Eradication, & Recovery: This stage encompasses three major operations that are critical to an effective response. After analysis, containing the incident is critical to limiting the impact and scope of attack. Once the threat is contained, eradication aims to rid the threat from the system by running anti-malware tools or manual removal. Then the CSIRT brings the system back into production and operations continue as normal.
4. Post-Incident Activity: This phase notifies those who need-to-know about the incident and reflects on the handling of the incident with the CSIRT team. Post-incident activity includes drafting and presenting an after-action report, documenting the incident, and discussing with team members on how to better improve incident response operations.

We handled preparation by utilizing the BIA and risk assessments that were conducted prior. We identified resources that analysts could use during an incident along with in-place methods and areas intended to prevent security breaches:

• Continuous Risk Assessments
• Host Security
• Network Security
• Malware Prevention
• User Awareness and Training
• Ready-to-go equipment at a moment's notice
• Incident Analysis Resources

Once preparation elements were established, detection and analysis protocols were defined:

• Intelligence operations for detecting various attack vectors in alignment with the organization's infrastructure
• Definitions for IoCs and precursors 
• Alert and auditing sources
• Ongoing monitoring during the entire phase of an incident
• Key individuals who may report incidents or provide key insight in analysis
• Notification procedures once an incident has been detected

The next stage for containment, eradication, and recovery included various strategies for containment, evidence gathering, remediation of systems, and restoration of services based on resources available to the CSIRT team, such as backup appliances.

The final stage outlined who should be notified once the incident concluded, what information to provide, questions during analysis of the incident, documentation of the steps performed, lessons learned, and what kind of data to gather from the incident for improvement analysis. 

While in-depth details of the IRP is too cumbersome to include in this blog post, the overall incident response plan was 45 pages in length. However it is worth mentioning that these documents can be over 100 pages long depending on the detail, depth, and requirements of the organization. It is also important to understand that these documents are not static procedures. The threat landscape is always changing in an organization, and so the incident response team should always be adapting to the threats around them.

Presentation to Stakeholders

Once the plan is developed, it must be communicated to stakeholders. Most non-cyber or IT personnel are not educated in the technical terminology used in the industry, so presenters must be cognitive of the level of understanding throughout the audience.

Our team used PowerPoint slides along with visuals to assist in conveying the overall plan. While the network spanned across multiple countries worldwide, we had simplified the concepts for anyone to understand. This develops a baseline of knowledge that everyone is aware of, which can be built on over time.

We started with a standard introduction of the team along with defining a scenario for which the plan can be deployed against. We described what ransomware is and how our network is currently configured from a birds-eye view. Here, we explained the attack vector (from the internet) and how it relates to the company network, code-named NOTINO.

Then, we expanded on these concepts with potentially affected assets and introduced more detailed information on each of the four phases of the IRLC with specific operational procedures and capabilities for the CSIRT team.

Conclusion

Having an incident response plan is critical to an organization's security posture as it provides consensus and understanding of response procedures for key faculty. Operationalizing the IRP can prevent devastating losses and preserve the reputation of the organization. Keeping a business secure relies on ensuring these processes are continuously being improved on in preparation and during post-incident activities.

For those interested in entering the field of cybersecurity, I encourage you to study the NIST guidelines and apply those principles on systems you manage at home. This practice will help you develop practical skills and deepen your understanding of cybersecurity concepts. Stay proactive in keeping up with emerging threats and continuously adapt your approach to remain effective in the ever-evolving threat landscape. 

Attributions & References

• NIST. (2012). Computer Security Incident Handling Guide. https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-61r2.pdf
• White, S. (2024). The George Washington University CPS Incident Response Lectures. 202401 Incident Response PSCS 2304. Washington D.C.