What is Threat Hunting, what activities and roles are involved, and how can you execute an effective hunt?
What is Threat Hunting?
Threat hunting is the process of repeatedly searching a hypothesis-based data collection, analytics, or operational environment, including networks, systems, devices, and endpoints, to identify anomalous or suspicious activities or behaviors and determine if there are any ongoing threats within the environment that may have evaded previously evaded detection with standard cybersecurity tools.
Threat Hunting Steps
Successfully executing the process of threat hunting requires that organizations follow certain steps. To consistently and successfully identify anomalous or malicious activity, it’s important to repeatedly search a hypothesis-based environment with the appropriate tools and tips in mind.
To that end, let's cover the five main threat hunting steps - coming up with a hypothesis, collecting data, the trigger, the investigation phase, and the resolution/response phase - in more detail below to help you execute a more effective hunt.
1. Come up with a hypothesis
If you're ready to start executing successful threat hunting campaigns, it's important to first formulate a hypothesis that's based on cybersecurity insights and organizational nuances. Such a hypothesis is informed by your organization's unique network architecture, systems, and user behaviors, allowing for a more targeted and effective identification of anomalies and potentially malicious behavior. It's crucial to focus on what is typical and atypical to increase your chances of detecting previously unidentified anomalies or threats.
2. Collect Data and Intelligence
Our next step is to begin collecting intelligence and data involving diverse sources of data. This collection phase involves logs from your organization's servers as well as applications and network devices that can provide a detailed record of activities and anomalies. Our experts at GuidePoint Security can help you analyze user behavior patterns to identify irregularities that may indicate the presence of threats. We can also help you unearth invaluable contextual insights that allow for the correlation of disparate points of data specific to your organization.
3. The Trigger
A hypothesis that's informed by cybersecurity insights and that's combined with data can help identify a trigger—a specific pattern or anomaly that warrants your investigation. This hypothesis-driven approach is what our experts use to direct your attention to particular behaviors or events within your amassed data, like your logs or user activities, that deviate from the norm. When these deviations align with the hypothesized threat scenario, they act as triggers, signaling a potential security threat.
4. Investigation Phase
The investigation phase involves rigorous data analysis that relies on advanced tools and techniques:
1. Sandboxing: This tool involves running code, applications, or suspicious programs in an isolated environment (a "sandbox") to observe their behavior without risking damage to your main system. Sandboxing helps you safely analyze potentially malicious software and understand its impact, making it easier to preemptively identify and mitigate threats before they infiltrate your network.
2. Behavioral Analytics: This involves analyzing patterns of your user and system behavior to identify anomalies and potential security threats. Our threat hunting experts use behavioral analytics to establish what your normal behavior looks like and help you spot deviations from that norm.
3. Manual Code Reviews: Our experts scrutinize your source code to identify vulnerabilities and coding flaws that automated tools may otherwise overlook. Manual reviews give you a deeper understanding of your security posture and enable the identification of complex security issues.
5. Resolution/Response Phase
Our final threat hunting step, the resolution/response phase, allows your organization to swiftly mitigate the threats we've identified thus far. Using a well-defined Incident Response Plan, our experts will promptly neutralize all identified threats, repair your systems, and restore your normal operations. We'll demonstrate how to continuously refine your tools, policies, and procedures based on our findings to strengthen your defenses for the long term.
What is the difference between threat hunting and threat detection?
Threat detection involves the monitoring of systems, users, and data to identify security problems. It leverages automated malware detection tools, data analytics, and threat intelligence information. While cyber threat hunting may also utilize all these tools, it goes one step beyond to look for new threats and vulnerabilities that the detection tools have missed.
How does threat hunting work?
Threat hunting is an ongoing, active exercise in which experts leverage existing data sources and toolsets within an enterprise infrastructure, combined with additional threat hunting solutions to determine whether the environment contains unknown threats. Threat hunters often examine recent acquisitions into the infrastructure, investigate suspicious activities, leverage threat hunting professional expertise, and engage in penetration testing.
Why is threat hunting important?
The main goal of threat hunting is to identify and detect threats faster and respond quicker. This reduces the dwell time of adversaries in the environment and results in a more effective reactive portion of the cybersecurity program. No defensive system is absolutely unbreachable and no detection solution is capable of capturing every potential vulnerability and threat. Cybercriminals are innovative and stealthy—and if they want to breach a network undetected, many are capable of doing so. Once a network is breached, cybercriminals are also capable of hiding their activities from standard threat detection tools, such as sandboxes and antimalware and antivirus solutions. Proactive threat hunting can help identify suspicious behaviors that may be leading to an eventual breach, as well as hidden threats already embedded in enterprise networks, devices, and datasets.
How does threat hunting fit in the threat intelligence lifecycle?
Much like threat hunting, threat intelligence is a multi-faceted cybersecurity discipline and consists of many moving parts. That said, threat hunting is largely dependent on threat intelligence and the threat intelligence lifecycle. Successful threat hunting relies on well-built and testable hypotheses. One component of successful hypotheses is the ability to mimic real-world, relevant threats. The most effective way to accomplish this goal is to utilize threat intelligence.
The threat intelligence lifecycle focuses on the consumption and analysis of threat data across many platforms, both internal and external. The goal of threat intelligence is to inform other teams of emerging, active, and targeted threats to the organization and make strategic, operational, and tactical security decisions. Ultimately, threat intelligence feeds into the threat hunting life cycle and provides critical input on what threats may be a priority when hunting.
How is threat hunting different from threat modeling?
Threat modeling is the threat hunting process of assessing an environment’s or organization’s cyber threats and defining the threat level posed by each actor or event. Threat modeling directly impacts the threat hunts that are being executed and how often each hunt will be completed. For example, threats that have a high likelihood of being successful during an attack and will have a high impact on the organization should be hunted for more frequently than threats with a lower likelihood of success or lower impact.
Threat models tend to be quite large since they try to model out all threats for the organization.
How does the threat hunting team use the threat model to drive threat hunts?
A practical solution is to break the threat model into smaller, more digestible threat scenarios. The hunting team can develop strong, robust, and testable hypotheses with limited scope by creating these threat scenarios. This contributes to a highly effective threat hunting team.
Who are the threat hunters?
Threat hunters are not necessarily limited to a single team or a single role. A threat hunter is anyone who develops a hypothesis for a threat in the environment, collects and analyzes data, and attempts to improve the organization’s cybersecurity posture. If we take a more traditional approach to threat hunting, a threat hunting team is going to be composed of curious, procedure-oriented people who love to analyze data, problem-solve, and think outside the box. Threat hunting is fun, but it’s also a lot of data analysis and problem-solving, which means it is beneficial to have a dedicated threat hunting team.
Threat Hunting Techniques
A variety of techniques are used by threat hunters to identify suspicious or unusual activities and behaviors, as well as locate threats that may have already breached systems. These include searching and analyzing data sources, as well as several specific techniques known as baselining, stack counting (or stacking), grouping, and clustering.
Analysis—Inspecting data sources and logs (e.g., DNS and firewall), examining network, file, and user data, and reviewing security information and event management (SIEM) and intrusion detection system (IDS) alerts to identify threats.
Searching—Defining search criteria and then querying data to identify anomalies.
Baselining—Establishing the ‘normal’ threat levels and then exploring possible deviations from the norm.
Clustering—Examining large groups of related data to help isolate similar anomalous data characteristics or correlations between system and network activities. Clustering often involves the use of machine learning and artificial intelligence (AI).
Grouping—Based on predetermined search criteria, analyzing unusual or suspicious data to determine if a threat or problem exists.
Stack Counting or Stacking—Inspecting certain data values and then putting them into ‘stacks’ based on characteristics. Outlying data is flagged for further examination.
Threat Hunting Tools
Threat hunters use a variety of tools to support their methodologies. Tools can include the following:
- Advanced analytics, artificial intelligence, and machine learning
- Spreadsheets
- Statistical analytics
- Intelligence analytics
- Security monitoring
- SIEM systems
- Threat intelligence
- Behavior analytics
It is important to remember, though, that cyber threat hunting tools alone are not the only weapon in a threat hunter’s arsenal. Experience threat hunters apply tools based on methodologies and the needs of the organization.
Threat Hunting Methodologies
There are three primary categories associated with threat hunting approaches:
- Awareness-based Hypotheses—This involves using situational awareness and existing information about the environment to identify the most advanced threats to target during the hunt.
- Intelligence-based Hypotheses—An intelligence-driven hypothesis is created based on typical threat actor “tactics, techniques, and procedures” (TTP). Using this hypothesis, the hunters observe and inspect the network and systems to ascertain if these TTP behaviors are observed within the environment. Intel-based hypotheses can also be based on ‘indicators of compromise’ (IoCs) or ‘indicators of attack’ (IoA).
- Analytics-driven Hypotheses—An analytics-driven hypothesis is created based on the use of existing structured frameworks and models, as well as information derived from machine learning and artificial intelligence.
Threat Hunting Tips
During a threat hunt, the threat hunting team will focus on three primary objectives: what data to collect and its context, how to collect that data, and how to analyze that data to prove or disprove the hunt hypothesis. Tips for threat hunters include:
- Leverage programming skills. Don’t be afraid of Python, PowerShell, and other scripting languages that may make your life more challenging in the short term but far easier in the long term. Use your internal tools as best as you can. (Automation where possible is critical).
- Your organization spends a lot of time and money on security tools. Use them as effectively as you can.
- Use third-party tools where it makes sense. Don’t reinvent the wheel if you don't have to (Microsoft’s LogParser tool comes to mind).
- Write your processes and scripts down repeatability. This is essential to being a successful threat hunter.
- Don’t be a silo. Bounce ideas off your team and other teams, and allow different perspectives to add depth to your threat hunts.
How often do I need to hunt to be effective?
The answer to this varies depending on a variety of factors. Some organizations have dedicated threat hunting teams, some have part-time threat hunters, and some have other full-time jobs and occasionally take on the additional duty of threat hunting. We all have varying amounts of time that we can dedicate to threat hunting, so a more exact answer could be, “as often as you can without putting a strain on you (or your team).” One way to amplify a threat hunting team’s capability is through automation and detection improvements, but more on that later.
Threat hunting can be tedious, and it can consume a lot of brain power and energy. Remaining refreshed and capable is crucial for managing a successful threat hunting cadence. Evaluate your situation and establish a threat hunting cadence, but remember, it can always change. If we are to be cliche about it, “fail fast.” Don’t be afraid to implement a cadence and change it if it doesn't work.
When should you use a threat hunting service?
Organizations have historically faced several challenges when establishing and maintaining in-house threat hunting programs. For one, the rapid evolution of threat hunting requires that threat hunters possess a large amount of highly specialized knowledge, which is increasingly difficult to keep up with due to the current cybersecurity skills gap. Additionally, effective threat hunting workflows demand continuous updates and expertise that many organizations simply cannot provide thanks to a lack of internal resources.
This is where our experts at GuidePoint Security come in to help: they bring a wealth of experience and insights while obviating your need to invest in an in-house team to help you leverage specialized threat hunting services that effectively counter sophisticated threats.
Next Steps
The difference between a threat and an actual attack or breach often comes down to engaging in active analysis of systems using threat hunting techniques. However, it is important to remember that threat hunting is more complex than just leveraging data from a SIEM or using the latest analytics tool. Threat hunting is most successful when approached within the context and needs of the organization—that is, by understanding the types of threats most likely to target the industry or sector. Successful threat hunting also involves avoiding threat biases and bad analytical habits, applying the right methodology, and knowing which tools and techniques are most appropriate given the threat environment, timeframe, and budget. In addition, threat hunting requires a high level of unique experience and expertise, which makes working with a threat hunting service provider an ideal approach. Schedule a customized security consultation with one of the GuidePoint Security experts to help you evaluate your threat hunting needs.