Blog

Three Considerations for Measuring Return on Investment from Threat Hunting

by Landon Winkelvoss | Jul 8, 2020 | Blog

Threat hunting often has ill-defined metrics for organizations attempting to measure “return on investment.” If an analyst isn’t finding bad actors in the environment, leadership may question the value they are bringing. If they are finding a lot of actors, leadership may question how effective they are at their job if incident response is constantly being called for false alarms. Furthermore, questions will arise, depending on how long the actors were present in the network, the severity of the breach and if disclosures need to occur.

In almost all of the above scenarios, a threat hunter’s job is usually limited by not having enough visibility to make the appropriate detections. In the case of threat intelligence, sometimes there is too much information, which we will cover in forthcoming blogs.

Below are some considerations for level setting expectations and deriving the appropriate metrics for threat hunters.

Create a Coverage Map

After determining the set of actors that are likely to target the enterprise (nation state, criminal groups, hacktivists, insider threat, and script kiddies), create a coverage map for the purpose of identifying collection gaps.

The MITRE ATT&CK framework identifies many tactics and techniques utilized by threat actors. While selecting various TTPs allows an organization to identify and address the gaps that matter, this may not be possible for a couple reasons, mainly:

Logging is insufficient
Detecting may require a defensive tool that is not in the budget

Therefore, a cyber analytics repository needs to be created.

Use a Cyber Analytics Repository

A Cyber Analytics Repository (CAR) is a catalog of product-speciﬁc queries used to hunt TTPs identiﬁed in the coverage map.

For example, if an organization uses Splunk, the query will be in SPL. f an endpoint detection and response agent (Endgame, for example) is used, the query will be in EQL, etc.

The CAR may also include use cases from which to create alerting rules. We shared some thoughts here on how to develop use cases for threat hunting.

Create Metrics Based on Company-wide Actions

Threat hunting is not just about finding bad actors; it’s about driving threat and risk mitigation in conjunction with security operations, security engineering, risk and compliance, policy and the appropriate business unit.

Threats include malicious or suspicious activity on the network. Risks include coverage gaps, missing logs, or missing alerts. Both should be tailored to an organization’s requirements.

It is not enough to discover a threat actor on a network or a coverage gap. Threat hunters must reach out to the appropriate teams in an organization with suggestions on how to remove threats and close gaps.

Further, not all ﬁndings will have equal value. For example, discovering an advanced threat actor on a network will have much more value than ﬁnding adware on an endpoint. It is important to deﬁne values for ﬁndings and to keep track of these values to demonstrate the return on investment that the threat hunt team brings to an organization.

To ensure that ﬁndings are addressed, the threat team must track closure. At the end of each quarter, the team can provide these metrics to management. In turn, management can use the metrics as a guide to allocate resources.

For example, the threat hunt team may have discovered risks and gaps that fell under the onus of the security engineering team. By the end of the quarter, if the security engineering team has only actioned a small percentage, management can consider allocating additional resources to that team.

Blog

Three Considerations for Measuring Return on Investment from Threat Hunting

Table of Contents