How to Start an Intelligence Program or Choose a Managed Intelligence Provider – Part 1 – Technology
When it comes to Threat Intelligence, there are some misnomers we think should be clarified. Data is not information, and information is not intelligence. Typical “Intelligence feeds” would be better called “data feeds”, with perhaps some earning the title of “information feed.” Data is the collection of raw facts. Information is the logical grouping of contextualized data. Intelligence delivers actionable information to drive a decision.
Feeds provide a collection of facts. The best tool vendors use technology to synthesize facts into information. More often, the feed they create cobbles together several commonly accessible data collections, like a Twitter firehose, list of TOR nodes, or good and bad IP address lists. The user interface and back-end is usually an Artificial Intelligence/Machine Learning reorganization and grouping of the data.
Turning Facts into Actionable Information
To successfully address risk, organizations must produce and disseminate intelligence within the organization. But how do you develop “Intelligence?”
True Intelligence is the intersection of critical data, an analysis platform, and the humans analyzing and delivering information in a timely, relevant, and actionable way. To build a successful Intelligence source, an organization must first curate a list of data feeds by evaluating dozens of feed vendors and selecting only those with the most relevant and timely data.
Weighing DIY Threat Intelligence versus Managed Intelligence
Attempting DIY is a costly endeavor. Organizations should thoroughly weigh the value of creating and maintaining a completely unique to them system versus leveraging a fully-tuned and tested solution from a Managed Intelligence provider, such as Nisos.
People, Process, and Technology
Calculating the return on investment for each side of that “build versus buy” equation requires understanding the cost of each element of developing an in-house intelligence program. In future articles, we will discuss the people and the process needed. This article focuses on data feed evaluation and acquisition strategy.
Next, they must build a robust, fault-tolerant, and secure infrastructure to acquire, process, and store the data. Finally, they must find and train experts who will be able to accurately combine multiple information sources, which will help the organization to produce intelligence, not just information.
Evaluating and Selecting Feeds
Step 1: Establish an External Data Acquisition Strategy
An intelligence program consumes external data to solve problems and address priority intelligence requirements based on the relevant threat model. Within security teams, most organizations use external data to address the following types of business risks:
- Cyber Threats
- Fraud and/or Platform Threats
- Physical Security
- Third Parties, Suppliers, and Subsidiaries
Our research and experience shows that successful management of these risks requires synthesis of four categories of information.
Data Feed Categories:
- Business: Information about U.S. and foreign corporations.
- Network and Telephony: External telemetry such as PDNS, malware samples (Virus Total), dark web, open web, domains, netflow, mobile data, and false positive aggregator (events not worth an analyst’s attention).
- Persons & Groups: Data solutions providers specializing in custom, scalable investigative and risk management tools for due diligence, threat assessment, identity verification, fraud prevention and debt recovery.
- Web & Social: Social media, dark web, news media, and foreign media.
These data categories are relevant across all risk types and against all threat actor types, whether they are physical, fraud-related, or cyber. As an organization grows, all of these data sources will be equally critical and should be considered as more threats emerge. Even the most mature organizations will be challenged by this process and if attempting to build your own program, a phased-approach will likely be required.
Don’t forget, in addition to the costs of licensing and procuring data, organizations must pay to store and exploit the data. These costs come in the form of Infrastructure (Data storage, databases, and data pipelines) and Analytical Productivity (Aggregation tools, social network analysis, and mind maps).
Value, Volume, and Velocity
If your organization is just beginning this process, be prepared for sticker shock and some tough choices. Feed prices vary wildly and they deliver different data. There are some great tools that are free; however, costs build quickly. The average cost for a single feed is approximately $10K/user/year. You’ll have to consider which matters most to you: the value, the volume, or the velocity of information, because it’ll be a tough balance with the resource constraints every security leader faces. (Or, you could partner with a Managed Intelligence partner that can deliver all three…)
When starting your own Intelligence program, you’ll need to consider how to split your budget. You’ll need to prioritize which information is crucial and work your way down from there, choosing key areas to focus.
If you were to try to build a robust program that incorporated the majority of threats, you could consider the following a good outline.
While you could certainly limit the areas of interest, your blindspots would also increase. That’s a consideration you’d need to incorporate as you evaluate whether to “build or buy.”
Step 2: Align Your Vision and Your Stakeholders
All data sources are not created equal. When choosing which provider you want to trust, you should:
- Develop a solid understanding of the business needs you want to fill
- Gather enough data in one place to feel confident selecting a solution
- Document justification for your CFO and CEO
- Provide responses/research to key stakeholders
- Provide materials to facilitate onboarding new staff
At Nisos, we systematically evaluate tools and feeds to select the right mix necessary to support our clients. If you are attempting this on your own, it’s important to remember that you’ll never have 100% visibility. To quote Jeff Bezos’s 2016 letter to shareholders: “Most decisions should probably be made with somewhere around 70% of the information you wish you had. If you wait for 90%, in most cases, you’re probably being slow.”
Step 3: Evaluate Data Feeds Required and their Value
The following process may be used as a guideline for how to evaluate fit:
- Establish business needs – It’s easy to see a flashy new tool and think you need it because it sounds like it covers what you’re looking for. First, start by documenting your business requirements, then look at which tools fit your minimum needs.
- Assign sponsors to needs – Engaging key stakeholder sponsors will help focus your search and prevent scope / feature creep. It will also help ensure there is alignment and valid justification for considering one technology over another.
- Prioritize functions and features – Most cyber security teams can’t afford to have it all. While there are many frameworks for prioritization, we have found one route we don’t recommend… complicated points systems. They just aren’t worth the energy for the return on insights. Keep it simple. Think of capabilities in terms of “must”, “should”, “could”, and “want” to have. Stay focused on those you must have.
- Canvas select vendors/peers – Once you’ve narrowed your list of candidates, ask around and see what others have to say about their experiences.
- Try them out – After an initial conversation or two with sales, it is common for them to offer a proof of concept (POC). This is usually a 2 – 4 week limited trial of the product. A newer term is Proof of Value (POV) for a paid POC.
Step 4: Select and Negotiate
Once you have completed the evaluation process and determined the offerings that best align with your strategy, your process, and your budget, you are ready to begin the negotiation process and move your program forward.
Intelligence analysis can be a time-consuming and costly process. A well prepared organization may evaluate dozens of data vendors a year. This is just the first step. As we will discuss in future posts, getting data into a platform, building the team to address threats, and ultimately delivering effective results that can be objectively measured are important characteristics of an intelligence program. Considering how expensive and complex this process can be, you might opt to choose to partner with experts who already possess these multi-sourced capabilities, data, technology, automation, and expertise to produce timely, relevant, actionable intelligence that can guide decision making.