We discuss the age old problem of spear phishing and why enterprises still struggle to fix this problem. We talk about the critical processes and technologies necessary to defend against spear phishing, including robust training programs and endpoint detections. We also cover how to use the telemetry collected from spear phishing and integrate this with outside threat intelligence to be useful.
Here are the 5 Topics We Cover in This Episode:
1) Security Teams Need to Make a Sensor Network from the Employee Base:
Attackers win consistently when they get employees to click malicious spear phishing links. They use social engineered communications, usually over email, that appear legitimate but have malicious intent to trick a user to open a document or click on a link to obtain sensitive information about a user.
Security training is boring and employees outside of security don’t pay attention to the annual reminders. Real education must be relatable to employees so that they can identify when a malicious link is deployed against them. The most critical training a security team can do is get a sensor network from their employees to spell out the ripple effects to employees for PII and intellectual property theft after a malicious link is executed.
2) Experts Must Create Critical Processes and Use Technologies Defend Against Spear Phishing:
A closed door approach to security is not efficient. Experts transparently interacting with the employee base defends against spear phishing. A phased approach will be necessary to assess the necessary logging in an automated way as this takes months to configure and properly alert. The building blocks of this approach are:
- An endpoint detection and response solution (EDR) is the most important tool to defend against spear phishing.
- An automated way to report incidents should be considered so users are not waffling on whether or not to report incidents. It should go without saying, but no one should get in trouble for reporting an incident.
3) Spear Phishing Typically Impersonates Executives; Executives Should Conduct PII Removal and PII Poisoning:
The sophistication and reconnaissance of advanced adversaries are challenging to detect, particularly when bad actors impersonate executives. Verifying information over the phone is often needed to circumvent advanced attempts to social engineer an employee base. Further, publicly available information about executives should be scrubbed and removed from the internet on a routine basis.
4) Use of Spear Phishing Telemetry with Threat Intelligence for Small and Medium Size Business:
Small companies with limited security personnel will be fortunate to get employees to get banners saying emails are coming from an external source. They will spend a small part of their day conducting internal threat hunting. They won’t be able to conduct external threat hunting to determine the sophistication of a spear phishing campaign. They need to partner with managed intelligence providers to do external threat hunting effectively.
5) “Defensibility” Measures are Critical Success Metrics: Threat Intelligence and Red Teams:
Quantifying reports and solutions that show how a security team is systematically reducing risks that affect their business is the only way budgets will get increased by the board. To prove that various attacks will matter to a business, threat intelligence with subsequent red teaming are the primary ways to illustrate the issues to an executive team.
Listen to other podcast episodes
Read Transcript
LANDON: Welcome to the “Cyber5” where security experts and leaders answer five burning questions on one hot topic in actionable intelligence enterprise. Topics include adversary research and attribution, digital executive protection, supply chain risk, brand reputation and protection, disinformation and cyber threat intelligence. I’m your host Landon Winkelvoss, co-founder of Nisos, managed intelligence company.
In this episode, I talk to senior security practitioner of SOC operations, Garrett Gross. We talk about the age old problem of spear phishing and why enterprises still struggle to fix this problem. We talk about the critical processes and technologies to defend against spear phishing, including robust training programs and endpoint protections. We also discuss how to use the telemetry collected from spear phishing and integrate this with the outside threat intelligence to be useful. Stay with us.
Garrett, welcome to the show, sir. Would you mind sharing a little bit about your background for our listeners?
GARRETT: Yeah, thanks so much for having me. My name is Garrett Gross. I run security operations at a mid-size software company here in the United States. Before that, I worked in several capacities, several roles, I’d say my forte is in penetration testing and social engineering.
Just to kinda give a quick caveat, you know, these are my opinions. They don’t necessarily represent the opinions of my employer, current or any employers that I’ve worked for in the past.
LANDON: I appreciate joining the show, and today we’re gonna be talking about spear phishing. Listeners probably think, “Oh, that’s been around, it’s a huge problem.” But it continues to be a huge problem. I don’t care who, what the enterprise is that you ask, whether it’s a large bank or a small mid sized tech company, or even of course, a small business, spear phishing is always gonna be the top concern.
So hopefully today we’ll bring a little bit of new ones to this issue, because I know it’s something that every security team deals with. It’s not just a cyber security problem, it’s certainly an executive protection problem. It is an employee process and procedure problem as well and touches on all those things.
So kicking off, Garrett, let’s call a spade a spade here. Before we’re gonna be in the game of not clicking links, we’re like that’s just not gonna be rational. The attackers are gonna win that fight all day. So understanding users are going to click a malicious link literally all the time. When you’re in your shoes, how do you not lose that battle?
GARRETT: Yeah, I mean, that’s a $64,000 question. I think the easy answer is education and over communication, but I think that’s a lot easier to say than to actually do.
I think, behind closed doors just joking around, we say “Well, you can’t fix stupid and that’s always gonna be a problem.” And that’s just a kind of a crappy way of saying you’re never gonna fix the human condition, which is you’re curious and you want to be contacted, you know, especially in the case of something like spear phishing, where they’re picking on something that you’re actually interested in, or you have a expertise in, you want that association to be made with you. So it’s almost like a curiosity. And then, of course, you know, if you’re just kinda going into it completely blind, it’s an education issue, I believe.
So, how do we address this? First and foremost deals with education about how to deal with these type of threats when they contact you, the actual procedures of how to report it, how to ignore it, you know, whatnot, but then also understanding how they came to you in the first place, because we’re talking about spear phishing, this isn’t just shotgun approach, spray and pray. This is actively targeted attacks on a user. And usually that is precipitated by something that they’ve put out there in the ether about themselves.
LANDON: So when you’re thinking of, when you’re talking about education, you know, the videos that you gotta click through to pass security training are probably not even paid attention to employees that don’t live security every day. You see these things as a nuisance, but nevertheless, you gotta develop a sensor network, right? How do you make employees feel comfortable coming to you, and that they’re not gonna get in trouble?
GARRETT: Yeah. Well, that’s a great question. You have to spell it out to them that you are celebrating the fact that they’re coming to you with these issues. Let’s talk about education. Let’s dial it back just a little bit. So I would say that what you hit on is actually pretty astute. That yearly security training that literally everyone has to go through is oftentimes ignored, your callous to it. It’s almost like, you know, if you’re watching a bunch of alerts on a screen, you know, unless you’re tuning them and you’re adding context to them, you kinda get dull to that.
And so, when you’re talking about security training, most folks are just clicking through that. They’re just ingesting enough to where they can pass that five question quiz at the end, right? I think real education is actually showing folks what these attacks are, what they can actually result and then learning how to thwart them. And, of course, you know, you’re dealing with different levels of technical abilities. So you can’t just go into your marketing team and start talking about wanna cry and how, you know, it abuses this certain protocol, and how that’s gonna have detrimental effects on your environment. You have to spell it out and you have to not dumb it down. That’s a pejorative. I would say that you have to make it relatable to your audience at all times, because again, much like most communication, if you’re not talking about a kinda global impact, it doesn’t really resonate with your audience.
And what I mean by that, Landon, is you have to tell them what their actions are gonna result in, explicitly. And you have to kinda go down that path and spell it out. Because if you just say, “Hey, don’t click on this link because we don’t want the bad people to get you.” Well, that’s not really tangible, but if you say, “Hey, listen, if you click this link and these folks get a hold of your credentials, they’re gonna use that to, you know, log into your assets, and then try to reuse all of your passwords and priv esc and do all the nasty things that attackers do once they get into your environment, but actually spelling that out and letting them know what the direct impact of that is. And maybe even spell out the ripple effects, you know, it’s gonna impact your business. It might impact your ability to have a job in the future. I mean, these are very real threats that I just, I’m not sure it’s getting across in a, you know, 15 minute yearly training that we put our users through.
LANDON: Is there any use in actually walking employees that are interested? I mean, there’s employees that just aren’t gonna care. But for employees that are interested in seeing what actually happens, and what’s the technical aspects of what happens after they click on that link, is there any way to do that at any kind of scale to show how the foothold results into the lateral movement, results into the privilege escalation, provide eds into the exfiltration, right? Or if you wanna take it from a fraud aspect of how giving over credentials then leads to identity theft.
I mean, is there any way you can do that at scale? And I guess which leads, I guess, really to my next question is, because you mentioned there’s targeted attacks that, why is this domain admin getting very targeted spear phishing? Like that’s something way different than spray and pray against an entire user base where an actor’s trying to get credentials. Then the next part of that is, what are the critical processes and technologies that you have to have in place to really do this and defend its scale?
GARRETT: So how do you do this at scale? My best approach, again, is to offer as many different avenues to learn as possible. And what I mean by that is different levels of technical detail, you know, different avenues like a webinar or a podcast like this, or a self-service video, or a zoom that you host, you know, company wide.
I mean, you kinda have to provide, you know, you have to meet the people where they are. And I know that’s kind of a cliche statement, but you really do. You need to make the information palatable, meaningful, and convenient to be quite honest, unless you say, you know, this is mandatory training and you have to be here and yada yada. So I’d say, how do you do that at scale? You have to work with experts that know how to tell that story. And again, it’s folks like Landon, folks like myself that have experience with these types of scenarios to where we can talk about the ripple effects. We can talk about how this would affect your credibility as it’s presented to your customers, you know, your customer confidence, stuff like that.
As far as what are the critical processes and controls to have in place to kinda understand this, again, you have to make it easy, and you have to make it straightforward. And I don’t think those two things are the same. I think you have to make it easy and straightforward to report these issues, and then you ask the question, how do you get people access to this that wanna know about these internal processes. We just open it up to anybody, and that’s kind of a controversial opinion, but I’ll give anyone access to any of our security tools that wants it read only, obviously, access.
Any employee that has signed an NDA with our company, they can have access to that stuff. And so, you know, if they wanna be a part of the investigation, that’s great. They can absolutely do that. If we have a major incident at the company that has to deal with the most confidential of confidential information, and it’s the biggest deal we’ve ever seen, and you’d like to sit in the zoom room and kinda watch this all go down, okay, that’s fine, I welcome that. I don’t get the whole closed door approach to security. I think it’s very silly. I think we’re often seen as this mysterious group of nerds in the back that they don’t know if we’re pen testing or playing D and D, but it’s like, “Come on in and we’ll tell you anything you wanna know.”
LANDON: And in terms of the technologies you have to have in place for a signature enterprise, what does a traditional tech stack look like to defend spear phishing? Are we talking about one or two types of technology? Are we talking about sometimes five to 10 pieces of technology?
Because again, understanding how the security landscape is, and you know how the security product business is, a lot of times just one technology for one specific avenue. What does that stack look like, and what does the data aggregation look like? It’s a lot of great telemetry. How do you make threat intelligence out of that?
GARRETT: Well, I mean, again, you’re asking all the very important questions because oftentimes we don’t find ourselves in a team of 15 analysts and you know, some super slick SIM that’s been in operation for four years. Typically we find ourselves, how I found myself, you know, walking into a fire, everything was just on fire. So your tech stack is what you have available to you. And so if you wanna say, well, what is the most desirable tech stack? I think you wanna do that in phases because even if you said, “Hey Garrett, here’s a blank check. You can spend as much money as you want on tooling.” I think it would be foolish to go buy all the industry’s leading tools out there, unless you have a team of folks that are well equipped and have the experience to run those.
So to answer your question, I would say the first step would be endpoint production, because the end goal of spear phishing or any kinda phishing attack is to compromise a machine or a system or data leak, or information repository or whatever. So if you have the endpoint protection that will thwart the attack or post exploit, it’ll be able to network with the machine, that at least gives you some time to breathe and assess the situation and deal with it. If you don’t have endpoint protection, you focus on something like IDS or a SIM like you kinda alluded to,
Landon, that’s a lot of telemetry and the chances of you noticing that and actually taking action in this sea of cacophony that we usually kind of, you know, live in, especially in smaller teams, you’re not gonna notice that.
But if you have some sort of endpoint protection that’s like a dead man switch type operation, it’s gonna identify that activity, it’s gonna block it, or it’s gonna terminate that process, or it’s gonna network contain the machine or whatever that looks like and give you time to notice it and then go, “Oh, here’s that thing that just happened. I can now deal with it.”
I would say number two on that list is an automated efficient way to report incidents or suspicious emails or something like that. So following the technical control, which again, I think is kind of the, and I don’t know what the policy on swearing is here, but you know, the OSHA switch, I think that kinda protects us in a way that even if you’re a novice security team, that allows you to, like I said, at least have some breathing room.
Second step, like I said, is to have an automated way to report these incidents so that your users are not waffling on, should I report this as a thing? Is this a thing? Is this important? So you provide them a way through, like, what we have is we use Slack, we use Slack and we use PagerDuty to go report these incidents where anyone can go into Slack, leverage some bot action, say, “Hey, I wanna report this incident.” That automatically pages someone on the sec ops team that’s on call. And we respond to that and we reach out to them and see what’s going on.
But the important thing to that, and this is the last thing I’ll say on this piece is that, before we even rolled that out, we made it very, very clear to air on the side of caution, and that no one is gonna get in trouble for reporting an incident. And even if you report something that’s not an incident like we don’t care, it’s totally fine, there is no penalty because, you know, again, I think if you live in the atmosphere of scrutiny and you know, “This better be a real incident or else you’re wasting our time.” I think you’re just gonna make people afraid to report incidents. And that’s just gonna result in a delay in our response.
LANDON: What is reporting incidents? Are we talking one to two a week? Are we talking 20 a week? And I know, of course, that depends on the size of the organization. But enterprise, what are you thinking?
GARRETT: Yeah, it also depends on what you classify as an incident. If you’re talking about just security anomalies, I mean, we see thousands of those a day. Again, to your point earlier, if we were to alert off of all that, we wouldn’t spend time doing anything else other than just looking at events. I would say that actual reported alerts from employees at an organization at our size, and we’re somewhere in the area of 1500 to 2000 people, I’ll just keep it kind of vague, I’d say we get about two a week. That’s not including suspicious emails. We probably get 15 or 20 of those a week.
So we also have an automated system within our email that just allows you to go to like message options, report suspicious emails. We get about, like I said, 15 or 20 of those a week. As far as people reaching out to us and manually saying, “Hey, we have an issue. We need some help.” Yeah, it’s one or two. And we typically have a, I don’t know, 75% false positive rate.
LANDON: What does that mean, 75% false positive rate in this regard?
GARRETT: Like three outta four reported incidents are usually not legitimate incidents. You know, maybe the EDR blocked something on their machine or a phishing email was sent out by the security team, or I noticed this performance hit on something and, you know, that’s not a security incident. So misclassification, misassigned severity, sometimes it’s not a security incident, you know, stuff like that.
LANDON: Let’s talk about what happens after you click on the link. There’s fraud and then there’s sophisticated actors’ ability to actually gain a remote code execution on your machine, meaning they have full access to basically everything you do on your computer.
What are some examples that you see in your career as they pertain to each of those types of results from spear phishing? ‘Cause me, spear phishing, that’s just the content delivery side, right? What happens after that?
GARRETT: Most of the spear phishing that we see is impersonating our executives, and they are targeting people’s mobile devices and sending text messages and saying, “Hey, this is your CEO, or this is your CFO,” and we’ll name drop them. I desperately need $500 in iTunes gift cards right now, something like that. Like that’s really the extent of spear phishing that we see in my organization. Most of it is drive by. Most of it is just a kind of spray and pray.
We did have an incident where someone was targeted and had them perform an action that they shouldn’t have done. And at the surface, it would’ve been hard to identify that to be quite honest. I mean, this is where education comes into play because, I guess what I’m saying is, with a nation state actor, it’s still so hard to tell what’s legit or not without calling that person on the phone and saying, “Hey, was this actually you that asked me to send me $500 in iTunes gift cards?” Because the sophistication, the persistence, the reconnaissance that has likely been done on your organization, it’s hard to thwart. And especially when you’re talking about your average security team, it’s gonna be hard to spot any difference between those two.
I, again, would say that if you’re talking about clicking links, if you’re talking about supplying credentials, you just have to verify, you know, trust but verify, and any kind of financial instructions that are given to you, you have a standard protocol to verify that with the person or the management or, you know, management of that person that issued that request.
So with spray and pray, again, it’s a numbers game. You’re trying to see how many they can get in the hopper, and then of those that are in the hopper, how many are actually gonna go through and, you know, statistically, it’s gonna pan out because again, statistically, people are just gonna click on things that I believe can be thwarted by education. That is, you know, “Hey, if you see a weird link, hover over it, or if someone asks you to send them some information that’s possibly sensitive, call ’em on the phone and confirm that.”
When you’re talking about spear phishing, specifically, and you’re talking about a nation state actor, and I alluded to this earlier, this is usually precipitated by the information that you’ve kinda put out in the ether about you and about what you’re into, what your hobbies are, where you went to school, where you grew up, you know, all this kind of profile information about you, we’ll call it a persistent actor can use that to, you know, send you a very specific email. And I’ll pick on myself, because I am not immune to this type of attack either which should tell you something. I’m also very much susceptible to this type of attack. So I love video games, big surprise there coming from a computer nerd loves video games. And I got an email that was from Microsoft and said, “Hey, you’ve been selected to get one of these new Xbox series Xs” or whatever. And, you know, it was a beautifully crafted email. The design was on point. There were no grammatical errors. The email address was actually, I think it was like an Azure address or something. So it actually looked like microsoft.com, but it was just something from someone’s Azure subscription. And I almost clicked it, like I was very close to clicking it because it just didn’t seem off. Like this seemed like something that I would ask for.
And so, that’s kind of the difference that we see in spear phishing is that, you have to go back to the source to really thwart that. For the folks that are being targeted, you need to understand, well, what information have you put out on the ether about yourself, about what you’re into, about what your responsibilities are, about what your wife’s favorite Broadway show is, but this information can be used against you and absolutely will take advantage of just some psychological tricks that make you let your guard down and that will cause you to click these links. Because if you don’t think that these people know psychology better than you do, you are wrong, because that is a very, very important component of crafting these lures.
LANDON: Before we get into what success looks like, ’cause I mean, this seems like almost like a no win game in so many ways. How do you take all the telemetry that you’re gathering on spear phishing? The tools and telemetry are there to record headers, banners, profiles, profile names, IP addresses that come from every single email that comes into an organization, that is there.
There’s also the ability to protect against it. There’s ability to aggregate it, understanding that almost the perimeter, in so many ways is almost a thing of the past, particularly with cloud coming into the picture, you know, with AWS, GCP and Azure, how do you take all the information that’s coming from email and leverage that against telemetry that’s recorded from the outside with regard to threat intelligence, how do you make that useful for the SOC, right?
‘Cause I’m just thinking like you just made the mention of, you get thousands of suspicious events. You get 20 that are interesting. Okay, let’s look at that as 20, but I’m still thinking, okay, if I gotta go investigate 20 suspicious emails and look at like what other IP addresses, is this a campaign that’s targeted? Can I go look and enrich this with outside telemetry and outside net flow that is going to extrapolate other IPs and other domains that can block at the proxy?
I mean, that’s still a huge lift that you could spend all day doing, but resources are time intensive. How do you leverage all that spear phish intelligence with external intelligence to make that useful?
GARRETT: I’m gonna give you an answer you’re probably not gonna like, we don’t. We, as a five person SOC, don’t have the ability to do that. We can use our telemetry to build, allow lists, or we can, you know, add in context to say, “Hey, this is from an external source.” Like for email, for example, it’s actually a pretty useful flag. “Hey, this is an email from outside your company, pay attention to this.” That just adds a little bit of extra credence to it or context.
As far as how we make sense of all this big data, I mean, that would’ve to be a full-time job. That would have to be someone’s full-time job to gather all of this information and make sense of it, and actually present it in a usable format. Because like you said, we’ve got all this data, it’s there. I can go grab it from all of my event sources, and I can put it into whatever, and then what? Because I’ll tell you what, we barely have enough time to threat hunt. And that is a very small amount of our day, and we’d like to open that window a lot more. We’d like to be a lot more proactive and a lot less reactive, but I gotta tell you, you kinda asked, I’m kinda getting ahead of myself, but you asked what success looks like.
I think success looks like realizing that there is no success. It’s a constant uphill battle. So let me get back to your question that you asked, how do you make sense of all this data? Well, you enrich it with threat intelligence provided by people who know what they’re doing. You enrich it with threat intelligence that has to do with your organization, and your organization only. You use real correlation that you know about, but then you also build correlation that may not make sense. And you kinda look at all the different correlations you can make between the data sets and just treat it like threat hunting. And that would be my approach.
And again, this is coming from someone who does not have the luxury of kinda sifting through all this data on a regular basis. For me specifically, I would 100% have to use a vendor to sift through this and make sense of all this, because I mean, again, even if I knew what I was doing, I don’t think I would have the time to do it with the size of the team that I have.
LANDON: Which I guess leads to the final question. You kind of already touched on it, but if success is an amorphous subject, realistically, success is we didn’t have a breach or there was no exfiltration of sensitive data. That’s of course success, but cyber security is really hard. And like, this is just one aspect of why this is hard. And we’re just talking about phishing. We’re not talking about application security, we’re not talking about the internal network, different levels of challenges within networking between IT and what’s happening with security tools or even application development.
We’re just talking about phishing, spear phishing. So understanding that this is where you have to leverage to have resources, what are critical success metrics in terms of lobbying the C-suite, if you will, that ultimately kind of speaks to that business risk to ultimately have more resources to be able to do external threat hunting that you’re talking about? Getting other different data sources that have a level of automation that, you know, you can take a five person SOC and be super effective, how do you translate that really to business risk?
GARRETT: That is a hard one. And that is one of the biggest challenges in our industry, especially when we’re dealing with things that are a bit outside of the box. We saw this with application security. We saw this with SIM. We saw this with cloud security, with DLP, with cloud security posture management. It’s like this constant battle of having to convince people like, “No, no, this is something we need to pay attention to.” With this, you hit on the nail on the head, which is, it is unacceptable to go into a boardroom and go, “Well, we haven’t had any issues, so we’re looking good. Any questions?” You’re gonna be walked outta that room very quickly because that’s not proving anything. What I’ve found success in.
And again, I think this is still a subject up for debate. I rely on defense ability. You know, I present things to my management and to my board of directors that elucidates the fact that I know what I’m doing, that I have performed an assessment to the best of my ability, and that I’ve, you know, collected that information, and that we have actions in place that are trying to address this existing or new behavior or whatnot. If we can quantify that risk, you know, number of incoming emails, number of reports, number of anomalous IP address connections or something like that, if we can identify that type of metric, then we can start to demonstrate a reduction in that type of metric.
Hey, we saw because we were gathering intelligence on these messages coming in, we’re able to use that to build more of an allow list or more of a context base. Like I was saying earlier, you know, you use something as simple as a label that says, “Hey, this email came from outside of your organization.” If you can demonstrate that you’re continuing to pay attention to things like that and actually come up with real solutions to these problems, and when those solutions don’t work, you have a contingency plan, or you’ve got a different avenue, that’s how I’ve been successful into proving to my peers, my higher ups that we’re actually doing something about it.
Now, I think the question that you didn’t ask is, “How do you convince them that this is like a legitimate issue?” And that, I’ve only been able to do through red teaming. You can only really convince someone of the gravity of the situation when it kinda blows up in their face. And in our industry, that’s not acceptable to wait for an attack. So that’s why we rely on, you know, scenario-based demonstrations and sometimes they’re known and sometimes they’re not. And, you know, depending on your executive team, your mileage may vary on how you roll that out.
LANDON: Garrett, I can’t thank you enough for joining the show today. You’re certainly very knowledgeable in this space. I appreciate you sharing your expertise with the community.
LANDON: For the latest subject matter expertise around management intelligence, please visit us at nisos.com. There we feature all the latest content from Nisos experts on solutions ranging from supply chain risk, adversary research and attribution, digital executive protection, merger and acquisition diligence, brand protection and disinformation as well as cyber threat intelligence. A special thank you to all Nisos teammates who engage with our clients to conduct some of the world’s most challenging security problems on the digital plane and conduct high state security investigations. Without the value the team provides day in, day out, this podcast would not be possible. Thank you for listening.