The Cyber5 Podcast

LANDON: Welcome to the Cyber5, where security experts and leaders answer five burning questions on one hot topic and actionable intelligence enterprise. Topics include adversary research and attribution, digital executive protection, supply chain risk, brand reputation and protection, disinformation, and cyber threat intelligence. I’m your host Landon Winkelvoss, co-founder of NISOS managed intelligence company. In this episode, I talk with DoorDash application security manager, Patrick Mathieu. We talk about threat intelligence role within application security programs, particularly programs that focus on fraud. We discuss the importance of prioritization of what could happen between threat actors as often seen penetration testing and what is happening between threat actors as often seen in threat intelligence. We talk about the different types of internal and external telemetry that can be used to drive a program and discuss the outcomes that are critical for an application security program to be successful. Stay with us. Patrick, welcome to the show, sir. Would you mind sharing a little bit about your background for our listeners, please?

PATRICK: For sure. So I’m Patrick Mathieu. I work in application security at DoorDash, but my background is mostly offensive security, which is actually where we met. I’ve done pen test on the application side for almost 10 years. And after that building offensive security teams in a few companies on the consulting side, again on the internal side. And this year, I decide to switch back to the blue team and go back at the AppSec side. So that’s interesting. And at the same time in Quebec, Canada, so we also run me and a few friends Hackfest, which is now the largest hacking event in Canada. So these are my opinions and not those of my employers.

LANDON: You’re a very busy person, always great to talk to the Pen test community who brings certainly a variety of experiences, certainly from application security, pen testing, and of course even how threat intelligence really overlays that, which are kinda what we talking about today. And that’s really integrating threat intelligence to an application security and fraud program. And certainly there’s no shortage of threats from that plane that you probably deal with on a day to day basis. I guess, let’s talk through the building blocks, right? So you’re day one, you gotta build an application security program from scratch. How are you doing that? Because I have to assume any gig economy platform or marketplace for that matter. And the tech sector is dealing with a heavy dose of fraud, certainly as they get over 500 million a billion in revenue. So I’m kind of curious of asking that question with the fraud context.

PATRICK: One of the thing that I was lucky joining at DoorDash is the team already existed. So it’s more about making it bigger, making it grow. So we already have the application security team that exists. There’s a SecApps, there’s fraud. So there’s a lot of things happening, but the merge between the two, what is interesting and not just from this perspective, but even in the past, like in other company, when I was off set is that fraud is usually individual in their corner. It’s more of looking at saving money and not really looking into the application security side of it, even though I think that most of the time from what I’ve seen, it’s super interconnected. It’s mostly the same. It’s just one is obviously to fix more availabilities on the other side it’s to save money. But most of the issue comes from things that could have been saved if a pen test a security scan, a code review or a design review was done, or didn’t miss sometime obviously errors happen and some vulnerabilities that can lead actually to fraud.

LANDON: And so from that context, there’s probably another conversation, probably a whole nother podcast. We could probably do around development security operations, just DevSecOps in terms of the building, those security practices into the development as they build. But, you know, look, I mean the tech sector, there’s not a lot of regulation, so therefore, there’s a huge desire to push production and productivity to drive revenue in those early days. Probably security’s not taken in mind. I don’t think that that’s probably gonna go away anytime soon, we’ll evolve and adapt, but I guess going off your question that you just mentioned, what would you tell an engineering team in the early days, if they can just do a couple things that makes these types of issues easier for when a security team has to come in down the line and start really kind of watching the guards and the gates, you know, of confidentiality and integrity and availability of data systems and networks.

PATRICK: I think we always go back at the treat model. It’s the first thing. So when we do offensive security or Threat Intel, or even AppSec, it’s all based on the threat model. So if you secure something that doesn’t make sense to your organization, obviously you’re gonna put money where it’s not the right place, but in AppSec it’s usually surrounded by SDLC. So there secure development life cycle that is binding to the development life cycle of the developer, the engineering, whatever they use, agile or other kind of sprints. So based on this, the first thing they need to do is know what the potential threat are for their application. It’s not just at the application level. Obviously it could be on the server, could be fraud, could be anything around how the application is used in abuse. Technically, the one I like that I’ve seen in the past is using Microsoft stride, like the card game, where you can sit with the developers, the engineers, and play a game that shows up your threats correctly. And from there you build the application, having all of this in mind, for sure.

LANDON: And then kind of diving into that. You mentioned using threat intelligence and penetration testing in this model. I know the penetration tester creativity that goes into those types of attacks. A lot of times aren’t necessarily what’s happening in the wild, but it’s a lot of creativity and ingenuity that’s really fascinating and amazing and value added. I’m kind of curious how you kind of blend ’cause threat intelligence ideally is supposed to be threat intelligence that’s targeting ideally an industry or even you as a company. So that certainly will rank on terms of priority kind of curious how you combine both disciplines to really be effective, especially in those early days when you’re combating fraud against the platform.

PATRICK: I mean, one of the point is in many of those pen tests, one, the thing that is the most complicated obviously is business logic, right? And fraud comes from the business logic. So when we look at the Threat Intel for that, it’s not just the threat actor that we see in the news mostly. So it’s not just about like those big rent, somewhere state actor that will lock your things, ask ransom, it’s small organized groups that want to make money or even like regular people abusing the system just to save a few bucks. But when the company goes really big as DoorDash or even a few others, those little groups, if they exist in most cities turn out to be a lot of money. So one of the thing that I like even from an offensive perspective in the past was to have specific question that find those abuse case in the development life cycle or in the pen test.

So it’s not just about finding availabilities or understanding the business logic of the app, but knowing where to look to make an abuse that will lead to money issues technically, and like a basic one it’s, it’s not even money, but most system will have a field to send an email. You know, like here’s a threat we found, here’s something that happened in the app in the console. Let’s send an email to the client to even register. It sounds basic. But most of those system, this little form that’s sent an email, the admin saying like you can register to a platform can be abused for fishing. And if you use this as the official email of the company, obviously to do fishing, well, you got an entry point, super easy. You don’t even have to buy a domain. You use their own system to send malicious email to get in. So if you take this example and you apply this to other business logic, I think that’s where the difficulty is. But also on the Threat Intel, that’s hard to… I think it’s something that’s not fully there yet. ‘Cause like I said, it’s not just a threat actor, it’s really about abusing the money.

LANDON: When you think about how to prioritize, what could happen versus what is happening, kind of like going on the same themes that you’re talking about. A lot of times when you’re talking about application security, I think one of the first things that you and I remember this from the early days, you know, NISOS, so I mean it was almost guaranteed that almost every application was gonna have input validation issues. What are some of the categories? And then thinking kind of like OWA top 10 type of categories. What are some categories that whether it’s happening or could happen? What are some of the traditional type of application problems that you typically a lot of times see regardless of the application?

PATRICK: So the injection is always there. There’s always a miss. I’m not saying it’s the most popular right now. Even OS doesn’t put it at the top anymore. So this has to be on your mindset just to verify. And even recently, like we found one that just one developer did not know about it. Maybe junior or not, the knowledge was not there. So like it still happens. But the big one is definitely everything regarding authorization and session management and even more in companies that use microservices or those kind of things or containers, because the main challenge is from the client side, if you got a dozen or a hundred microservice in the back end, how do you know who’s doing which action? It’s really challenging just from a engineering perspective, but from a security perspective, you need something in place to know who’s doing what, the when, where the request come from.

‘Cause sometime you’ll have a request that goes to multiple service and it’s all asynchronous, it’s in the time it could be even 10 minutes later, it have to come back with something else. So losing track of this is a big challenge, which leads to a potential abuse of those systems. So we talk about like GWT token session. There’s tons of attack on those. Obviously like you just put like an empty session into it and things like this, there’s a lot of system that doesn’t validate this correctly or doesn’t follow the protocol like it should be. And even in my pen testing consulting days or statistics were around 80% of all application add authorization issues. So I go and create an object into whatever website. This subject have an ID, you switch the ID to someone else, you gain access to someone else’s data. It still sounds again really basic, but it’s everywhere.

LANDON: And then, so from a defensive perspective, you have the pen test methodology where you throw the kitchen sink and you actually pop that sequel injection. You certainly have an array of different forums that you can have Threat Intel experts go comb to see who’s actually talking about these things. But then of course there’s like an attack surface management side of that issue. And there’s no shortage of external telemetry where you can ultimately see what is hitting the firewall, so to speak, from net flow and passive DNS and those types of things, I guess, like how do you think in terms of leveraging that type of defense as well, you know, with the layered approach and if you’re kind of trying to stand all of this up, kind of how do you pick and choose? ‘Cause any security team is gonna be strapped with resources, right? Like what do you kind of try to prioritize in terms of what tools we don’t need to get into specifics actually, right? But just like overall different types of methodologies, of course, of like how to tackle these types of things, particularly around like the fraud issues.

PATRICK: I mean it’s kind of a Threat Intel, it’s more technical, but you said it like surface of attack. That’s definitely one of the main new market right now on top of the main techniques, even for bug bounties, that’s how to find issues. And it goes back to the number one thing that InfoSec should be about the first thing you need is always your inventory. Most company, what they don’t have is an inventory of their system. So we see the gap right there, but having a system that does the detection automatically for you every day or like life for, like you said, DNS sub domains, what’s exposed, what’s a available is definitely a big input for AppSec and also for possible like everything SecApps and infrastructure security. The other one that is a lot useful on the knowledge is everything WAF. So if you use CloudFlare or any other like signal science and similar things like this, the board of data, like the dashboard you can get from this, if you build rules that make obviously sense for your service, where things goes in the application, what is supposed to happen versus what is not supposed to happen, give you a really good insight.

We discover tons of information by just having some alerts that triggers at the right time for specific attacks or behavior doesn’t even have to be an attack. But seeing like a button net coming in, looking at specific IPS or backend system will tell us that something is coming at the same time. We sometimes see spike in specific brute forcing of attack and spray things like this password spray and all the one that are not stealth, obviously that do, they spam all their lists of ESCR injection and things like this can be detected super fast, but on top of it, this same system is used to go back and find where the fraud happen and see the patterns. So the fraud team used this a lot to see how and how many cases happen of a specific fraud so they can figure out the number of attempt, the number of money back that a specific attack occur in the past or is happening right now. I think those technical point of view is really a good trend intelligence way to do this.

LANDON: Interesting kind of like picking up and just like a basic use case. For instance, if an IP is scanned that that’s not exactly something to jump up and you know, really do anything about ’cause IPs get scanned all the time. But if potentially you can detect and automate, not only when an IP is scanned, but then shortly after followed up with an authentication attempt and having the type of telemetry to see that and then be able to see those types of patterns. I guess you’re saying that’s pretty strong. Would that be a sophisticated intelligence program or are you seeing that done more with an enter overall enterprise?

PATRICK: I see it more on the fraud side because it’s direct value for them. But I think even on AppSec and other parts of security, it’s definitely a big value. Well, the difficulty is that you need to know how your application works to identify those patterns. So you can see patterns happening of things you don’t want to happen. And this can cover parts where you might know that security was not fully in place there. So you have like specific alerts to detect if someone is going after an endpoint that is not supposed to, you can see which endpoint is the most used and all those regular versus in entire patterns is really useful. The difficulty is definitely that it takes time to build ’cause you need to know how your things are working. So it’s not an automatic thing, even though I see a few vendor out there scanning all the APIs and telling you all those statistics, they’re not at a threat level. Yeah. It’s more about just showing you what’s the use that is happening of your APIs. But I think if we could merge this plus the WAF plus Threat Intel, like the whole thing together in a few years, like that would be awesome.

LANDON: What you’re really talking about ultimately there, and this is always the critical challenge within thread Intel is ultimately what is legit and what is a false positive. There can be a lot of noise and Threat Intel. What are the challenges of really understanding those applications to understanding what is rational, what is legit and you know, how do you kind of go about? ‘Cause I mean, I have to assume that you’re really talking to the business there. How do you kind of tackle those types of challenges? And even more so, how do you push your potential partners and vendors to really understand those challenges as well? ‘Cause I mean that any vendor that wants to make their clients successful truly wants to be sought that you look at as a partner. How do you kind of work through those challenges?

PATRICK: I would say that’s the biggest challenge as all at the beginning. I mean, not just at the beginning, just in general, the main difficulty, especially joining a company that is already massive in terms of engineering is that you cannot know everything really fast. So you’re not at the beginning. You’re not at the threat model, even though you can go back there, but you cannot know everything that is happening at the same time and you have to scale back up. So that’s a challenge of the industry, obviously. So having a tool that can do it for you would be helpful. That’s why I’m saying some vendor are scanning everything that is in place to show you kind of an inventory of what’s happening live. So that’s super useful. But the main difficulty is definitely to have engineers in relationship with AppSec engineers to get the old program and knowledge of the application shared as every time that something is changing. Obviously you cannot check every line. You cannot check every PRS that is put in the code, but knowing what is important, sensitive data and things like this would assure that the most important parts are looked at. At the same time, one of the program that we’re pushing, then I think but many companies are doing this.

And I think it’s one of the most valuable for application security is the concept of security champions. So having engineers that are the champions of security in their own team to help you build the application security side without hiring a hundred more people. So technically they are part of your thing, their extension. And by doing this, they can do those, this old inventory. They can tell you what is normal, what is not. So you get at the same time, a human Threat Intel, ’cause they will report the bugs. They will tell you what you need to look at. And even at the last offensive security role that I had, that’s people will just email us like, “Please look at this, I know it’s vulnerable”, super useful.

LANDON: No, that’s fascinating. You know, paring that security champions program. I think that’s kind of where this next question will really go. How do you layer in outcomes to this process? Right? I think a lot of traditional security have a process for implementing controls and having the security champions is a critical part of that. There’s a lot of other outcomes. Certainly we’ve seen security teams that have outcomes to truly blow the cover of the adversaries and going public with some of the information that they discover and disrupting the networks and doing attribution and sharing with law enforcement and sharing with industry and researchers kind of curious how you’ve seen that be successful and like how we think about outcomes within intelligence and the greater security paradigm.

PATRICK: I mean at the security champion side, it has to stay internal. So those outcomes going out, I don’t think it’s usually a good idea, but it’s useful because the engineers are your eyes and you can know where a priority in terms of security should be. So that’s always useful. In terms of most outcome, the thing that is missing, I think, in most organization, is sharing this knowledge between each of them. The main point of this it’s mostly transparency and there’s luck coming up saying like organizations should declare attacks and things like this, but most organizations are not doing this right now. If there’s not a law obviously. And I think that caused a lot of issue because it’s cool to work at a company and doing security for them. But why not? We’re technically all on the same site. Even if you work at a different company, like we should all share the same, maybe not framework, but tools and intelligence and things like this. I know there’s obviously like public Threat Intel feed and things like this, but it’s not a relationship. You know, it’s definitely just a dashboard that tells you, you should look at this thing and that’s finished there versus if you’re in a specific organization market and your competitor is under attack under a specific thing.

Like if we could all know this, that would be awesome. But obviously there’s so many corporate rules and things like this that makes this difficult. So I know that it’s not happening most of the time because of this, but I think we would be so much more in a good place if this was happening. Especially when we look at the news for rent somewhere recently, like you see a market getting attacked and none of them knows, but it’s happening one after the other. They could just say like, “Hey, we went through it, fix this. You should be fine. At least for this specific attack.” You know? And I mean even personally true access here locally, we push for this, we go in the news and talk about this. So it’s a big challenge for companies to do, but some do. We saw one era transport company transit that did an official public post more time showing how bad they were and how they fix it and how proud they are that they did it. And they explain all the details like the technical one, and even I think it was GitLab or another one that someone delete a full database by error. They explain it totally. Instead of saying like something happened and we won’t tell you. So I think the collaboration is definitely something that we need to improve. Mostly when we know that attackers do collaborate, well, they fight together too. But you know, it’s a bit different.

LANDON: Attackers certainly collaborate at a much greater scale than unfortunately organizations do. I think we’ll probably eventually get there and there’s certainly no shortage of policy, your decisions that are coming down on this to be determined how effective that’ll be. But when you think about threat intelligence and vulnerability management and application security and all the communities that those really come together, you can’t have a conversation about these types of topics and not talk about metrics and frameworks that you’ve seen successful. What do you see as successful metrics to say that your program is successful in truly mitigating? Is it fraud? Is it just the controls that are implemented with the engineers? Is it a dollar loss that could be affected? Is it a combination of both kind of curious, you know, what you’ve seen to be successful?

PATRICK: That’s a good question. I mean, yes metrics is definitely one of the most important things, but like I said, fraud is a bit outside of AppSec. So we have different metrics. It’s not the same objective. I think your money is usually a good way to say like you have a risk ears is the amount, but when the organization is quite big, the amount have to be quite big. So it’s not easy to say like here’s is an square injection, here’s is how many million it’s supposed to be. And even if you can say it, it’s really difficult to get the right numbers because it’s person time and things like this that can be involved. So the other approach that I like that we’re putting in place is the abduction rate. Instead of having metrics from here’s the number of vulnerabilities here’s what’s bad go the opposite. Here’s what’s good. Here’s how many teams are adopting this secure framework. Here’s how many teams are using this secure library and getting to know how much you can push security framework technically on the engineering side. So if you got a another team that have all their own services, you can know how much spread out is your security framework between them.

And that gives a positive metrics that have an official value ’cause they get secure. And it’s not just showing how bad they’re doing, which is I don’t think a good way to just go after security and being the police and being the person that says no, instead it’s more a relationship, a positive one, working with the engineering, making their life easier in the end ’cause you gave them a way to make things secure and you say, “Use this, here’s how to use it.” And it’s done. They don’t have to reinvent the wheel. They just have to call this thing. It works. It could be at the server side, it could be at the application level. It doesn’t matter. As long as you can push this on their side.

LANDON: Patrick, that’s a fascinating conversation today. Can’t thank you enough for joining the show today. Thank you for being great colleague. Thank you sir. For the latest subject matter expertise around managed intelligence, please visit us at nisos.com. There we feature all the latest content from NISOS experts on solutions ranging from supply chain risk, adversary research and attribution, digital executive protection, merger and acquisition diligence, brand protection and disinformation as well as cyber threat intelligence. A special thank you to all NISOS teammates who engage with our clients to conduct some of the world’s most challenging security problems on the digital plane and conduct high state security investigations. Without the value the team provides day in, day out, this podcast would not be possible. Thank you for listening.