Podcast Transcription


Christine Izuakor:
Welcome to the Veriato Insider, a biweekly podcast covering some of the latest trends and things to know in cybersecurity. This podcast is sponsored by Veriato, an award-winning employee monitoring and insider threat detection software provider. To learn more about how Veriato can help your company, check out Veriato.com.

Christine Izuakor:
Reporting to you from Chicago today. I'm Dr Christina Izuakor, your host. And our podcast today covers confessions of a cyber security threat hunter, and to dig into that, we have a very special guest joining us. Jason Colyar is one of the best threat hunters that you can find with nearly 15 years of experience in the security industry and a true gift when it comes to finding these serious and often terrifying threats that most people can't. Welcome and thanks for joining us.

Jason Colyar:
Thanks for having me.

Christine Izuakor:
So this week we're covering the topic of threat hunting. I'm going to let Jason do most of the talking here because he has a wealth of experience to share in this space. However, as always towards the end of the episode, I'll wrap up with a summary of three recent cyber happenings and trends that we should all be aware of. Now let's jump right in. So Jason, let's start with the basics. Maybe in layman's terms, walk me through a day in the life, the name sounds really cool, but what do you actually do as a threat hunter?

Jason Colyar:
So first I will acknowledge that it is a really cool title for a professional or a role. And it really varies from day to day. Often time leads are provided by membership leaders and we're supposed to determine the extent of the impact based on what we find in our findings or the lack thereof. Other things that we do are curating of indicators or compromise, or also escalating new finds and new leads to threat intelligence, and or security operation centers so that they can modify their security controls and improve security posture. So it's kind of a mixed bag.

Christine Izuakor:
So you touched on this a little bit, but I hear this question a lot. To be clear, how does threat hunting differ from penetration testing or from vulnerability scanning?


Jason Colyar:
So that's actually a really good question. And so I think that more than there is differences, they really go hand in hand. What I've found is the best threat hunters of the ones that I have met, tend to have experience either in technical capture the flag competitions like CCDC, or also penetration testing skills themselves, either from a previous profession or just as a passion in their own time.

Jason Colyar:
And I think the old adage takes true is, it just takes a thief to catch a thief. And so for threat hunting, what you're looking for is not the run of the mill or the off the shelf, but you're looking for more of the sophisticated and more complex threat actors. And to catch the those you have to be more familiar with the tactics. Like how do you break into a FTP server or how do you break into a web server to even know what to look for or what to look around. And so that's kind of why it really, one of the best things in a threat hunter or somebody aspiring to be a threat hunter could do is start to look into penetration testing and offense of security.

Christine Izuakor:
I like the saying that it takes a thief to know a thief.

Jason Colyar:
Job security, we hate them and we love them.

Christine Izuakor:
So kind of to pivot here, last year IBM reported that it takes an average of about seven months to detect a breach. Another report by Ricotta disclosed that 62% of respondents felt that they were better able to detect advanced threats, which you touched on a little bit earlier too, but within minutes sometimes thanks to threat hunting. So what are your thoughts on kind of the benefits of threat hunting and have you had any experiences or examples that you can share on those benefits?

Jason Colyar:
Yeah, so I think that really we have a second chance with threat hunting, it feels like almost. I think that one of the reasons we have so few qualified professionals in information security and so many positions, is because nerds back in the 80s were like, "Hey, protect your network, look out for your assets." And people didn't really take those warnings. They didn't heed them because they hadn't suffered losses. And so when people started suffering losses in the millions and or billions, now everybody wants security, but a bit late.

Jason Colyar:
But I think threat hunting is our chance to kind of heed that warning and be proactive and be preemptive. And I do think an ounce of prevention is worth a pound of cure. I think it's also a lot more cost effective to start hunting and looking in setting up controls on the front end proactively than it is to suffer a breach, have reputational impact, or liability impact. Let's say you have European clients and so now you're dealing with legal liabilities and to clean that up, and then you still have to set up the security controls anyways, and you still have to do the investigation and go to court as well. And it's going to cost you. So you might as well do it up front while you can and invest on the front end.

Christine Izuakor:
Yeah. So the kind of core benefit is being more proactive when it comes to cybersecurity, right, versus some of the more reactive approaches we've seen throughout the years.

Jason Colyar:
Definitely.

Christine Izuakor:
So for anyone who's listening who either already has a program around threat hunting or is considering starting one, what's a common mistake that you think companies should avoid when it comes to threat hunting strategies?

Jason Colyar:
I think the biggest, there's a couple that I've come across and in a weekly discussion that I have with other threat hunters and threat hunting teams. Some of the biggest challenges we see are hiring people with the same exact skillset. Or bumping them up, let's say you bring up a couple of your star analysts from the security operations center, they might have the same exact skill set and with threat hunting, you really want to diversify and have a versatile team. Maybe you want to have somebody that's really strong in scripting or coding, has some penetration testing experience. You want to have another person that's really good at analysis and triage, or has firewall experience, things like that. And so you want to kind of stack the deck in that way. And I haven't seen that done too often in my limited experience with others. So that's one thing.

Jason Colyar:
Another, I believe is just having poor network visibility. And so that's a fundamental, right. It's just understanding your own infrastructure, where your assets and logs are, and how well you can see through your network to the end points into the users in their accounts and activities. Are we even logging certain things like that? Because you can't catch what you can't see, right? And so a lot of that goes back to just kind of infrastructure 101. So that's a huge proponent too, and it can be a boon or an ailment to threat hunting.

Jason Colyar:
And I think maybe just information seeking, just researching as much as you can on the topic. There's not a lot of information about it and when it is, a lot of information you come across as really marketing and branding and people trying to sell you a product, but it's not actually on the core topic of threat hunting itself for the sake of it. And so just doing as much research as you can. For instance, I think there's a couple of different standup groups for threat hunting where you can kind of listen in and chime in and they can actually mentor you. And so how to stand up a program.

Christine Izuakor:
Yeah. I think what you said about visibility is so true and the fact that you can't protect what you can't see, which is another good point. So I'm curious, when you hunt, like are you constantly finding things, or are there kind of dry spells depending on the season, like what happens?

Jason Colyar:
This question.
Christine Izuakor:
What happens?
Jason Colyar:
It's like a desert out there. So with threat hunting there is, and I know this because I've spoken with a dozen other companies that either want to get into threat hunting or have seasoned teams, and we kind of laugh about it. There are a lot of dead ends in threat hunting. And that's okay, and what you have to do is kind of set expectations and manage the expectations for your leadership and for your organization that hey, they're going to be quite a few dead ends and a lot, because it's a hypothesis that you're chasing down most times.

Jason Colyar:
And just like in science, not all hypothesis pan out. People work for decades to get a breakthrough. And so it's similar in a smaller scale with threat hunting. You may look through six to 10 different investigations before you find one thing and maybe it's not huge. Maybe it's a huge deal and maybe it's, , just some commodity malware or something that just went under the radar. So there will be a lot of times where you'll have dead ends, but to set the expectation for your management and for your team to not be deterred by that, because you can't find if you don't try, same thing. It just takes repetition and work. And as you mature your program, you'll be able to find things more quickly and more accurately.

Christine Izuakor:
Yeah, that makes sense. So to expand on that a little bit, because we've talked about the manual work that hunters have to do quite a bit, but why have traditional methods been missing the mark, if you think at all that they're missing the mark, and what are some of your favorite kind of next generation or forward thinking threat hunting techniques that can help address some of those gaps?

Jason Colyar:
Yeah, so I think that some of the more fledgling and grassroots ways of doing it are around indicators of compromise or file names, hashes, things like that. And those are still fine places to start. You just don't want to stay there for years. You want to stay there for a year or so. And I think that one of the reasons that they are just not as effective, why some of the AV is not as effective is because it's been out for a long time. A lot of these vendors, these big name off the shelf vendors have been around for a long time.

Jason Colyar:
And while they update and rehash their products, threat actors are becoming very sophisticated. And they're professional criminals and they're aware of these products, and so they do everything and anything they can to disable them, to get around them, and to evade them. And they're very, very good at that. And so while we may be growing at a gradual pace on the white hat side, on the black head side, they're literally developing new things all the time to counteract and to defeat that. And so we have to get as cunning and as innovative, and a lot of the off the shelf products just aren't doing that.

Christine Izuakor:
I've done a ton of research in this space just around the difference that things like artificial intelligence, user behavior, analytics, and those kinds of things are making in this space. What are your thoughts on those?

Jason Colyar:
Well those are huge. Definitely user behavior analytics and AI are both good. One of the great things is they can utilize and leverage metadata to find those heuristic based findings. And that's really what we're looking for. We're trying to get away from the file names and the hashes and get more to how did this person log in from Rhode Island and China in the same day, two times. You want to be able to see that.

Christine Izuakor:
Right.

Jason Colyar:
And with AI and some of these next gen products, you can do that quickly and set up alerts for that so that it's always looking for that. And so things like on Outlook or Exchange, you can also do impossible travel where it tracks the logins like that, or access to, let's say finance data from a person that never accesses finance data and works with the mechanics, and things like that you might not see because you won't know to look for it. Your signatures won't be tuned for it. Neither will your AVS, but the AI can be set to in the UBA or the user behavior analytics can be set to look for those things.

Christine Izuakor:
So you talked about somebody logging you on, let's say from Rhode Island and China at the same time, which clearly is not good. So for kicks, give us your craziest or most interesting threat hunting experience.

Jason Colyar:
Oh goodness, there's a few, there's a few. One was a pretty nice case, so it was back in 2013. And so just by nature for a long time I didn't know what threat hunting was. And so I would just do proactive analysis, not knowing that there were even terms for it. That's how I pretty much opened up every role that I took. And so at one point I was an agency, a government agency actually, and I started to see some pretty suspicious activity. And I went and asked my supervisor, this is week one, actually, maybe like day three or four. And so I asked my supervisor, I say, "Where can I find the raw logs?" And he says, "What do you need that for?" And I was like, "For evidence to prove my hypothesis." And he's, "You don't need that." And he said, "Sit down son."

Jason Colyar:
So I go back to my desk and I just kind of toil away for a week or so. And later some directors came by, and unlike my supervisor at the time who is brilliant, I think he was just having a rough day. They kind of gave me time, they gave me their ear and looked at my research and as I started to kind of connect the dots from one piece to another in different pieces of research I pulled together. They said, "Sure enough, yeah, it looks like that's a worm infection." And I had spoken to other teams too and they kind of just shooed me away. This new guy, what's he talking about? And it sure turned out to be a compromise of more than like 250 hosts within that network. And so a month or two later we got to cleaning it up. And I refrained from saying I told you so. But that was a pretty ironic and fun win. And so it goes, right. So it goes, that's kind of how threat hunting was before they kind of titled it and everything.

Christine Izuakor:
Oh yeah. And that even I think drives the point home about needing to have the visibility, because if you needed logs and things to do your research, which if at that point you didn't even have that, then you wouldn't be able to prove it out.

Jason Colyar:
True.

Christine Izuakor:
Cool. Well thanks so much for sharing all of that insight. I think it's been very interesting just to learn more about your world of threat hunting. And now it's cyber trending time. Here are three recent cyber happenings and trends that we should all be aware of.

Christine Izuakor:
First up, even the beauty industry is feeling the sting of cyber attacks. According to the Info Sec magazine, Estee Lauder is the latest big name brand to suffer a mega breach after a researcher discovered 440 million records, including plain text emails, exposed through an online database. Now, Jeremiah Fowler from security discovery found this back in January and claims that the unprotected database exposed not only those few hundred million records, but also disclosed quite a bit about the middleware used by the company, which raises its own concerns. Estee Lauder has been praised as acting, quote unquote fast and professionally, to block public access to the database on the same day of discovery. So quick action, super important there.

Christine Izuakor:
Next up, the latest phishing attack victim, the Puerto Rican government lost $2.6 million to scammers after an employee made a wire transfer for a payment sent to what appeared to be a genuine bank account last month. They later confirmed that the account was fraudulent, and they've since changed their procedures for processing payments to avoid incidents like this in the future. However, no news on whether they've been able to recover that money or find the culprits behind the attack. The incident is still currently under investigation.

Christine Izuakor:
Lastly, as reported in the Wall Street Journal, a fraud case in South Carolina has many concerned about this first in its kind scheme. A small tech company, Micfo is facing 20 counts of wire fraud after allegedly deceiving the American Registry for Internet Numbers, which is a nonprofit that assigns IP addresses to devices. Now according to reports, the owner of this company created fake shell companies to trick the nonprofit into dishing out about 800,000 IP addresses, which he then turned around and sold or leased to other companies. The fraud scheme has been valued at $14 million and the accused corporation has pled not guilty. This case is also currently under investigation, so lots of activity going on in the cyber fraud realm right now.

Christine Izuakor:
And that concludes the Veriato Insider podcast for this week. This podcast is brought to you by Veriato, an award winning cybersecurity company, recently recognized by Gartner. Their solutions are anchored around four core pillars of cybersecurity protection, including employee monitoring and web filtering, insider threat detection, employee investigations, and ransomware support. To learn more about how Veriato can protect your company, or if there are questions you want answered in the next podcast, visit Veriato.com and send the team a quick note.

Christine Izuakor:
Thanks for tuning in and a special thanks to Jason for joining us and sharing so much expertise on threat hunting.

Jason Colyar:
Thanks for having me, it's a pleasure.

Christine Izuakor:
I'm Dr Christina Izuakor, the CEO of Cyber Popup, and it's been my pleasure to share these insights with you. Until next time, stay safe and secure insiders.


Listen To Podcast