Security Confidential S12 E3 Eric Allard

MANOJ (00:01.141)
Hello everyone and welcome to another episode of Dark Rhino Security’s Security Confidential. Today we are honored to have Eric Allard back. He’s a veteran of this show. He’s an entrepreneur, if you guys don’t remember. He’s a CTO of Soos Software. And he has 15 years plus of experience in leadership, business strategy, and software team transformation. His timing on this show is very appropriate as he’s gonna…

talk about S-bombs and we’re gonna let him get into what all that is. And so this is going to be somewhat of a primer, everyone on the whole S-bomb topic. Um, so let’s, without further ado, Eric, thank you for being here on the show and you have the floor, sir.

Eric Allard (00:47.65)
Thank you. Thank you for having me back. Yeah. I am going to give a talk. It has a scary name. I’m going to preface this by saying I gave this talk before at a healthcare conference earlier this year. And when you build these talks, they want you to submit a title and you have to know what you’re going to do long before you give the presentation. So maybe there’s a bit of a clickbait title, but I assure you this is for people that…

don’t know anything about software, bill of materials, what an S-bomb is, if you’ve never heard the term, I will demystify that as we walk through this presentation. So.

MANOJ (01:24.193)
That’s right. And why all our audience should care here is if you folks all remember the solar winds attack, the dragonfly, Kaseya, Code Cove, all those have been S-bomb related attacks. So it’s about the software supply chain. And so stay tuned, please. Go ahead, Eric.

Eric Allard (01:47.086)
Great, I will do that. I’m gonna break this down and make it real simple and make sure everyone understands why you care about SBOM. So I mentioned it has a scary name, how government regulations are driving vulnerability, transparency, and accountability. SBOMs are about vulnerability, transparency, and accountability, but what I want to make sure you understand is I’m not an expert on government regulation.

Manoj said I have a background in software engineering. I started as a software engineer. I’m an engineering manager, I’m a product manager, and the government has interesting reasons for wanting a software bill of materials, but none of us needs to be experts in regulations to get to the bottom of why that is. And that’s what we’re gonna dive into here.

Eric Allard (02:39.094)
First of all, nobody develops anymore without using open source software. There’s many statistics out there about the prevalence of open source software. This, these stats are from GitHub’s Octaverse report from 2022. And some people see big numbers like 90% and they say, that’s fear-mongering. Some of this software is never called. You don’t invoke a vulnerable method or maybe it’s not configured in a vulnerable way. And…

MANOJ (02:53.502)
Wow.

Eric Allard (03:08.242)
I think you’ll understand why, even if that’s the case, open source is everywhere and you still want to have a good understanding of what’s in the application you’re building, or perhaps what’s in the application you are bringing into your organization from a vendor, because you’re gonna bundle it up and use it in another product, or just run it as part of the tool set you use in your organization. I mentioned I gave this talk at a healthcare conference. And…

I wanted to point out that open source is everywhere and it’s in healthcare news. Pick any industry. If you’re in the financial industry or the automotive industry or it doesn’t matter. Open source is used by all industries in any possible software application you can think of. And that also means things like industrial computers even, machine computers, PLCs. So

Doesn’t matter if you’re in healthcare, tech, finance, open source is a big deal.

MANOJ (04:09.053)
Now, Eric is open source in embedded systems in healthcare.

Eric Allard (04:14.618)
Oh, surely. Yeah, absolutely. I mean, it’s in just about any connected device that you can think of through things like TCP IP stacks. It’s in it’s really quite comprehensive. So I can without question say that it’s in every industry, including surely healthcare. But all of them. All of them.

MANOJ (04:38.725)
So those heart rate monitors and respirators and what have you.

Eric Allard (04:42.598)
insulin pumps and anything you can think of. It’s all powered by software these days, you know, to miniaturize and make things cost efficient. A lot of that grunt work is done in code, not with hardware. So why is that a problem? Well, it’s not per se, but there are things people don’t often think about that come along with using open source software. Legal.

risk is one consideration. I picked out some recent decisions, semi recent, these were a couple of years ago, but I tried to find ones between companies that people have probably heard of. So VMware, MongoDB, Couchbase. You’ll even see that VMware has been on both sides, they’ve been a plaintiff and a defendant. So when you don’t understand the licensing of the open source software you’re bringing into your organization,

there is legal risk. And you might not know that the thing you’re using has some sort of a A-GPL license, for example, and that may compel you to have to open source your own source code. There’s a lot of restrictions and considerations when you’re using open source software. So understanding how it’s licensed is important so that you don’t get sued.

Big money and big disruption to your business if you had to rip out some key component of your platform because you weren’t really using it in a way that was acceptable to the terms of the license of that software. And I mentioned this one first because I think it’s forgotten about because everyone thinks about the more scary stuff like vulnerabilities. Last year.

MANOJ (06:32.009)
Now, Eric, just real quick question. Sorry to interrupt on that one. We all know everyone kind of clicks past the accept user terms and agreements and jams through it. As part of the, you’ve been a part of software development life cycles for a very long time. Is there a place at the beginning where someone should do a legal review or does that ever really happen? Just curious.

Eric Allard (06:36.659)
Yeah, no, go ahead.

Eric Allard (06:43.565)
Yeah.

Eric Allard (07:02.594)
I think it’s more and more common. It’s almost always going to come up nowadays in any sort of due diligence if you’re going through an acquisition. And part of what we’ll talk about sort of later on about what can you do about this? The goal is really to automate the awareness that these things exist. So when you bring a new dependency, a new library into your organization, because you’re gonna use it for an application.

Making sure that how it’s licensed is acceptable to your business should be part of that process. Licensing for software is a tricky, ugly, gnarly thing. The way someone might specify that license is varied. It could even be buried in some comments in a file that you may not even be looking in. So there’s a lot to consider.

But most of the time, things are pretty standard. There are agreed upon ways to share that license information. So with tools, you can pick up on that and even set rules to prevent certain license types from being brought into your application.

MANOJ (08:17.153)
Okay. And I just got a curiosity, how much development time would increase if you started lessening the use of open source because of these legal considerations?

Eric Allard (08:18.412)
All right.

Eric Allard (08:32.042)
Um, I don’t, well, here’s the thing. I, I don’t, but I have a different answer. I don’t think you have to, because there are so many options. If you had the visibility and awareness and were being kind of cognizant of that as part of the development process, you could simply choose something that met the needs of your business and, and really avoid that legal risk. You know, there’s, there’s often alternatives or.

MANOJ (08:33.317)
If you have an answer, I mean, just…

MANOJ (08:45.281)
Okay.

Eric Allard (09:01.514)
you may be deciding to go look for a piece of open source code and not find anything with an acceptable license term, but decide, well, it’s not a lot of work for me to accomplish this thing. I may write this portion myself instead of relying on a third party or an open source application to do it for me. And there’s other reasons to make those considerations beyond just license risk.

I actually have an example on this slide. If you’re seeing this 26,000 CVEs reported in 2022, this is shifting a little bit from legal risk and looking at security risk. Number three here, I listed Faker.js, Colors.js. These were two JavaScript libraries. If you use JavaScript in your applications, there’s a good chance you’ve heard of these. You may have even been affected by them. The maintainer…

MANOJ (09:36.448)
Yeah.

Eric Allard (10:00.698)
of these libraries, they intentionally corrupted them, you know, to kind of make a statement about compensation for open source maintainers. But regardless of their intentions, this would effectively DOS the person’s application. And I’m not saying they’re not valuable libraries, but they were relatively simple. And they’re things that someone probably could have.

implemented themselves pretty easily. So going back to your license question, you may find this colors library and it does some simple thing and it’s going to save you time. You’re not going to have to write that code yourself. But if it weren’t licensed in a way that you could accept in your business, well, you can find that out at the beginning of the development process and make the decision to not use it and write it yourself.

And that’s not always going to be an option, but it’s certainly a possibility if you know, if you have that information, right? Log4Shell, that was the series of CVEs that were related to Log4J. So everyone’s probably heard about that at this point. That was a really big deal. I don’t want to spoil things, but we’ll talk about that again later in this presentation. Then Spring4Shell along with same lines, another Java.

exploit like exploitable Java piece of Java code. Equifax, big deal. Everyone’s heard of this. I think I still probably have credit monitoring from, from five or six years ago. When this first went down, I think. You know, that 1.4 billion down arrow is my way of saying, and that number is surely bigger, probably closer to 2 billion. 1.4 billion was the amount that Equifax wrote down as losses.

that they attribute to the breach. And I’ve worked on big legacy applications and I have some empathy for the people that were probably maintaining this code base. It’s a hard job to do, but the Struts vulnerability that was ultimately exploited, there was a CVE published in March of that year, March of 2017. It was given a CVSS score of 10. That’s…

Eric Allard (12:23.23)
in the CVSS system, the most critical. But the exploit didn’t happen for about two months later. So there was some time in there for this to be remediated. Granted, there’s probably a lot of instances in that big of a code base, but if they had taken this more seriously, and I can’t say for sure that they didn’t, but if it had been taken more seriously, could they have prevented the breach altogether? I think they probably tried, but.

knowing what we know now, this is the kind of thing that you may have to take some really exceptional steps to protect yourself. And knowing about these vulnerabilities is the fundamental step to being able to do something about it, right? You can’t really manage what you can’t monitor. And then finally, I’m going to go back even further in time, and then we’re going to get into SBOM. So I promise we’re getting there. But I had to…

MANOJ (13:20.645)
Oh, this is good. These are some of these are new to me.

Eric Allard (13:23.674)
Oh, good. Well, this one’s incredibly interesting. I had to learn about a case study talking about this tragedy that happened in the 80s when I was in college. And this was a device that was software controlled, and it was unfortunately maiming and even killing some people by improperly irradiating them. So the operator would think the machine was in one mode.

and there was a defect in the software and they had omitted hardware fail-safes. So the machine could be put into a really dangerous state and they were actually cooking people. And you may ask, well, what, this was not open source. So I’m gonna caveat that in the next slide. This wasn’t open source, but the reason I bring it up is there’s a number of similarities in some of these.

MANOJ (14:06.793)
This was open source.

Eric Allard (14:19.874)
disasters, be it Equifax or something like this, that actually caused loss of life. And that is, there’s a certain amount of hubris, misjudging of risks, a lack of required process controls. All of these things need to be considered to prevent a tragedy. And you can imagine, as you asked at the beginning, well, is open source or being used in devices like this today? Yes. So I kind of want to draw.

everyone’s attention to the fact that no, in 1985, it wasn’t using open source software, but surely today’s version of this machine is. And we’ll hit on that again a little bit later when we talk more about why you need to be thinking about software building materials if you work anywhere that kind of has a software as a key part of the business. So what’s the solution? You can probably guess.

We’re talking about an SBOM primer and the title of my talk had SBOM in it. SBOM is a big part of the solution. And by itself, a software bill of materials is just a text document. It could be JSON, it could be XML, but I have a little snippet here on the screen. It’s very straightforward. It is even human readable, but it’s also machine readable. The idea is this is something that could be ingested with another software.

software application and you could learn some things from analyzing an SBOM. So the vulnerabilities may be directly in the software bill of materials. They may be provided separately in another format, something like VEX. We won’t get into the weeds there, but at a high level, just think of it as an ingredients list or just what it sounds like, a bill of materials. If you were to get a shipping container from overseas,

There would be a piece of paper in the old days, and nowadays I’m sure it’s electronic bill of materials, but there’d be a piece of paper and it would say, you know, there’s 24 cases of air conditioners in this shipping container. And this is the equivalent for software. So let’s put software bill of materials into terms that I think will help you remember what it really is forever. It’s kind of a pneumatic device. And I like to use this.

Eric Allard (16:47.47)
to get people on board with what we’re talking about. So let’s pretend you were gonna make a drink. You were gonna entertain some friends or you just watched Tom Cruise in Cocktail and you’re spinning bottles and you’re gonna make a really fancy looking drink. You have a great recipe. You see we have, there’s some bitters in there. It has to have that cherry to make it a Manhattan.

MANOJ (17:16.541)
Oh, is this your recipe, by the way?

Eric Allard (17:19.938)
I don’t even remember where I got this. I think I found it on Wikipedia, to be honest, or somewhere. But let’s pretend it is my recipe and it’s really unique and I’ve learned the art and there’s just something really special. So what am I gonna do with that great idea? Well, obviously I’m gonna sell out and I’m gonna call up Anheuser-Busch in Bev and I’m gonna tell them about the canned Hatton. And I said, you know, these canned alcoholic drinks

MANOJ (17:22.217)
Hey.

Okay.

Eric Allard (17:49.006)
It’s all the rage. What am I gonna need to know before they even consider selling this and distributing it? Well, surely they’re gonna need to know the ingredients list. And I might think, well, that’s not a problem. We started with that right here. Here’s the ingredients list. There’s some whiskey and vermouth and bitters and the cherry. But if you take a step back, you need to…

think, hold on a minute, do I really know all the ingredients? And the punchline is you don’t. I actually went and found a 40 page college study. They used it, they had this as sort of a case study probably for chemistry students or something. And they published, here’s the most common ingredients for a sample of maraschino cherries. And cherries isn’t surprising.

But some of those other things you probably didn’t expect to see, right? Sodium, benzoate, potassium, sorbet. But those are, yeah, but these things are in there. Okay. So if I went to, if I went to AB InBev and said, no, there’s only one, two, three, four ingredients, that’s not quite true. That’s not at all true. In fact, these sub-ingredients really do matter. And there’s, you need a way to not only.

MANOJ (18:50.973)
Sulfur dioxide?

Eric Allard (19:14.154)
You need to make sure you know that, of course, but you also need a way to disclose that. And I propose you need an F-bomb and, okay, you need an F-bomb. You need a food bill of materials. Now in the real world, we call this the nutrition facts label, right? Everyone’s seen this on any packaged good that you buy at the grocery store. There’s a nutrition facts label. Even on those canned.

MANOJ (19:22.657)
That’s wonderful.

Eric Allard (19:41.778)
spirits or at least on the case but surely on the can too. There’s a printed nutrition facts label and it’s going to tell you all the ingredients and that includes the sub-ingredients. You know, if you go look at a nutrition label, you’re going to see some parenthetical entries and those are all sub-ingredients for some bigger component in that food. So you need a full and accurate disclosure of what’s in your product, including things you know to be dangerous.

You have to tell someone if there’s nuts in that product, or even if it was processed in a facility that included nuts, right? You could make someone sick or even potentially kill someone. And then there’s other common disclosures on a nutrition label, right? The country of origin. Where did this thing come from? Are there any other terms? You can’t resell this. It was sold at Costco and it’s meant not for retail sale, it’s a wholesale thing. Maybe Costco is a bad example.

Imagine a wholesalers where they would get their products for distribution. So let’s just tweak a few things on the slide. And notice I’ve highlighted that in the lower left corner here, you need a full and accurate disclosure of what’s in your product, including things, you know, to be dangerous. I didn’t change that at all between the food bill of materials and the software bill of materials, the ingredient list. It’s all of the software dependencies in your product, including

transitive dependencies. Those are the sub-ingredients in the software world. So I may go out in GitHub and find some application that I wanna use and I depend on that. Maybe it’s an imagery sizing program. Well, I may depend on that imagery sizing application. Meanwhile, that application itself has dependencies and those dependencies have more dependencies. So there’s a giant tree that sort of branches out

that one thing that you depended on. And we need to be aware, just like with the food example, of anything that could be potentially dangerous. So instead of a nut allergy or shellfish that we’re concerned with, we’re concerned with software vulnerabilities. So we need to say that this dependency has some known vulnerabilities associated with it. And similarly to the country of origin or the not for resale language,

Eric Allard (22:08.706)
there’s probably, there almost certainly is going to be some licensing information that accompanies it. So that’s the equivalent of those disclosures in an S-bomb. Oh yeah, stop me whenever.

MANOJ (22:19.689)
Now, can I ask a quick question here, Eric? Is there, is this not a problem that would really benefit from blockchain?

Eric Allard (22:33.242)
Um, I have had discussions with people that are, that have talked about sharing as bombs and using, uh, distributed ledger as a, as a mechanism of storing that and potentially using it as a way to federate access. Um, I think fundamentally though, this is all stuff that works really well with classic databases. It’s basic relationships.

Um, so there’s, there’s perhaps some novel applications where blockchain or distributed ledger types of technologies do get used here in the future, particularly for things like, imagine there was like a global registry of software and anyone could independently verify here’s the S-bomb for this particular version. Um, but in we can, this could be a great, uh, post presentation discussion.

MANOJ (23:07.937)
Okay.

Eric Allard (23:32.918)
The industry is still pretty skeptical. There’s a lot to be figured out about, how to trade and share these things and how openly that would occur. As you can imagine, there are some pretty big companies that are using S-bombs already and they’re starting to trade these and they’re hesitant about how this might be protected. So there could be opportunities there, but I think that might be a…

MANOJ (23:59.617)
Okay.

Eric Allard (24:01.75)
a little further down the road, if it ever ends up making sense, yeah.

So let’s say we had that software bill of materials, right? And now we’re kind of all in the same place. If you’re thinking of it as that ingredients list, or even better, the nutrition facts label has certain fundamental information. How does it help you? Well, I led in the beginning and I said, I mentioned regulations shaping the way S-bombs would be used for accountability and transparency. There have been some…

legislative or executive actions like the executive order a couple years ago. There’s been some more recent things. It’s not listed on this slide, but actually if you look very at the very bottom, Section 3305, this is the ensuring the cybersecurity of medical devices, which is what this headline is referring to. So new medical devices looking to get FDA approval are going to have to…

generate a software bill of materials to get through that approval process. So there are industries that are starting to expect software bills of materials. Being in the cybersecurity space and making tools that help with SBOM, I hear from our clients that this is something that is being expected and asked for more frequently. But you know.

because the government says so or wants you to isn’t a great reason. So I’m leading with that, but I think there’s other benefits that we really should talk about, not just the because you’ll probably have to do it for legal reasons someday. I think there’s better reasons. SBOM help your business. If you’re building an SBOM, you’re able to share that information with your vendor.

Eric Allard (25:57.326)
To have the information required to build an SBOM means you’ve learned a lot about the software that you’re using. It allows you to manage those dependencies and fix issues. If you know there are dependencies in your application with known vulnerabilities, there’s a certain awareness and ability to plan that fix, get that into your development process. And then you’re gonna be able to also continuously monitor.

your software and be alerted when there’s new vulnerabilities. So you may have a certain what’s the footprint of open source software in your application and there may not be any known vulnerabilities and that software may be Static you’re not actively developing it. That doesn’t mean tomorrow. There won’t be a new vulnerability discovered that you may need to do something about so

I kind of have a whole other talk track I can go into about SBOM and the discipline that helps you create an SBOM. One of those is software composition analysis and how that works within a development environment. But at a high level, just kind of take it by word that the process of creating an SBOM enables a lot of organizational benefits, particularly with risk and…

risk mitigation and compliance and development work. If you want an interesting example of how dependencies in your application can get out of control really quickly, if you look at the, I went and looked at the simple spring starter web project. So this is meant for you to use as a jumping off point to build an application, okay? Before you even added a single line of your own code,

the simple spring starter web project has already pulled in 63 dependencies. Okay, so they’re not all on my screen, but just imagine that tree and how big that might be. And you haven’t even put any of your own code. But all of these are potential vectors for license risk, vulnerability risk. So just think about that the next time you go find something. There’s a lot there that you don’t see below the surface. The web test. This

MANOJ (27:56.436)
Yep.

Eric Allard (28:15.63)
this archive spring boot starter at the very top brings in then relies on many, many other pieces of software.

Eric Allard (28:25.998)
Think about zero days, okay? We talked about Log4Shell in the beginning. Alyssa Miller gave a great talk about understanding S-bomb. It was probably a year and a half ago, maybe two years ago at CircleCityCon. And she said, hey, when Log4Shell came up, we had an idea within hours, the complete inventory of all those places where we were using Log4J. Without a tool in place,

That’s an incredibly big manual effort. I remember meeting someone, I don’t remember if it was RSA or that Vive Healthcare Conference, but I said, hey, what was your experience when Log4j was first known to have some pretty big exploitability issues and they said, well, we didn’t have a tool and it took us weeks to find all the usages of Log4j and that wasn’t just me, that was like a team of

for people. So this could be a very big gnarly task. And you could have to look for another one tomorrow, a different one. So it’s not a manual, this is not a manual undertaking as you saw from this tree. And this is only the most basic example. You need tools that are going to help you solve these problems. And As Bomb is a great way to do that. When I gave this talk.

We had just published a public database for software builds and material. At this point, we’ve actually published about 60 million, almost 61 million SBOMs. Any open source package that’s been put out on the package managers, you can go to sues.io. We have some ways to lead you into our public SBOM database. And you can download a software build material for something you might be using for free. But…

I think the important part is to think about how an S-bomb can help your business and to think about how you might implement tools that help you generate these in real time. On every commit, you look at the dependencies in an application and see if there’s known vulnerabilities. And then when your client comes calling for the next Log4j and they say, oh, geez, I’m using your product. Does it rely on Log4j? You’ll have an answer. Hopefully it’s no.

Eric Allard (30:50.594)
but there’s a good chance it’s yes. And you’ll be able to understand quickly how much does that affect your product. You can be proactive instead of reactive. You can actually reach out to your clients and say, we know this zero day happened, we know we use it. Unfortunately, we are vulnerable or we’re not vulnerable because of the way we’re configured or we are vulnerable, but if you put this firewall rule in place, you can mitigate some of this. It gives you a chance to not be surprised.

not be manually trying to dig this up on your own. And everyone likes to say shift left. Well, you can be doing this on every building knowing early in the process, a lot more about the vulnerability risk, license risk of the software that you’re building. So that’s the primer. And now I think, Manoj, I think you probably may have some other paths we go down, but.

Hopefully that gives a basic understanding of what software building materials is.

MANOJ (31:52.588)
It absolutely does. But it raises a lot of questions. I don’t think there’s any really solid answers that we have right now. I mean, thank you, first of all, for giving us that little bit of a presentation so we understand a lot more about this topic. But in listening to you, one of the things that caught me, you know, this whole embedded system, especially when it’s in hardware.

Eric Allard (32:17.527)
Mm-hmm.

MANOJ (32:19.721)
That’s a real problem because if we find a vulnerability like a log4j type thing in a CT scanner or in, there’s so many life critical things like the control system of a locomotive or what have you, it’s a non-trivial exercise to update all those things. And then the question is if we update it,

Eric Allard (32:36.898)
Mm-hmm.

MANOJ (32:49.509)
are we introducing new vulnerabilities that we’re not aware of? And then, you know, another item that comes to mind in the same, at the same time, is because we’re using open source, and by definition, open source is open. So, just like you had pointed out, where someone had put in, in Faker.js, or, you know, had intentionally created vulnerabilities.

Eric Allard (33:11.479)
Whatever.

MANOJ (33:15.893)
Who’s to say that those don’t already exist or someone doesn’t maliciously add vulnerabilities to a new version of open source. And now we updated our embedded system with that patched one thing, but we created a whole nother source of problems here, right?

Eric Allard (33:32.846)
Yeah, you hit on a few really good topics that come up a lot. One, so you mentioned the embedded devices. And I talked about FARAC 25, the device that tragically hurts and killed people in the 80s. Yeah, so surely the FDA hasn’t forgotten that, right? And they’re one of the first kind of trying to compel people to.

MANOJ (33:47.933)
Yeah, the radiation machine, yeah.

Eric Allard (33:59.554)
have a better understanding of what is in this software as part of the approval process. Now, that’s only for new devices. So as you mentioned, there is an enormous base of deployed devices out there that are gonna continue to be out there for a long time. And this doesn’t fix that. And that’s very difficult to go create a software bill of materials for many of those things. The source, who knows some of the stuff, it may even be difficult to get ahold of and determine what that is now. That’s a real…

MANOJ (34:27.966)
Right.

Eric Allard (34:29.581)
That is a real problem that nobody knows the answer to yet. I like to talk about it though, in more of a software engineering practice and understanding of the lifetime of software. So if you were an IT person and you, maybe a few years ago when we weren’t doing everything in the cloud, you had racks and racks of physical servers.

There’s only a certain useful lifetime of that hardware, right? Three to five years. And you have a whole program to life cycle that and move it out of commission, decommission it.

MANOJ (35:03.777)
Correct.

MANOJ (35:13.633)
But Eric, there’s a ton of things that don’t, that are mission critical. You look at all the mainframes, number of AS-400s that are still out there. Yeah.

Eric Allard (35:21.546)
No, I mean, I think IBM just released a new version of ZOS, like a couple of weeks ago, I saw something like.

MANOJ (35:28.769)
And you look at, you know, all the missile systems and whatnot on an F-18 Hornet are still programmed off what we remember as UNIX, you know, the VAX VMS operating system running in emulation mode. So not to challenge you on that, but there seems to be still a lot of plethora of things out there that are mission critical that are…

Eric Allard (35:35.223)
Yeah.

Eric Allard (35:43.02)
Right. Yeah.

Eric Allard (35:54.782)
No, no, no dispute, no argument for me. And I’m not even, that is a giant part of the challenge. I think what I’m saying is there needs to be also a rethinking about software in general and how do you life cycle software? Just because it can run indefinitely, you know, it doesn’t mean that it should. I think there needs to be more thought put into the upgrade path.

and the maintainability of software, not just after it’s first released, but for its entire intended lifetime. Because you’re right, how do you change, how do you do an upgrade to a piece of software in an invented device that has to go through any number of validations, government approvals, you know, a medical device can’t just be updated.

process again. So there’s so much in the software life cycle that has to be, I think, modernized. And that’s going to be a big part of solving any of these problems, not at all touching the deployed base of things that are already 30 years old.

MANOJ (37:12.821)
You know, one thing that comes to mind, and this is totally from a tangential industry, and I’ve been removed from it for over two decades, more than that actually. But like when you look at the aviation industry and you look at engine control software, it used to be, and I don’t know if it’s still true today, but for the longest time on an aircraft,

Eric Allard (37:29.741)
Mm-hmm.

MANOJ (37:36.073)
they used to deliver the engine control system software updates on floppy disks. I don’t know if people still remember what those were. And even when USB drives were ubiquitous and you could do it, but the whole, and they could have enabled downloading from the web, but they, the FAA, I’m assuming the FAA probably wanted a chain of custody that said, okay, where did this, did this originate from the OEM?

Eric Allard (37:41.102)
Thank you.

Eric Allard (38:04.674)
Mm-hmm.

MANOJ (38:05.765)
is it certified from the OEM, and it’s clear of all known nonsense, and only then could you actually upgrade the engine control software on a aircraft that we all fly on, right?

Eric Allard (38:15.135)
Mm-hmm.

MANOJ (38:22.225)
Is there a lesson to be taken from that and applying maybe, it’s a sneaker net approach, but for applications and embedded systems where we say, look, we need to update this, but we’re gonna really air gap them and we’re not gonna allow any kind of connectivity. I know that generates a whole nother plethora of issues.

Eric Allard (38:43.618)
Mm-hmm.

MANOJ (38:49.577)
But for mission critical things, is there any real way around it? I mean, I…

Eric Allard (38:55.122)
You know, I mean, these are questions that people are grappling with. Again, we talked about all industry. Think about energy as well. You know, there’s huge advantages to having the control systems be connected because you can do lots of adaptive load. You know, I’m not an expert in that industry, but you can imagine, like, shifting load to different segments of your power grid. All of that is…

MANOJ (39:04.591)
Oh yeah.

Eric Allard (39:24.01)
requires a certain amount of interconnectedness, but at the same time that introduces a substantial new element of risk. So if it is completely disconnected on its own isolated network, that makes a lot of challenges for what sorts of capabilities it can have. And it’s a big trade off. I don’t have any good answers for that.

MANOJ (39:47.017)
Well, yeah, well, yeah, it is, but maybe we need to. And I think it would take government regulation to do this because no private industry is going to do it on their own limit the capability of products that is, they just won’t that’s totally against profit behavior. But you know, if you look at

Eric Allard (40:00.45)
Hmm

MANOJ (40:12.381)
An aircraft, it only has one mission, fly you from point A to B safely. And anything that interferes with that mission is not allowed regardless of the capability involved. Same thing if you’re, you know, if someone’s unfortunately going for radiation treatment, they should, they already have enough troubles in their life from whatever unfortunate diseases got them. The last thing they should worry about is, is my treatment gonna kill me? You know?

Right, so those are, I don’t have the answers, but I think it’s a tough one. Like it’s at least should be in the conversation stack. The other thing is what’s your opinion of actually, there was a time when we didn’t use open source software. Is there a play to get source code for a lot of things?

run it through the wringer and then make it your own and abandon the open source from where it originates. Again, that probably has some lawyers probably saying, uh-uh, that doesn’t work, but.

Eric Allard (41:25.918)
I mean, you certainly, a lot of licenses do allow you to do it and you could fork a project. I think it’s not impossible. And I mean, a lot of things move in a cyclical nature, right? I mean, the cloud is kind of the main, the new mainframe, right? I’m sharing a computer. So you could fork a big project and decide like, we’re going to keep it and maintain our own fork from henceforth. But everything you started with at…

MANOJ (41:40.592)
It is.

Eric Allard (41:53.782)
that point in time at least was known. So somebody already knows a lot about that application and the amount of knowledge, the amount of engineering prowess to then take that on and be an expert in that is probably quite substantial. So maybe you have a team that’s already well-versed in maintaining and understanding that open source that you brought into your organization, but for something big like,

you’re using some Linux variation, like there’s a reason Red Hat exists and you pay them to support it, right? Because it’s incredibly big and complicated. So there’s possibly, I’m sure people do that, but how feasible is that for most situations? Probably a tough sell. And is there any going back? Is there any going back? Is the cat out of the bag at this point?

MANOJ (42:31.669)
Right.

MANOJ (42:48.338)
Yeah.

MANOJ (42:54.205)
Right. I mean, we’ve traded all these things for convenience, safety, security. So maybe, you know, one of the things you can do with risk is accept it. So perhaps that’s also in the cards that there’s a certain level of acceptance that it

Eric Allard (43:12.618)
Mm-hmm. Well, we are doing it now just without an understanding of what you’re accepting, because people don’t typically know what is in the software. So S-bomb is part of changing that aspect. You can at least know more about the thing you’re accepting risk with, right?

MANOJ (43:31.497)
Yeah, absolutely. So at SUS, Eric, how are you guys there’s so many open source platforms, applications, things out there. How do you build your like, how are you guys staying on top of it? What’s your intelligence mechanism to make it happen, if you will?

Eric Allard (43:40.652)
Mm-hmm.

Eric Allard (43:50.702)
Um, what’s our intelligence mechanism? Well, we pull, so there’s a couple different aspects you look at to kind of tie a vulnerability to a piece of software and you use a lot of open source information to do that. You look at package managers, you look at published information on sources like GitHub, you look at the NVD, you look at Sysa’s, uh, I’m trying to remember the name KVE, um, there’s an exploitability database.

with an EPSS score, there’s all these pieces of data that are publicly available that we try to overlay to help you make a decision. And then there’s lots of companies whose, and possibly open source organizations, whose mission it is to uncover those things and figure out what’s risky and exploitable and they provide it.

MANOJ (44:30.945)
Okay.

Eric Allard (44:48.91)
extensive services and trying to be the discoverer of a lot of these vulnerabilities in the first place. But what we do effectively is we look at that top level of your software. We understand something about the dependencies in your application, and then we give you information to try to map that to what is known about the vulnerability landscape. And there’s of course, lots of known unknowns. There’s…

new vulnerabilities every day. I showed the slide of 20,000 CVEs last year. So, and that’s only a tiny fraction of probably the vulnerabilities out there. Those are just the ones that have been discovered and shared publicly in NIST’s NVD. So.

MANOJ (45:20.618)
Sure.

MANOJ (45:36.925)
Now, in the fledgling era of chat GPT, is this not a problem that’s really well suited for natural language processing?

Eric Allard (45:42.093)
Mm-hmm.

Eric Allard (45:47.53)
I think it is. I think there are some technologies in particular that we don’t dabble in. And part of the reason is because I think AI-driven solutions are going to be very good at doing a lot of this. Something like a SAS, which is a static application analysis. So it’s actually looking at the lines of code in your application and looking at the source. So you can imagine.

Um, AI is there’s probably going to be some very good AI based solutions that can help say, Hmm, this, this is probably risky the way you’re doing this based on this giant library of source code that we’re we’ve ingested legally or not. But I mean, I mean, GitHub has your code, right? GitLab has your code. There’s, there’s some pretty big organizations that could be running, building those models.

MANOJ (46:46.817)
Sure. I guess it’s a bigger problem now, but back in the day, it was still a problem. I mean, for those of us that were writing code in C, you were always including those libraries, include mat.h, include this.h, whatever it might be. We never knew what was in there. We just assumed that the compiler manufacturer had actually done their job with the damn thing.

Eric Allard (46:47.822)
Yeah.

Eric Allard (47:03.31)
Mm-hmm.

Eric Allard (47:16.002)
Yeah, true. You didn’t. And you were just programming to some interface and you saw that in the header file and you, yeah, and there was a bunch of standard libraries, but you also were probably buying some of those capabilities from a trusted vendor. And it was assumed not only were they responsible for doing a good job, it wasn’t publicly, there was some security by obscurity in that.

MANOJ (47:23.881)
That’s right. That’s exactly right.

Eric Allard (47:44.93)
case, right? Not everybody knew what was in that built, what made that binary tick.

MANOJ (47:52.193)
True. Very true. I guess I keep coming back to this notion of, you know, if we take some kind of technology that you’re offering and we do reasonable due diligence on it, and I guess that’s the word I would wanna use, then does that absolve us of the responsibility of the outcome?

Eric Allard (48:10.21)
Thank you.

MANOJ (48:19.121)
of what that technology is doing. Because how can we, like there’s a proposition by the federal government of holding software development firms accountable for all security vulnerabilities in their software. On the surface, that sounds real nice, but the reality of it is that nobody could do it. Even the Microsofts and Oracles of the world use tremendous amount of open source software in there.

Eric Allard (48:21.614)
Mm-hmm.

Eric Allard (48:34.83)
Mm-hmm.

Eric Allard (48:48.288)
Mm-hmm.

MANOJ (48:49.265)
Adobe uses a ton of open source in their offerings, right? How could they guarantee it? And how could you hold them accountable, really? Any thoughts on that, Eric?

Eric Allard (49:03.85)
Um, you may not. I mean, you, you probably can’t reasonably guarantee anything that big. Right. But I do think there is a certain element and a certain duty to disclose what it is you’re using, you know, just like when we talked about the food example, because today, um, you’re choosing to use Adobe, but you don’t.

know any of your, you know, you said yourself, they’re surely using lots of things, but you don’t even have the opportunity, either you personally or with some other highly proficient and educated tool or person to make an assessment and say, geez, I don’t really want to be using it because it’s built upon this thing. And I think there is going to, it is probably going to take not only a

change in expectations between vendors and their clients, but also regulatory changes before people willingly spit those out at scale. I think we’re starting to see it little by little, but I guess what I’m saying is, I don’t think you could realistically guarantee that, but I think you could provide an assessment of what it is inside your software.

to demonstrate that you’ve done some of that due diligence and someone else can make their own decision.

MANOJ (50:31.773)
Exactly. I think there’s a point of reasonableness, right? These digital outcomes that the public wants will hold them accountable. On the surface seems like a great idea, but you’re creating an environment where people may not voluntarily disclose or they’re going to say, you know what, we’re not going to scan it. This has been out there for ages.

Eric Allard (50:32.546)
Yeah.

Eric Allard (50:42.114)
Ahem.

MANOJ (50:57.697)
We don’t know what’s all in there. And if we go and scan this thing for vulnerability and it spits back 100 of them, we can’t afford to fix it and our customers can’t afford to pay for it.

Eric Allard (51:11.65)
Yeah, but you may find that you scan it and you can’t afford to fix them until you figure out, you know, that it could be some life, like maybe there’s no known vulnerability at that moment, but you could find out tomorrow there is, and that could be a serious life safety issue. You know, you now know, and that’s your duty to make it the right decision at that point. And if you don’t, and people really are harmed,

then surely you’ll be responsible, right? There’s no way you’re not gonna get.

MANOJ (51:42.474)
Well.

Eric Allard (51:46.247)
into oblivion. Yeah, well, sure.

MANOJ (51:47.521)
Well, you have coverage for that. So I’ll give you an example. I had an, I’ll keep, you know, an attorney from a very well-known law firm once tell me this, because we were talking about doing, you know, vulnerability scanning in their environment. And they really were not too fond of the idea because it is, if vulnerabilities are discovered,

Eric Allard (52:05.772)
Okay.

Eric Allard (52:12.194)
Mm-hmm.

MANOJ (52:17.021)
and they’re put on paper and documented. And then eventually that results in a breach and the loss of data, they are completely liable, right? Whereas if they don’t know about it and they’re exercising reasonableness in their cybersecurity program and data gets out there, they have some plausible deniability. I hate to say that, but in layman’s terms, that’s kind of what I got out of that discourse that…

Eric Allard (52:25.57)
Mm-hmm.

MANOJ (52:47.133)
And they don’t want to lose that.

Eric Allard (52:47.342)
Yeah. Oh, I’ve been told that indirectly many times. That’s a absolutely I’ve heard that before. I bet at some point, I feel like that’s doesn’t hold up, you know, like, you, you probably have your, you know, you have an old house, and you may know there’s asbestos in those walls, but you say, well, I’m just not going to test it. I’m going to do some work. First of all, I can’t do any work. I can’t tear any walls down or do any work.

MANOJ (53:07.403)
Yeah.

Eric Allard (53:17.314)
because it would need to be tested and then it would need to be remediated. And that would, you’re gonna get, it’s gonna catch up with you someday. I think I understand people taking that approach and I’ve heard it, but I don’t know that that’s really gonna hold up for long, for much longer.

MANOJ (53:40.589)
Well, Eric, we’re here almost at the hour. Do you have anything new coming out in terms of other talks you’re doing? Any other writings you’re doing out there keeping us all honest in the world of Vespom’s?

Eric Allard (53:56.746)
I might, I mean, I, you know, I’m not an SBOM expert. I keep up with it. I try to attend. There’s a number of interesting working groups. If it’s something that your audience decides they want to learn more about that CISA puts on. There’s all kinds of interesting activity out there by people that have kind of made this their life mission. There’s some interesting blog posters that talk about SBOM. I’d be happy to share those resources. Me personally.

I’m not sure what my next talks will be about yet. So I have some ideas, but I’m not gonna say what they are yet. But I will let you know when they’re ready. Yep. Well, thank you for having me. I appreciate being back on the show. It’s always fun.

MANOJ (54:35.818)
Please do.

Please do.

MANOJ (54:44.265)
Well, we, you know, we’re an educational platform and you certainly help us fulfill that mission. So we’re really grateful.

Eric Allard (54:54.09)
I’m grateful to be here. Thank you for always being so welcoming and I enjoy a lot of the people and topics you have. So I appreciate it too.

MANOJ (55:03.869)
Hey, thank you, Eric. I really appreciate that. It’s always nice to get something nice is said about you. Although, you know, the trollers are the ones that usually get the light.

Eric Allard (55:12.459)
I’m sorry.

Eric Allard (55:16.62)
NAH!

Eric Allard (55:20.474)
You’re never gonna get rid of, you think you’re never gonna find all the vulnerabilities. You’re never gonna get rid of all the trolls.

MANOJ (55:25.793)
Yeah, you’re the owner. You know what? You need some of those guys too, because they do make the world a little interesting.

Eric Allard (55:34.386)
Oh, you know what? That might be my next talk. I have a little side project right now where all of these scammers that reach out to me on social media, LinkedIn, I just have a great time talking to them now and pretending like I have no idea. And that is a lot of fun. So maybe I’ll make something out of that because that is a little more lighthearted.

MANOJ (55:56.624)
Well, when you do, let us know. All right, Eric. Well, hey, take care and have a great day. Bye-bye.

Eric Allard (55:58.518)
All right, I will. Thank you. You too. Bye-bye.