SBOM Challenge - Day Two

Day 2 of the SBOM Challenge is in the books– again if anyone didn’t catch our blog post on Day 1, this challenge is the first of its kind technology challenge issued to vendors to identify the current state of innovation for generating and managing SBOM’s - at the S4X23 Conference in Miami, FL. 

No sunburns yet! Just the broken hearts of loved ones NOT in Miami!

Day 2 kicked off with INL dropping new VEX files for applicability to the SBOMs that were generated. This phase of the SBOM Challenge was centered around vulnerability enrichment / vulnerability data reporting (VDR), vulnerability mitigation information (VEX) and the more dynamic vulnerability mitigation information (VEX CSAF).

A single word that could be used to describe the Day 1 portion of the challenge would be information. For Day 2, that word would be context - can you drill deeply into the firmware, and how comprehensively can you analyze it along with a few wrenches thrown in?

For the most part, the market understands that having a highly accurate SBOM for each artifact in your environment is important. There’s a massive amount of information about the components that create the software supply chain, and that allows organizations to make informed decisions about how to architect various components to ensure that an organization’s security posture is validated to protect those artifacts.

However without that context around our information in an SBOM, building that architecture is extremely difficult. If SBOMS are likened to the content labeling on our food, we now have a list of ingredients - some make the analogy that its all the “stuff in the soup. However, based on that list of ingredients, what should I as a consumer do about it?

That’s where applying the context of CVEs, CWEs, and other vulnerability data through analysis of vulnerability enrichment to an SBOM enhances our ability to start building that security architecture for our artifacts.  

Unfortunately, we’re often left with too much contextual information just out of that vulnerability enrichment analysis. Wouldn't it be nice, if for each possible vulnerability, we could be given some indication of confidence that the discovered security issues were actual issues to address?

This is where VEX can help us narrow down what arguably, is the most important issues on sour artifacts that we should address with our security architecture AS WELL as with those upstream in our supply chain so that we can reduce risk.

Of course the VEX data is a static, point in time observation of how each component is identified in an SBOM and how it may be vulnerable to attack. The vulnerability landscape changes over time as new vulnerabilities are discovered across all of the identified components.  The landscape also changes about how each provider in the software security supply chain chooses to address newly discovered security issues.  

This is where VEX CSAF comes to the rescue. VEX CSAF  allows us to have a more dynamic conversation over time to update how we should change our security architecture and posture to more accurately address newly discovered threats.  VEX CSAF is delivering that ongoing conversation.

What We Analyzed and How We Did It:
As part of the Day 2 challenge, INL provided VEX and VEX CSAF data. To level the playing field, INL also provided their “ground truth” versions of SBOMs for each of the three artifacts. 

Each participant was left to make a decision: Leverage our own SBOMs that we’d generated, leverage INL’s ground truth SBOMs, or both.

Here’s what we performed today as our next step in the challenge based on what was required by INL: 

The first task of the morning was to provide INL with vulnerability enrichment data.  We didn’t technically need to do much with this one, as the CycloneDX SBOMs that were submitted for all three artifacts on Day 1, already included the Vulnerability Disclosure Report (VDR). 

For the sake of completeness, and to demonstrate Finite State’s capabilities, we also went on to submit vulnerability enrichment data in SPDX + VDR output as well as standalone VDR data.

Getting to this point, however, was not without its challenges. As it turns out, there were some inconsistencies in the way the INL SBOMs were generated, in combination with VDR and VEX data. All of this confusion is not surprising, given the overall state of CycloneDX, SPDX, VEX, CSAF and VDR standards - that is to say, while there are standards, there are many inconsistencies and incompatibilities between the versions, cross linking, and even agreement on what the standards entail. 

Our engineers, who know these standards like the back of their hands, were able to immediately notice issues with the document formats and VDR cross-linking. They knew immediately what needed to be done (and had largely already accounted for in the Finite State platform) to rescue information provided by INL’s data while still ensuring that both the input and output data was valid, accurate, and generated useful results.

Overall, we are able to generate some substantial output in CycloneDX and SPDX formats, all enhanced to various levels of applicability, after ingesting or generating VDR, VEX, and CSAF VEX.  Today, however, INL wanted a presentation of this data in each individual participants platform, to gauge how actual users would interact with the data.

Based on this aggregate data, after it was combined, cross referenced, and deduplicated, here’s what we found for each of the artifacts:

Artifact 1, Windows .msi installer: 

Artifact 2, Motorola 68000 series bare metal firmware image:

Artifact 3, Linux based firmware based on OpenWRT:

Our Perspective:
Today’s input formats not only represented a challenge, likely to all participants, due to the inconsistent nature of the data provided. While we understand that “bad data” can happen, even from the tools created by the SBOM format standard creators, it has to be stated here that this was a simulation only, and from our perspective, it does not accurately represent how this technology would be applied for a customer. 

In addition to the features of each format that we had to understand and process, there were other technical challenges that needed to be addressed:

The two incompatible formats referenced each other. This is a use case that we have never seen previously, as the data that was received was confusing and highlights the lack of overall use of this kind of data in the industry right now. Providing data in multiple formats led to a mish-mash of types of data spread between the two competing standards (CycloneDX and SPDX).  

While the overlap of the two formats is to exchange SBOM data, they quickly diverge on extra features (vulnerability information and mitigations) and the depth of information they provide. This challenge highlighted the need for a more unified format with some flexibility to convert between the two standards and an understanding of what the formats provide based on what the industry is trying to directly accomplish.

The information used to reference vulnerabilities was malformed - extra spaces were added to component names, purls that were used were invalid. This leads to collecting incorrect vulnerability information and, in practice, would give a false sense of security to a user of this data.

Impact Analysis:
How are the Day 2 results impactful to the market? And what did we learn? 

As an overall result, we were able to generate data based on, what was arguably inaccurate data from INL to generate data usable for attempting to mitigate actual supply chain introduced risk.

Yes, there were challenges - that is why this is the SBOM Challenge after all? The Finite State team quickly overcame the challenges posed as we recognized them, and has helped us to expand our strategy for processing this data. The team navigated these speed bumps to produce a demo for applying VEX on top of a VDR-enriched SBOM that was generated by Finite State through binary analysis on Day 1, as well as the data provided as “ground truth” by INL. 

As a reiteration of yesterday, our customers are focused on driving down software supply chain risk. When we create SBOMs for customers, we do it to make their jobs easier by having the comprehensive workup of the software and components present in any embedded system or connected device. 

We can deliver more value to help manage any and all security impacts of supply chain security issues, through accurate application of VDR, VEX and CSAF VEX data, allowing our customers to make informed decisions about any additional security architecture they may need to implement.

Stay tuned for our Day 3 recap!

If you want to see our platform in action, come to the SBOM Pavillion for a demo this morning or to the cabana sessions at the Surfcomber starting at 1:00. If you’re not at S4, you can easily request a demo here.