debsbom

https://siemens.github.io/debsbom
Admin
debsbom generates Software Bills of Materials (SBOMs) in SPDX and CycloneDX formats for Debian-based...
Tokens:45,646
Snippets:290
Trust Score:9.7
Update:1 month ago
Show doc for...
Tokens:
Raw
### Define a Resolver Plugin Setup Function

Source: https://siemens.github.io/debsbom/_sources/plugins.rst.txt

The setup function signature required for the debsbom.download.resolver entry point.

```python
from request import Session
from debsbom.download.plugin import Resolver

def setup_resolver(session: Session) -> Resolver
    pass
```

--------------------------------

### Verify debsbom installation

Source: https://siemens.github.io/debsbom/_sources/starting.rst.txt

Displays the help menu to confirm the tool is installed and accessible.

```bash
debsbom -h
```

--------------------------------

### GET /mr/package/

Source: https://siemens.github.io/debsbom/_modules/debsbom/snapshot/client.html

Iterate all known packages on the mirror.

```APIDOC
## GET /mr/package/

### Description
Iterate all known packages on the mirror. Note that this request is costly.

### Method
GET

### Endpoint
/mr/package/
```

--------------------------------

### Install python3-apt system package

Source: https://siemens.github.io/debsbom/_sources/starting.rst.txt

Optional system-level installation to improve deb822 parsing performance.

```bash
apt install python3-apt
```

--------------------------------

### Generate SBOM with different ingress methods

Source: https://siemens.github.io/debsbom/_sources/design-decisions.rst.txt

Compares standard rootfs generation with the --from-pkglist approach, noting that the latter treats all packages as manually installed.

```bash
debsbom generate -r /my-root

cat /my-root/var/lib/dpkg/status | debsbom generate -r /my-root --from-pkglist
```

--------------------------------

### Example Debian Package Status Entry

Source: https://siemens.github.io/debsbom/design-decisions.html

A sample status file entry for the lmodern package showing the 'all' architecture and its dependencies.

```text
Package: lmodern
Status: install ok installed
Priority: optional
Section: fonts
Installed-Size: 33268
Maintainer: Debian TeX Task Force <debian-tex-maint@lists.debian.org>
Architecture: all
Multi-Arch: foreign
Version: 2.005-1
Replaces: lm, lmodern-x11
Depends: tex-common (>= 6.13), xfonts-utils, fonts-lmodern (= 2.005-1)
[...]
```

--------------------------------

### Retrieve binary packages from repository

Source: https://siemens.github.io/debsbom/_modules/debsbom/apt/cache.html

Fetches binary packages from the repository, applying filters and checking manual installation states.

```python
    def binpackages(
        self,
        filter_fn: Callable[[BinaryPackageFilter], bool] | None = None,
        ext_states: ExtendedStates = ExtendedStates(set(), set()),
    ) -> Iterable[BinaryPackage]:
        """Get all binary packages from this repository"""
        if self.components:
            for component in self.components:
                for arch in self.architectures:
                    packages_file = "_".join(
                        [self.repo_base, component, f"binary-{arch}", "Packages"]
                    )
                    for p in self._parse_packages(packages_file, filter_fn):
                        p.manually_installed = ext_states.is_manual(p.name, p.architecture)
                        yield p
        else:
            for arch in self.architectures:
                packages_file = "_".join([self.repo_base, f"binary-{arch}", "Packages"])
                for p in self._parse_packages(packages_file, filter_fn):
                    p.manually_installed = ext_states.is_manual(p.name, p.architecture)
                    yield p
```

--------------------------------

### SBOM Structure Examples

Source: https://siemens.github.io/debsbom/commands/delta.html

Visual representation of base and target SBOM structures and the resulting delta output.

```text
base-root
|- binary-dep1
|  |- source-dep1
|- binary-dep2
```

```text
target-root
|- binary-dep1
|  |- source-dep1
|- binary-dep2
|- binary-dep3
|  |- source-dep3
```

```text
delta-doc-root
|- binary-dep3
|  |- source-dep3
```

--------------------------------

### Install debsbom via pip

Source: https://siemens.github.io/debsbom/_sources/starting.rst.txt

Installs the debsbom package with various dependency options.

```bash
pip3 install debsbom[cdx,spdx,download]
```

```bash
pip3 install -e .[dev]
```

--------------------------------

### Get repository base

Source: https://siemens.github.io/debsbom/_modules/debsbom/apt/cache.html

Calculates the repository base path from the release file attribute.

```python
@property
def repo_base(self):
    return "_".join(str(self.release_file).split("_")[:-1])
```

--------------------------------

### Manage APT Extended States

Source: https://siemens.github.io/debsbom/_modules/debsbom/apt/cache.html

Use the ExtendedStates class to track manually installed packages and parse state information from files.

```python
@dataclass
class ExtendedStates:
    """
    The apt extended states encode information if a package is manually
    installed or installed via a dependency only. Note, that the PackageFilter
    maps arch=all binary packages to the architecture they are installed for
    (usually that's the distro arch).
    """

    PackageFilter = namedtuple("BinaryPackage", "name arch")

    auto_installed: set[tuple[str, str]]
    distro_archs: set[str]

    def is_manual(self, name: str, arch: str) -> bool:
        """
        True if package is explicitly installed. Architecture unspecific
        packages are mapped to the arch of the package that had it as a dependency.
        As we don't know that when parsing, we simply scan in all architectures.
        """
        if arch == "all":
            return not any([(name, _arch) in self.auto_installed for _arch in self.distro_archs])
        return (name, arch) not in self.auto_installed

    @classmethod
    def from_file(
        cls, file: str | Path, filter_fn: Callable[[PackageFilter], bool] | None = None
    ) -> "ExtendedStates":
        """Factory to create instance from the apt extended states file"""
        auto_installed = set()
        distro_archs = set()
        with open(Path(file), encoding="utf-8") as f:
            for s in Deb822.iter_paragraphs(f, use_apt_pkg=HAS_PYTHON_APT):
                name = s.get("Package")
                arch = s.get("Architecture")
                if s.get("Auto-Installed") != "1":
                    continue
                if (filter_fn is None) or (filter_fn(cls.PackageFilter(name, arch))):
                    auto_installed.add((name, arch))
                    distro_archs.add(arch)

        return cls(auto_installed=auto_installed, distro_archs=distro_archs)
```

--------------------------------

### Create a virtual environment

Source: https://siemens.github.io/debsbom/_sources/starting.rst.txt

Initializes a new Python virtual environment with system site packages enabled.

```bash
python3 -m venv --system-site-packages <env_name>
```

--------------------------------

### Execute license-clearing workflow

Source: https://siemens.github.io/debsbom/_sources/examples.rst.txt

Generate an SBOM from a rootfs and download all associated source packages for license review.

```bash
debsbom --progress generate -r path/to/the/rootfs -t cdx -o sbom
# output in sbom.cdx.json
```

```bash
debsbom --progress download --outdir downloads --sources sbom.cdx.json
# the downloaded files will be in downloads/sources/<archive>
```

--------------------------------

### Create and Activate Virtual Environment

Source: https://siemens.github.io/debsbom/starting.html

Commands to initialize and activate a Python virtual environment for the project.

```bash
python3 -m venv --system-site-packages <env_name>
```

```bash
source <env_name>/bin/activate
```

--------------------------------

### GET /mr/file/{hash}/info

Source: https://siemens.github.io/debsbom/_modules/debsbom/snapshot/client.html

Retrieve information about a file by its hash.

```APIDOC
## GET /mr/file/{hash}/info

### Description
Retrieve information about a file by hash.

### Method
GET

### Endpoint
/mr/file/{hash}/info

### Parameters
#### Path Parameters
- **hash** (string) - Required - The hash of the file to retrieve information for.
```

--------------------------------

### Package Filtering Utilities

Source: https://siemens.github.io/debsbom/_modules/debsbom/dpkg/package.html

Utility functions to filter collections of packages by type or installation status.

```python
def filter_sources(pkgs: Iterable["Package"]) -> Iterable["SourcePackage"]:
    """Free function to only return source packages"""
    return filter(lambda p: p.is_source(), pkgs)


def filter_binaries(pkgs: Iterable["Package"]) -> Iterable["BinaryPackage"]:
    """Free function to only return binary packages"""
    return filter(lambda p: p.is_binary(), pkgs)


def filter_installed(pkgs: Iterable["BinaryPackage"]) -> Iterable["BinaryPackage"]:
    """Free function to only return installed binary packages."""
    return filter(lambda p: p.status in (DpkgStatus.INSTALLED, DpkgStatus.DEBSBOM_UNKNOWN), pkgs)
```

--------------------------------

### Define DpkgStatus Enum

Source: https://siemens.github.io/debsbom/_modules/debsbom/dpkg/package.html

Represents the installation status of a Debian package with a helper method to parse status strings from dpkg.

```python
class DpkgStatus(Enum):
    NOT_INSTALLED = "n"
    CONFIG_FILES = "c"
    HALF_INSTALLED = "H"
    UNPACKED = "U"
    HALF_CONFIGURED = "F"
    TRIGGERS_AWAITING = "W"
    TRIGGERS_PENDING = "t"
    INSTALLED = "i"
    DEBSBOM_UNKNOWN = "0"

    @classmethod
    def from_dpkg(cls, status: str) -> "DpkgStatus":
        if len(status) > 1:
            status = status.lower()
        for checkStatus in DpkgStatus:
            if checkStatus == DpkgStatus.DEBSBOM_UNKNOWN:
                continue
            if status in [checkStatus.value, checkStatus.name.lower().replace("_", "-")]:
                return checkStatus
        raise ValueError(f"Unknown dpkg status '{status}'")
```

--------------------------------

### Generate Local System SBOM

Source: https://siemens.github.io/debsbom/_sources/examples.rst.txt

Create a CycloneDX SBOM for the current system.

```bash
debsbom --progress generate -t cdx -o sbom
# output in sbom.cdx.json
```

--------------------------------

### Download specific packages

Source: https://siemens.github.io/debsbom/examples.html

Downloads specific packages by providing name, version, and architecture via standard input.

```bash
cat <<EOF | debsbom download --binaries --sources
cpp 4:12.2.0-3 amd64
guestfs-tools 1.52.3-1 source
EOF
```

--------------------------------

### Generate Container Rootfs SBOM

Source: https://siemens.github.io/debsbom/_sources/examples.rst.txt

Create an SBOM for a Debian-based container using Podman or skopeo/umoci.

```bash
CRT=$(podman create debian:bookworm)
CHROOT=$(podman unshare podman mount $CRT)
podman unshare debsbom generate -t spdx --root $CHROOT
```

```bash
skopeo copy docker://debian:bookworm oci:debian:bookworm
umoci raw unpack --rootless --image debian:bookworm ./rootfs
debsbom generate -t spdx --root ./rootfs
```

--------------------------------

### Define SBOM types and validate dependencies

Source: https://siemens.github.io/debsbom/_modules/debsbom/sbom.html

The SBOMType enumeration manages supported SBOM standards and provides a method to verify if necessary third-party libraries are installed.

```python
[docs]
class SBOMType(Enum):
    """Supported SBOM types"""

    CycloneDX = (0,)
    """CycloneDX"""
    SPDX = (1,)
    """SPDX"""

    @classmethod
    def from_str(cls, bomtype: str):
        match bomtype.lower():
            case "cdx":
                return cls.CycloneDX
            case "spdx":
                return cls.SPDX
            case _:
                raise RuntimeError(f"Unknown SBOM type '{bomtype}'")

    def __str__(self):
        match self:
            case self.CycloneDX:
                return "cdx"
            case self.SPDX:
                return "spdx"
            case _:
                assert False, "unreachable"

[docs]
    def validate_dependency_availability(self) -> None:
        """
        Check if the required imports for the SBOM type are available.
        Raises RuntimeError if not available.
        """
        if self == SBOMType.CycloneDX:
            try:
                import cyclonedx.model.component
            except ModuleNotFoundError as e:
                raise RuntimeError(
                    "Missing dependency: cyclonedx-python-lib; install it by enabling the"
                    "dependency extra `cdx`: `pip install debsbom[cdx]`"
                )
        elif self == SBOMType.SPDX:
            try:
                import spdx_tools.spdx.model.package
            except ModuleNotFoundError as e:
                raise RuntimeError(
                    "Missing dependency: spdx-tools; install it by enabling the"
                    "dependency extra `spdx`: `pip install debsbom[spdx]`"
                )
```

--------------------------------

### Initialize and Run SecurityScanner

Source: https://siemens.github.io/debsbom/_modules/debsbom/securityscan/scanner.html

Main scanner class to load a security database and perform vulnerability checks on a collection of source packages.

```python
[docs]
class SecurityScanner:
    """
    A security scanner that checks source packages against a Debian
    security database for known vulnerabilities.
    """

    def __init__(self, db: Path, distro: str = "trixie"):
        with open(db, "r") as f:
            self.ct = CveTriage(json.load(f), distro=distro)

    def scan(
        self,
        src_pkgs: Iterable[SourcePackage],
        min_urgency: CveUrgency = CveUrgency.NOT_YET_ASSIGNED,
        name_filter: str | None = None,
    ) -> Iterable[ScanResultItem]:
        for p in src_pkgs:
            if name_filter and p.name != name_filter:
                continue

            vulns = self.ct.candidates(p)
            for v in vulns:
                if v.urgency > min_urgency:
                    continue
                yield ScanResultItem(package=p, vulnerability=v, affected=self.ct.affected_by(p, v))
```

--------------------------------

### Download Source Packages

Source: https://siemens.github.io/debsbom/examples.html

Download source packages based on a generated SBOM.

```bash
debsbom --progress download --outdir downloads --sources sbom.cdx.json
# the downloaded files will be in downloads/sources/<archive>
```

--------------------------------

### Download via container image

Source: https://siemens.github.io/debsbom/examples.html

Executes the download command using the official debsbom Docker image, mounting a local directory for output.

```bash
echo "guestfs-tools 1.52.3-1 source" | \
docker run -v$(pwd)/downloads:/mnt/downloads -i ghcr.io/siemens/debsbom:latest \
    debsbom download --outdir /mnt/downloads --sources
```

--------------------------------

### Download Artifacts from SBOM

Source: https://siemens.github.io/debsbom/_sources/examples.rst.txt

Download binary and source artifacts referenced in an SBOM, or download specific packages directly.

```bash
debsbom --progress \
    download --outdir downloads --sources --binaries sbom.cdx.json
find downloads -mindepth 1 -maxdepth 1
# downloads/.cache   <- debsbom metadata to map packages to artifacts
# downloads/sources  <- files related to source packages (e.g. .dsc, .orig.tar)
# downloads/binaries <- .deb files
```

```bash
cat <<EOF | debsbom download --binaries --sources
cpp 4:12.2.0-3 amd64
guestfs-tools 1.52.3-1 source
EOF
```

```bash
echo "guestfs-tools 1.52.3-1 source" | \
docker run -v$(pwd)/downloads:/mnt/downloads -i ghcr.io/siemens/debsbom:latest \
    debsbom download --outdir /mnt/downloads --sources
```

--------------------------------

### Generate SBOM from Package List

Source: https://siemens.github.io/debsbom/_sources/examples.rst.txt

Create an SBOM from a provided list of packages, optionally injecting a dpkg status file.

```bash
echo "htop 3.4.1-5 amd64" | debsbom generate --from-pkglist
# or in isar manifest format
echo "json-c|0.16-2|libjson-c5:amd64|0.16-2" | debsbom generate --from-pkglist
# or with PURLs
echo "pkg:deb/debian/htop@3.4.1-5?arch=amd64" | debsbom generate --from-pkglist
```

```bash
cat path/to/dpkg/status | debsbom generate --from-pkglist
```

--------------------------------

### Download command usage

Source: https://siemens.github.io/debsbom/commands/download.html

Displays the command-line interface usage for the download command.

```text
usage: debsbom download [-h] [-t {cdx,spdx}] [--outdir OUTDIR] [--sources]
                        [--binaries] [--skip-pkgs SKIP]
                        [--resolver {debian-snapshot}]
                        [bomin]
```

--------------------------------

### repack command usage

Source: https://siemens.github.io/debsbom/commands/repack.html

Displays the command-line interface usage and available arguments for the repack command.

```bash
usage: debsbom repack [-h] [-t {cdx,spdx}]
                      [--compress {no,bzip2,gzip,xz,zstd,lz4}]
                      [--apply-patches] [--mtime MTIME] [--dldir DLDIR]
                      [--outdir OUTDIR]
                      [--format {standard-bom,standard-bom-package}] [--copy]
                      [--validate] [--sources] [--binaries]
                      bomin bomout
```

--------------------------------

### Construct SourcePackage from Binary

Source: https://siemens.github.io/debsbom/_modules/debsbom/dpkg/package.html

Creates a SourcePackage instance from the binary package's source dependency.

```python
def source_package(self) -> SourcePackage | None:
    """Construct a source package from the referenced source dependency."""
    if self.source:
        return SourcePackage(self.source.name, self.source.version[1], self.maintainer)
    else:
        return None
```

--------------------------------

### Repack Artifacts

Source: https://siemens.github.io/debsbom/_sources/examples.rst.txt

Re-layout downloaded artifacts and recreate the SBOM, or process a subset of packages.

```bash
debsbom --progress repack \
    --dldir downloads \
    --outdir source-archive \
    --compress zstd \
    --apply-patches \
    --validate \
    sbom.cdx.json sbom.packed.cdx.json
```

```bash
echo "bash 5.2.37-2 source" | debsbom -v repack sbom-in.json sbom-out.json
```

--------------------------------

### Initialize SourceArchiveMerger

Source: https://siemens.github.io/debsbom/_modules/debsbom/repack/merger.html

Constructor for the SourceArchiveMerger class, requiring a download directory and optional output directory and compression format.

```python
def __init__(
    self,
    dldir: Path,
    outdir: Path | None = None,
    compress: Compression.Format = Compression.NONE,
):
    self.dldir = dldir
    self.outdir = outdir or dldir
    self.outdir.mkdir(exist_ok=True, parents=False)
    self.compress = compress
    self.dpkg_source = shutil.which("dpkg-source")
    if not self.dpkg_source:
        raise RuntimeError("'dpkg-source' from the 'dpkg-dev' package is missing.")
```

--------------------------------

### Interact with Source Packages

Source: https://siemens.github.io/debsbom/_modules/debsbom/snapshot/client.html

Classes and methods for querying source package versions, associated files, and binary packages.

```python
class Package:
    """
    Source package name (without specific version)
    """

    def __init__(self, sdl: "SnapshotDataLake", name: str):
        self.sdl = sdl
        self.name = name

[docs]
    def versions(self) -> Iterable["SourcePackage"]:
        """
        Iterate all versions of a ``SourcePackage``.
        """
        data = self.sdl.get(path=f"/mr/package/{self.name}/").json()
        for v in data["result"]:
            yield SourcePackage(self.sdl, self.name, v["version"])



[docs]
class SourcePackage:
    """
    Source package in a specific version
    """

    def __init__(self, sdl: "SnapshotDataLake", name: str, version: str):
        self.sdl = sdl
        self.name = name
        self.version = version

[docs]
    def srcfiles(
        self, archive: str | None = None, sha1: str | None = None
    ) -> Iterable["SnapshotRemoteFile"]:
        """
        All files associated with the source package. Returns multiple SnaphshotRemoteFile
        instances for a single hash in case the file is known under multiple names.
        If the package is not known to the snapshot mirror, raises NotFoundOnSnapshotError.
        If the filtering does not match any, return empty iterator.
        """
        data = self.sdl.get(
            path=f"/mr/package/{self.name}/{self.version}/srcfiles?fileinfo=1"
        ).json()
        fileinfo = data.get("fileinfo")
        for s in data.get("result", []):
            hash = s["hash"]
            for res in fileinfo[hash]:
                rf = SnapshotRemoteFile.fromfileinfo(self.sdl, hash, res)
                if archive and rf.archive_name != archive:
                    continue
                if sha1 and rf.hash != sha1:
                    continue
                rf.architecture = "source"
                yield rf

[docs]
    def binpackages(self) -> Iterable["BinaryPackage"]:
        """
        All binary packages created from this source package
        """
        data = self.sdl.get(path=f"/mr/package/{self.name}/{self.version}/binpackages").json()
        for b in data.get("result", []):
            yield BinaryPackage(self.sdl, b["name"], b["version"], self.name, self.version)
```

--------------------------------

### Implement Package Caching

Source: https://siemens.github.io/debsbom/_modules/debsbom/download/resolver.html

Provides a base class for caching package resolutions and a persistent file-backed implementation using Zstandard.

```python
class PackageResolverCache:
    """
    Maps packages to RemoteFile instances to avoid expensive calls to the upstream mirror.
    This dummy implementation can be used to not cache.
    """

[docs]
    def lookup(self, p: package.SourcePackage | package.BinaryPackage) -> list["RemoteFile"] | None:
        """Lookup package files in cache"""
        return None

[docs]
    def insert(
        self, p: package.SourcePackage | package.BinaryPackage, files: list["RemoteFile"]
    ) -> None:
        """Insert package files into cache"""
        pass
```

```python
class PersistentResolverCache(PackageResolverCache):
    """
    Trivial implementation of a file-backed cache. Each cache entry is stored as individual file
    in the cachedir.
    """

    def __init__(self, cachedir: Path):
        self.cachedir = cachedir
        self.cctx = ZstdCompressor(level=10)
        self.dctx = ZstdDecompressor()
        cachedir.mkdir(exist_ok=True)

    @staticmethod
    def _package_hash(p: package.Package) -> str:
        return hashlib.sha256(
            json.dumps(
                {
                    "purl": p.purl().to_string(),
                    "checksums": p.checksums,
                },
                sort_keys=True,
            ).encode("utf-8")
        ).hexdigest()

    def _entry_path(self, hash: str) -> Path:
        return self.cachedir / f"{hash}.json.zst"

[docs]
    def lookup(self, p: package.SourcePackage | package.BinaryPackage) -> list["RemoteFile"] | None:
        hash = self._package_hash(p)
        entry = self._entry_path(hash)
        if not entry.is_file():
            logger.debug(f"Package '{p.name}' is not cached")
            return None
        with (
            open(entry, "rb") as _f,
            self.dctx.stream_reader(_f) as f,
        ):
            try:
                data = json.load(f)
                for d in data:
                    d["checksums"] = {ChecksumAlgo(int(k)): v for k, v in d["checksums"].items()}
            except json.decoder.JSONDecodeError:
                logger.warning(f"cache file {entry.name} ({p}) is corrupted")
                return None
        logger.debug(f"Package '{p.name}' already cached")
        return [RemoteFile(**d) for d in data]

[docs]
    def insert(
        self, p: package.SourcePackage | package.BinaryPackage, files: list["RemoteFile"]
    ) -> None:
        hash = self._package_hash(p)
        entry = self._entry_path(hash)
        with (
            open(entry.with_suffix(".tmp"), "wb") as _f,
            self.cctx.stream_writer(_f) as cf,
            io.TextIOWrapper(cf, encoding="utf-8") as f,
        ):
            json.dump([dataclasses.asdict(rf.as_base()) for rf in files], f)
        entry.with_suffix(".tmp").rename(entry)
```

--------------------------------

### Define License and Exception Mappings

Source: https://siemens.github.io/debsbom/_modules/debsbom/apt/copyright.html

Contains dictionaries for mapping non-standard Debian license strings and exception names to their official SPDX identifiers.

```python
WELL_KNOWN_EXPRESSIONS = {
    "AGPL-1": "AGPL-1.0-only",
    "AGPL-1+": "AGPL-1.0-or-later",
    "AGPL-3": "AGPL-3.0-only",
    "AGPL-3+": "AGPL-3.0-or-later",
    "Apache-1": "Apache-1.0",
    "Apache-2": "Apache-2.0",
    "Artistic": "Artistic-1.0",
    "Artistic-1": "Artistic-1.0",
    "Artistic-2": "Artistic-2.0",
    "BSD-2": "BSD-2-Clause",
    "BSD-3": "BSD-3-Clause",
    "BSD-4": "BSD-4-Clause",
    "BSL": "BSL-1.0",
    "BSL-1": "BSL-1.0",
    "CC-BY": "CC-BY-1.0",
    "CC-BY-1": "CC-BY-1.0",
    "CC-BY-2": "CC-BY-2.0",
    "CC-BY-3": "CC-BY-3.0",
    "CC-BY-4": "CC-BY-4.0",
    "CC-BY-NC": "CC-BY-NC-1.0",
    "CC-BY-NC-1": "CC-BY-NC-1.0",
    "CC-BY-NC-2": "CC-BY-NC-2.0",
    "CC-BY-NC-3": "CC-BY-NC-3.0",
    "CC-BY-NC-4": "CC-BY-NC-4.0",
    "CC-BY-NC-ND": "CC-BY-NC-ND-1.0",
    "CC-BY-NC-ND-1": "CC-BY-NC-ND-1.0",
    "CC-BY-NC-ND-2": "CC-BY-NC-ND-2.0",
    "CC-BY-NC-ND-3": "CC-BY-NC-ND-3.0",
    "CC-BY-NC-ND-4": "CC-BY-NC-ND-4.0",
    "CC-BY-NC-SA": "CC-BY-NC-SA-1.0",
    "CC-BY-NC-SA-1": "CC-BY-NC-SA-1.0",
    "CC-BY-NC-SA-2": "CC-BY-NC-SA-2.0",
    "CC-BY-NC-SA-3": "CC-BY-NC-SA-3.0",
    "CC-BY-NC-SA-4": "CC-BY-NC-SA-4.0",
    "CC-BY-SA": "CC-BY-SA-1.0",
    "CC-BY-SA-1": "CC-BY-SA-1.0",
    "CC-BY-SA-2": "CC-BY-SA-2.0",
    "CC-BY-SA-3": "CC-BY-SA-3.0",
    "CC-BY-SA-4": "CC-BY-SA-4.0",
    "CC0": "CC0-1.0",
    "CC0-1": "CC0-1.0",
    "CDDL": "CDDL-1.0",
    "CDDL-1": "CDDL-1.0",
    "CPL": "CPL-1.0",
    "CPL-1": "CPL-1.0",
    "EFL": "EFL-1.0",
    "EFL-1": "EFL-1.0",
    "EFL-2": "EFL-2.0",
    "EPL": "EPL-1.0",
    "EPL-1": "EPL-1.0",
    "EPL-2": "EPL-2.0",
    "Expat": "MIT",
    "GFDL": "GFDL-1.1-only",
    "GFDL+": "GFDL-1.1-or-later",
    "GFDL-1.1": "GFDL-1.1-only",
    "GFDL-1.1+": "GFDL-1.1-or-later",
    "GFDL-1.2": "GFDL-1.2-only",
    "GFDL-1.2+": "GFDL-1.2-or-later",
    "GFDL-1.3": "GFDL-1.3-only",
    "GFDL-1.3+": "GFDL-1.3-or-later",
    "GFDL-NIV": "GFDL-1.1-no-invariants-only",
    "GFDL-NIV+": "GFDL-1.1-no-invariants-or-later",
    "GFDL-NIV-1.1": "GFDL-1.1-no-invariants-only",
    "GFDL-NIV-1.1+": "GFDL-1.1-no-invariants-or-later",
    "GFDL-NIV-1.2": "GFDL-1.2-no-invariants-only",
    "GFDL-NIV-1.2+": "GFDL-1.2-no-invariants-or-later",
    "GFDL-NIV-1.3": "GFDL-1.3-no-invariants-only",
    "GFDL-NIV-1.3+": "GFDL-1.3-no-invariants-or-later",
    "GPL": "GPL-1.0-only",
    "GPL+": "GPL-1.0-or-later",
    "GPL-1": "GPL-1.0-only",
    "GPL-1+": "GPL-1.0-or-later",
    "GPL-2": "GPL-2.0-only",
    "GPL-2+": "GPL-2.0-or-later",
    "GPL-3": "GPL-3.0-only",
    "GPL-3+": "GPL-3.0-or-later",
    "GPL3+": "GPL-3.0-or-later",
    "LGPL": "LGPL-1.0-only",
    "LGPL+": "LGPL-1.0-or-later",
    "LGPL-1": "LGPL-1.0-only",
    "LGPL-1+": "LGPL-1.0-or-later",
    "LGPL-2": "LGPL-2.0-only",
    "LGPL-2+": "LGPL-2.0-or-later",
    "LGPL-3": "LGPL-3.0-only",
    "LGPL-3+": "LGPL-3.0-or-later",
    "LPGL-2.1": "LGPL-2.1-only",
    "LPGL-2.1+": "LGPL-2.1-or-later",
    "LPPL": "LPPL-1.0",
    "LPPL-1": "LPPL-1.0",
    "MPL": "MPL-1.0",
    "MPL-1": "MPL-1.0",
    "MPL-2": "MPL-2.0",
    "PSF-2": "PSF-2.0",
    "Perl": "Artistic-1.0-Perl",
    "Python": "Python-2.0",
    "QPL": "QPL-1.0",
    "QPL-1": "QPL-1.0",
    "RubyLicense": "Ruby",
    "SIL-1": "OFL-1.0",
    "SIL-1.1": "OFL-1.1",
    "SIL-OFL-1": "OFL-1.0",
    "SIL-OFL-1.1": "OFL-1.1",
    "Zope": "ZPL-1.0",
    "Zope-1": "ZPL-1.0",
    "Zope-2": "ZPL-2.0",
}

WELL_KNOWN_EXCEPTIONS = {
    "GPL-3+ with Bison exception": "GPL-3.0-or-later WITH Bison-exception-2.2",
    "GPL-3+ with Bison-2.2 exception": "GPL-3.0-or-later WITH Bison-exception-2.2",
    "GPL-2 with Linux-syscall-note exception": "GPL-2.0-only WITH Linux-syscall-note",
    "GPL-3+ with texinfo exception": "GPL-3.0-or-later WITH Texinfo-exception",
    "GPL-3.0-or-later-WITH-Autoconf-exception-macro": (
        "GPL-3.0-or-later WITH Autoconf-exception-macro"
    ),
    "BSD-3-clause-Cambridge with BINARY LIBRARY-LIKE PACKAGES exception": (
        "BSD-3-Clause WITH PCRE2-exception"
    ),
    "GPL-3+-WITH-BISON-EXCEPTION": "GPL-3.0-or-later WITH Bison-exception-2.2",
}
```

--------------------------------

### Generate CycloneDX SBOM

Source: https://siemens.github.io/debsbom/examples.html

Generate a CycloneDX SBOM from a root filesystem.

```bash
debsbom --progress generate -r path/to/the/rootfs -t cdx -o sbom
# output in sbom.cdx.json
```

--------------------------------

### Identify and process new components

Source: https://siemens.github.io/debsbom/_sources/examples.rst.txt

Extract new PURLs from SBOMs and use them to download, merge, or repack source packages.

```bash
PURLS=$( \
    diff -U0 \
        <(jq -r --sort-keys '.components[].purl' sbom.old.cdx.json) \
        <(jq -r --sort-keys '.components[].purl' sbom.cdx.json) \
        | grep ^+pkg | sed 's/^+//' \
)
```

```bash
echo "$PURLS" | debsbom download --sources --binaries
```

```bash
echo "$PURLS" | debsbom source-merge --apply-patches
```

```bash
echo "$PURLS" | debsbom repack \
    --apply-patches
    sbom.cdx.json \
    sbom.cdx.repacked.json
```

--------------------------------

### debsbom download

Source: https://siemens.github.io/debsbom/commands/download.html

Processes a SBOM and downloads the referenced packages. If no SBOM is provided, it reads line separated entries (name version arch) from stdin.

```APIDOC
## Command: debsbom download

### Description
Processes a SBOM and downloads the referenced packages. If no SBOM is provided, it reads line separated entries (name version arch) from stdin to define what shall be downloaded.

### Usage
`debsbom download [-h] [-t {cdx,spdx}] [--outdir OUTDIR] [--sources] [--binaries] [--skip-pkgs SKIP] [--resolver {debian-snapshot}] [bomin]`

### Positional Arguments
- **bomin** (string) - Optional - sbom file(s) to process. Use '-' to read from stdin.

### Named Arguments
- **-t, --sbom-type** (string) - Optional - SBOM type to process (cdx, spdx). Required when reading from stdin.
- **--outdir** (string) - Optional - Directory to store downloaded files. Default: 'downloads'.
- **--sources** (boolean) - Optional - Operate only on source packages (skip binaries).
- **--binaries** (boolean) - Optional - Operate only on binary packages (skip sources).
- **--skip-pkgs** (string) - Optional - Packages to exclude from the download, in package-list format.
- **--resolver** (string) - Optional - Resolver to use to find upstream packages (default: 'debian-snapshot').

### JSON Output Schema
When run with JSON output enabled, the command emits objects with the following fields:
- **status** (string) - The status of the download operation ('ok', 'not_found', 'checksum_mismatch').
- **package** (object) - Object identifying the package (name, version, purl).
- **filename** (string) - The name of the processed file.
- **path** (string) - The absolute path to the downloaded file.
```

--------------------------------

### Interact with Binary Packages

Source: https://siemens.github.io/debsbom/_modules/debsbom/snapshot/client.html

Class for representing binary packages and retrieving their associated files.

```python
class BinaryPackage:
    """
    Binary package in a specific version
    """

    def __init__(
        self,
        sdl: "SnapshotDataLake",
        binname: str,
        binversion: str,
        srcname: str | None,
        srcversion: str | None,
    ):
        self.sdl = sdl
        self.binname = binname
        self.binversion = binversion
        self.srcname = srcname
        self.srcversion = srcversion

[docs]
    def files(self, arch: str | None = None) -> Iterable["SnapshotRemoteFile"]:
        """
        All files associated with this binary package (e.g. per-architecture)

        If no architecture is specified, all packages are returned.
        Otherwise, only the packages with the matching architecture are returned.
        If we have information about the source package as well, we precisely resolve the binary package
        including the original path on the debian mirror. If not, we just resolve the file.
        The difference is only in the metadata, the file itself is the same in both cases.
```

--------------------------------

### trace-path command

Source: https://siemens.github.io/debsbom/commands/trace-path.html

Processes an SBOM and a package identifier and emits paths from this package to the root.

```APIDOC
## trace-path

### Description
Processes an SBOM and a package identifier and emits paths from this package to the root.

### Usage
`debsbom trace-path [-h] [-t {cdx,spdx}] [--format {text,json,ref,dot}] [--mode {shortest,all-shortest,all}] [bomin] source`

### Positional Arguments
- **bomin** (string) - Required - SBOM file(s) to process. Use '-' to read from stdin.
- **source** (string) - Required - Source node (PURL).

### Named Arguments
- **-t, --sbom-type** (choice) - Optional - SBOM type to process (cdx, spdx). Required when reading from stdin.
- **--format** (choice) - Optional - Path output format (text, json, ref, dot). Default: 'text'.
- **--mode** (choice) - Optional - Path finding mode (shortest, all-shortest, all). Default: 'shortest'.

### JSON Output Schema
When using `--format json`, output adheres to the following schema:
- **name** (string) - Name of the component
- **ref** (string) - Reference identifier for the component
- **version** (string|null) - Version of the component
- **maintainer** (string|null) - Maintainer of the component
- **purl** (string|null) - Package URL of the component
```

--------------------------------

### Activate the virtual environment

Source: https://siemens.github.io/debsbom/_sources/starting.rst.txt

Activates the created virtual environment for the current shell session.

```bash
source <env_name>/bin/activate
```

--------------------------------

### Implement PackageDownloader

Source: https://siemens.github.io/debsbom/_modules/debsbom/download/download.html

Manages the retrieval of package artifacts, ensuring files are only downloaded once by checking the local directory.

```python
class PackageDownloader:
    """
    Retrieve package artifacts from upstream. Files are only retrieved once by comparison
    with the data in the local downloads directory.
    """

    def __init__(
        self,
        outdir: Path | str = "downloads",
        session: requests.Session = requests.Session(),
    ):
        self.outdir = Path(outdir)
        self.sources_dir = self.outdir / "sources"
        self.binaries_dir = self.outdir / "binaries"
        self.to_download: list[tuple[package.Package, RemoteFile]] = []
        self.rs = session
        self.known_hashes = {}

        self.outdir.mkdir(exist_ok=True)
        for p in [self.sources_dir, self.binaries_dir]:
            p.mkdir(exist_ok=True)

    def _target_path(self, pkg: package.Package, f: RemoteFile):
        if pkg.is_source():
            base = self.sources_dir
        else:
            base = self.binaries_dir
        base = base.resolve()
        target = (base / f.archive_name / f.filename).resolve()
        if not target.is_relative_to(base):
            purl = pkg.purl()
            raise ValueError(
                f"{purl}: malicious path in remote file detected: '{f.archive_name}/{f.filename}'"
            )
        else:
            return target

    def register(self, files: list[RemoteFile], package: package.Package) -> None:
        """Register a list of files corresponding to a package for download."""
        self.to_download.extend([(package, f) for f in files])

    def stat(self) -> StatisticsType:
        """
        Returns a tuple (files to download, total size, cached files, cached bytes)
        """
        unique_dl = list(
            {frozenset(v.checksums.items()): (pkg, v) for pkg, v in self.to_download}.values()
        )
        nbytes = reduce(lambda acc, x: acc + x[1].size if x[1].size else 0, unique_dl, 0)
        cfiles = list(filter(lambda x: self._target_path(x[0], x[1]).is_file(), unique_dl))
        cbytes = reduce(lambda acc, x: acc + x[1].size if x[1].size else 0, cfiles, 0)
        return StatisticsType(len(unique_dl), nbytes, len(cfiles), cbytes)
```

--------------------------------

### trace-path command usage

Source: https://siemens.github.io/debsbom/commands/trace-path.html

Displays the command-line syntax for the trace-path command, including positional and named arguments.

```bash
usage: debsbom trace-path [-h] [-t {cdx,spdx}] [--format {text,json,ref,dot}]
                          [--mode {shortest,all-shortest,all}]
                          [bomin] source
```

--------------------------------

### Create SourcePackage from deb822

Source: https://siemens.github.io/debsbom/_modules/debsbom/dpkg/package.html

Parses a deb822 representation to instantiate a SourcePackage, handling VCS information and checksums.

```python
@staticmethod
def from_deb822(package) -> "SourcePackage":
    """
    Create a package from a deb822 representation. If the deb822 input
    is a .dsc file, the name is read from the source property.
    """
    name = package.get("Source") or package["Package"]
    version = Version(package.get("Version"))
    maintainer = package.get("Maintainer")
    binary_entry = package.get("Binary")
    if binary_entry is not None:
        binaries = [b.strip() for b in binary_entry.split(",")]
    else:
        binaries = []
    homepage = package.get("Homepage")
    vcs = None
    for type in VcsType:
        locator = package.get(f"Vcs-{type.name}")
        if locator:
            if vcs:
                logger.warning(
                    f"Multiple VCS types found for package {name}: {vcs.type} and {type}"
                )
            # remove additional details (e.g., branch, path)
            locator = locator.split()[0]
            vcs = VcsInfo(type, locator)

    # Checksums according to Debian policy 5.6.24
    pkg_chksums = {}
    chksums = checksums_from_dsc(package)
    for c in filter(lambda c_: c_.endswith(".dsc"), chksums):
        pkg_chksums = chksums[c]

    return SourcePackage(
        name=name,
        version=version,
        maintainer=maintainer,
        binaries=binaries,
        homepage=homepage,
        vcs=vcs,
        checksums=pkg_chksums,
    )
```

--------------------------------

### Export command usage

Source: https://siemens.github.io/debsbom/commands/export.html

Displays the command-line syntax for the export operation.

```bash
usage: debsbom export [-h] [-t {cdx,spdx}] [--format {graphml}] [bomin] [out]
```

--------------------------------

### PackageResolver.create

Source: https://siemens.github.io/debsbom/_modules/debsbom/resolver/resolver.html

Factory method to create a PackageResolver instance from a file path. It automatically detects the SBOM type based on the file extension.

```APIDOC
## PackageResolver.create(filename: Path, bomtype: SBOMType | None = None)

### Description
Creates a PackageResolver instance for the given SBOM file. The resolver is used to iteratively process packages contained within the SBOM.

### Parameters
- **filename** (Path) - Required - The path to the SBOM file.
- **bomtype** (SBOMType | None) - Optional - The specific SBOM type if not detectable from the file extension.
```

--------------------------------

### debsbom.resolver.resolver.PackageStreamResolver

Source: https://siemens.github.io/debsbom/api.html

Handles universal package ingress and emits partial package instances.

```APIDOC
## PackageStreamResolver(pkgstream: IOBase)

### Description
Initializes the resolver to handle universal package ingress from the provided stream.
```

--------------------------------

### source-merge command usage

Source: https://siemens.github.io/debsbom/commands/source-merge.html

Displays the command-line interface syntax and available arguments for the source-merge command.

```text
usage: debsbom source-merge [-h] [-t {cdx,spdx}]
                            [--compress {no,bzip2,gzip,xz,zstd,lz4}]
                            [--apply-patches] [--mtime MTIME]
                            [--pkgdir PKGDIR] [--outdir OUTDIR]
                            [bomin]
```
Tokens:
Raw