### Define a Resolver Plugin Setup Function Source: https://siemens.github.io/debsbom/_sources/plugins.rst.txt The setup function signature required for the debsbom.download.resolver entry point. ```python from request import Session from debsbom.download.plugin import Resolver def setup_resolver(session: Session) -> Resolver pass ``` -------------------------------- ### Verify debsbom installation Source: https://siemens.github.io/debsbom/_sources/starting.rst.txt Displays the help menu to confirm the tool is installed and accessible. ```bash debsbom -h ``` -------------------------------- ### GET /mr/package/ Source: https://siemens.github.io/debsbom/_modules/debsbom/snapshot/client.html Iterate all known packages on the mirror. ```APIDOC ## GET /mr/package/ ### Description Iterate all known packages on the mirror. Note that this request is costly. ### Method GET ### Endpoint /mr/package/ ``` -------------------------------- ### Install python3-apt system package Source: https://siemens.github.io/debsbom/_sources/starting.rst.txt Optional system-level installation to improve deb822 parsing performance. ```bash apt install python3-apt ``` -------------------------------- ### Generate SBOM with different ingress methods Source: https://siemens.github.io/debsbom/_sources/design-decisions.rst.txt Compares standard rootfs generation with the --from-pkglist approach, noting that the latter treats all packages as manually installed. ```bash debsbom generate -r /my-root cat /my-root/var/lib/dpkg/status | debsbom generate -r /my-root --from-pkglist ``` -------------------------------- ### Example Debian Package Status Entry Source: https://siemens.github.io/debsbom/design-decisions.html A sample status file entry for the lmodern package showing the 'all' architecture and its dependencies. ```text Package: lmodern Status: install ok installed Priority: optional Section: fonts Installed-Size: 33268 Maintainer: Debian TeX Task Force Architecture: all Multi-Arch: foreign Version: 2.005-1 Replaces: lm, lmodern-x11 Depends: tex-common (>= 6.13), xfonts-utils, fonts-lmodern (= 2.005-1) [...] ``` -------------------------------- ### Retrieve binary packages from repository Source: https://siemens.github.io/debsbom/_modules/debsbom/apt/cache.html Fetches binary packages from the repository, applying filters and checking manual installation states. ```python def binpackages( self, filter_fn: Callable[[BinaryPackageFilter], bool] | None = None, ext_states: ExtendedStates = ExtendedStates(set(), set()), ) -> Iterable[BinaryPackage]: """Get all binary packages from this repository""" if self.components: for component in self.components: for arch in self.architectures: packages_file = "_".join( [self.repo_base, component, f"binary-{arch}", "Packages"] ) for p in self._parse_packages(packages_file, filter_fn): p.manually_installed = ext_states.is_manual(p.name, p.architecture) yield p else: for arch in self.architectures: packages_file = "_".join([self.repo_base, f"binary-{arch}", "Packages"]) for p in self._parse_packages(packages_file, filter_fn): p.manually_installed = ext_states.is_manual(p.name, p.architecture) yield p ``` -------------------------------- ### SBOM Structure Examples Source: https://siemens.github.io/debsbom/commands/delta.html Visual representation of base and target SBOM structures and the resulting delta output. ```text base-root |- binary-dep1 | |- source-dep1 |- binary-dep2 ``` ```text target-root |- binary-dep1 | |- source-dep1 |- binary-dep2 |- binary-dep3 | |- source-dep3 ``` ```text delta-doc-root |- binary-dep3 | |- source-dep3 ``` -------------------------------- ### Install debsbom via pip Source: https://siemens.github.io/debsbom/_sources/starting.rst.txt Installs the debsbom package with various dependency options. ```bash pip3 install debsbom[cdx,spdx,download] ``` ```bash pip3 install -e .[dev] ``` -------------------------------- ### Get repository base Source: https://siemens.github.io/debsbom/_modules/debsbom/apt/cache.html Calculates the repository base path from the release file attribute. ```python @property def repo_base(self): return "_".join(str(self.release_file).split("_")[:-1]) ``` -------------------------------- ### Manage APT Extended States Source: https://siemens.github.io/debsbom/_modules/debsbom/apt/cache.html Use the ExtendedStates class to track manually installed packages and parse state information from files. ```python @dataclass class ExtendedStates: """ The apt extended states encode information if a package is manually installed or installed via a dependency only. Note, that the PackageFilter maps arch=all binary packages to the architecture they are installed for (usually that's the distro arch). """ PackageFilter = namedtuple("BinaryPackage", "name arch") auto_installed: set[tuple[str, str]] distro_archs: set[str] def is_manual(self, name: str, arch: str) -> bool: """ True if package is explicitly installed. Architecture unspecific packages are mapped to the arch of the package that had it as a dependency. As we don't know that when parsing, we simply scan in all architectures. """ if arch == "all": return not any([(name, _arch) in self.auto_installed for _arch in self.distro_archs]) return (name, arch) not in self.auto_installed @classmethod def from_file( cls, file: str | Path, filter_fn: Callable[[PackageFilter], bool] | None = None ) -> "ExtendedStates": """Factory to create instance from the apt extended states file""" auto_installed = set() distro_archs = set() with open(Path(file), encoding="utf-8") as f: for s in Deb822.iter_paragraphs(f, use_apt_pkg=HAS_PYTHON_APT): name = s.get("Package") arch = s.get("Architecture") if s.get("Auto-Installed") != "1": continue if (filter_fn is None) or (filter_fn(cls.PackageFilter(name, arch))): auto_installed.add((name, arch)) distro_archs.add(arch) return cls(auto_installed=auto_installed, distro_archs=distro_archs) ``` -------------------------------- ### Create a virtual environment Source: https://siemens.github.io/debsbom/_sources/starting.rst.txt Initializes a new Python virtual environment with system site packages enabled. ```bash python3 -m venv --system-site-packages ``` -------------------------------- ### Execute license-clearing workflow Source: https://siemens.github.io/debsbom/_sources/examples.rst.txt Generate an SBOM from a rootfs and download all associated source packages for license review. ```bash debsbom --progress generate -r path/to/the/rootfs -t cdx -o sbom # output in sbom.cdx.json ``` ```bash debsbom --progress download --outdir downloads --sources sbom.cdx.json # the downloaded files will be in downloads/sources/ ``` -------------------------------- ### Create and Activate Virtual Environment Source: https://siemens.github.io/debsbom/starting.html Commands to initialize and activate a Python virtual environment for the project. ```bash python3 -m venv --system-site-packages ``` ```bash source /bin/activate ``` -------------------------------- ### GET /mr/file/{hash}/info Source: https://siemens.github.io/debsbom/_modules/debsbom/snapshot/client.html Retrieve information about a file by its hash. ```APIDOC ## GET /mr/file/{hash}/info ### Description Retrieve information about a file by hash. ### Method GET ### Endpoint /mr/file/{hash}/info ### Parameters #### Path Parameters - **hash** (string) - Required - The hash of the file to retrieve information for. ``` -------------------------------- ### Package Filtering Utilities Source: https://siemens.github.io/debsbom/_modules/debsbom/dpkg/package.html Utility functions to filter collections of packages by type or installation status. ```python def filter_sources(pkgs: Iterable["Package"]) -> Iterable["SourcePackage"]: """Free function to only return source packages""" return filter(lambda p: p.is_source(), pkgs) def filter_binaries(pkgs: Iterable["Package"]) -> Iterable["BinaryPackage"]: """Free function to only return binary packages""" return filter(lambda p: p.is_binary(), pkgs) def filter_installed(pkgs: Iterable["BinaryPackage"]) -> Iterable["BinaryPackage"]: """Free function to only return installed binary packages.""" return filter(lambda p: p.status in (DpkgStatus.INSTALLED, DpkgStatus.DEBSBOM_UNKNOWN), pkgs) ``` -------------------------------- ### Define DpkgStatus Enum Source: https://siemens.github.io/debsbom/_modules/debsbom/dpkg/package.html Represents the installation status of a Debian package with a helper method to parse status strings from dpkg. ```python class DpkgStatus(Enum): NOT_INSTALLED = "n" CONFIG_FILES = "c" HALF_INSTALLED = "H" UNPACKED = "U" HALF_CONFIGURED = "F" TRIGGERS_AWAITING = "W" TRIGGERS_PENDING = "t" INSTALLED = "i" DEBSBOM_UNKNOWN = "0" @classmethod def from_dpkg(cls, status: str) -> "DpkgStatus": if len(status) > 1: status = status.lower() for checkStatus in DpkgStatus: if checkStatus == DpkgStatus.DEBSBOM_UNKNOWN: continue if status in [checkStatus.value, checkStatus.name.lower().replace("_", "-")]: return checkStatus raise ValueError(f"Unknown dpkg status '{status}'") ``` -------------------------------- ### Generate Local System SBOM Source: https://siemens.github.io/debsbom/_sources/examples.rst.txt Create a CycloneDX SBOM for the current system. ```bash debsbom --progress generate -t cdx -o sbom # output in sbom.cdx.json ``` -------------------------------- ### Download specific packages Source: https://siemens.github.io/debsbom/examples.html Downloads specific packages by providing name, version, and architecture via standard input. ```bash cat < None: """ Check if the required imports for the SBOM type are available. Raises RuntimeError if not available. """ if self == SBOMType.CycloneDX: try: import cyclonedx.model.component except ModuleNotFoundError as e: raise RuntimeError( "Missing dependency: cyclonedx-python-lib; install it by enabling the" "dependency extra `cdx`: `pip install debsbom[cdx]`" ) elif self == SBOMType.SPDX: try: import spdx_tools.spdx.model.package except ModuleNotFoundError as e: raise RuntimeError( "Missing dependency: spdx-tools; install it by enabling the" "dependency extra `spdx`: `pip install debsbom[spdx]`" ) ``` -------------------------------- ### Initialize and Run SecurityScanner Source: https://siemens.github.io/debsbom/_modules/debsbom/securityscan/scanner.html Main scanner class to load a security database and perform vulnerability checks on a collection of source packages. ```python [docs] class SecurityScanner: """ A security scanner that checks source packages against a Debian security database for known vulnerabilities. """ def __init__(self, db: Path, distro: str = "trixie"): with open(db, "r") as f: self.ct = CveTriage(json.load(f), distro=distro) def scan( self, src_pkgs: Iterable[SourcePackage], min_urgency: CveUrgency = CveUrgency.NOT_YET_ASSIGNED, name_filter: str | None = None, ) -> Iterable[ScanResultItem]: for p in src_pkgs: if name_filter and p.name != name_filter: continue vulns = self.ct.candidates(p) for v in vulns: if v.urgency > min_urgency: continue yield ScanResultItem(package=p, vulnerability=v, affected=self.ct.affected_by(p, v)) ``` -------------------------------- ### Download Source Packages Source: https://siemens.github.io/debsbom/examples.html Download source packages based on a generated SBOM. ```bash debsbom --progress download --outdir downloads --sources sbom.cdx.json # the downloaded files will be in downloads/sources/ ``` -------------------------------- ### Download via container image Source: https://siemens.github.io/debsbom/examples.html Executes the download command using the official debsbom Docker image, mounting a local directory for output. ```bash echo "guestfs-tools 1.52.3-1 source" | \ docker run -v$(pwd)/downloads:/mnt/downloads -i ghcr.io/siemens/debsbom:latest \ debsbom download --outdir /mnt/downloads --sources ``` -------------------------------- ### Download Artifacts from SBOM Source: https://siemens.github.io/debsbom/_sources/examples.rst.txt Download binary and source artifacts referenced in an SBOM, or download specific packages directly. ```bash debsbom --progress \ download --outdir downloads --sources --binaries sbom.cdx.json find downloads -mindepth 1 -maxdepth 1 # downloads/.cache <- debsbom metadata to map packages to artifacts # downloads/sources <- files related to source packages (e.g. .dsc, .orig.tar) # downloads/binaries <- .deb files ``` ```bash cat < SourcePackage | None: """Construct a source package from the referenced source dependency.""" if self.source: return SourcePackage(self.source.name, self.source.version[1], self.maintainer) else: return None ``` -------------------------------- ### Repack Artifacts Source: https://siemens.github.io/debsbom/_sources/examples.rst.txt Re-layout downloaded artifacts and recreate the SBOM, or process a subset of packages. ```bash debsbom --progress repack \ --dldir downloads \ --outdir source-archive \ --compress zstd \ --apply-patches \ --validate \ sbom.cdx.json sbom.packed.cdx.json ``` ```bash echo "bash 5.2.37-2 source" | debsbom -v repack sbom-in.json sbom-out.json ``` -------------------------------- ### Initialize SourceArchiveMerger Source: https://siemens.github.io/debsbom/_modules/debsbom/repack/merger.html Constructor for the SourceArchiveMerger class, requiring a download directory and optional output directory and compression format. ```python def __init__( self, dldir: Path, outdir: Path | None = None, compress: Compression.Format = Compression.NONE, ): self.dldir = dldir self.outdir = outdir or dldir self.outdir.mkdir(exist_ok=True, parents=False) self.compress = compress self.dpkg_source = shutil.which("dpkg-source") if not self.dpkg_source: raise RuntimeError("'dpkg-source' from the 'dpkg-dev' package is missing.") ``` -------------------------------- ### Interact with Source Packages Source: https://siemens.github.io/debsbom/_modules/debsbom/snapshot/client.html Classes and methods for querying source package versions, associated files, and binary packages. ```python class Package: """ Source package name (without specific version) """ def __init__(self, sdl: "SnapshotDataLake", name: str): self.sdl = sdl self.name = name [docs] def versions(self) -> Iterable["SourcePackage"]: """ Iterate all versions of a ``SourcePackage``. """ data = self.sdl.get(path=f"/mr/package/{self.name}/").json() for v in data["result"]: yield SourcePackage(self.sdl, self.name, v["version"]) [docs] class SourcePackage: """ Source package in a specific version """ def __init__(self, sdl: "SnapshotDataLake", name: str, version: str): self.sdl = sdl self.name = name self.version = version [docs] def srcfiles( self, archive: str | None = None, sha1: str | None = None ) -> Iterable["SnapshotRemoteFile"]: """ All files associated with the source package. Returns multiple SnaphshotRemoteFile instances for a single hash in case the file is known under multiple names. If the package is not known to the snapshot mirror, raises NotFoundOnSnapshotError. If the filtering does not match any, return empty iterator. """ data = self.sdl.get( path=f"/mr/package/{self.name}/{self.version}/srcfiles?fileinfo=1" ).json() fileinfo = data.get("fileinfo") for s in data.get("result", []): hash = s["hash"] for res in fileinfo[hash]: rf = SnapshotRemoteFile.fromfileinfo(self.sdl, hash, res) if archive and rf.archive_name != archive: continue if sha1 and rf.hash != sha1: continue rf.architecture = "source" yield rf [docs] def binpackages(self) -> Iterable["BinaryPackage"]: """ All binary packages created from this source package """ data = self.sdl.get(path=f"/mr/package/{self.name}/{self.version}/binpackages").json() for b in data.get("result", []): yield BinaryPackage(self.sdl, b["name"], b["version"], self.name, self.version) ``` -------------------------------- ### Implement Package Caching Source: https://siemens.github.io/debsbom/_modules/debsbom/download/resolver.html Provides a base class for caching package resolutions and a persistent file-backed implementation using Zstandard. ```python class PackageResolverCache: """ Maps packages to RemoteFile instances to avoid expensive calls to the upstream mirror. This dummy implementation can be used to not cache. """ [docs] def lookup(self, p: package.SourcePackage | package.BinaryPackage) -> list["RemoteFile"] | None: """Lookup package files in cache""" return None [docs] def insert( self, p: package.SourcePackage | package.BinaryPackage, files: list["RemoteFile"] ) -> None: """Insert package files into cache""" pass ``` ```python class PersistentResolverCache(PackageResolverCache): """ Trivial implementation of a file-backed cache. Each cache entry is stored as individual file in the cachedir. """ def __init__(self, cachedir: Path): self.cachedir = cachedir self.cctx = ZstdCompressor(level=10) self.dctx = ZstdDecompressor() cachedir.mkdir(exist_ok=True) @staticmethod def _package_hash(p: package.Package) -> str: return hashlib.sha256( json.dumps( { "purl": p.purl().to_string(), "checksums": p.checksums, }, sort_keys=True, ).encode("utf-8") ).hexdigest() def _entry_path(self, hash: str) -> Path: return self.cachedir / f"{hash}.json.zst" [docs] def lookup(self, p: package.SourcePackage | package.BinaryPackage) -> list["RemoteFile"] | None: hash = self._package_hash(p) entry = self._entry_path(hash) if not entry.is_file(): logger.debug(f"Package '{p.name}' is not cached") return None with ( open(entry, "rb") as _f, self.dctx.stream_reader(_f) as f, ): try: data = json.load(f) for d in data: d["checksums"] = {ChecksumAlgo(int(k)): v for k, v in d["checksums"].items()} except json.decoder.JSONDecodeError: logger.warning(f"cache file {entry.name} ({p}) is corrupted") return None logger.debug(f"Package '{p.name}' already cached") return [RemoteFile(**d) for d in data] [docs] def insert( self, p: package.SourcePackage | package.BinaryPackage, files: list["RemoteFile"] ) -> None: hash = self._package_hash(p) entry = self._entry_path(hash) with ( open(entry.with_suffix(".tmp"), "wb") as _f, self.cctx.stream_writer(_f) as cf, io.TextIOWrapper(cf, encoding="utf-8") as f, ): json.dump([dataclasses.asdict(rf.as_base()) for rf in files], f) entry.with_suffix(".tmp").rename(entry) ``` -------------------------------- ### Define License and Exception Mappings Source: https://siemens.github.io/debsbom/_modules/debsbom/apt/copyright.html Contains dictionaries for mapping non-standard Debian license strings and exception names to their official SPDX identifiers. ```python WELL_KNOWN_EXPRESSIONS = { "AGPL-1": "AGPL-1.0-only", "AGPL-1+": "AGPL-1.0-or-later", "AGPL-3": "AGPL-3.0-only", "AGPL-3+": "AGPL-3.0-or-later", "Apache-1": "Apache-1.0", "Apache-2": "Apache-2.0", "Artistic": "Artistic-1.0", "Artistic-1": "Artistic-1.0", "Artistic-2": "Artistic-2.0", "BSD-2": "BSD-2-Clause", "BSD-3": "BSD-3-Clause", "BSD-4": "BSD-4-Clause", "BSL": "BSL-1.0", "BSL-1": "BSL-1.0", "CC-BY": "CC-BY-1.0", "CC-BY-1": "CC-BY-1.0", "CC-BY-2": "CC-BY-2.0", "CC-BY-3": "CC-BY-3.0", "CC-BY-4": "CC-BY-4.0", "CC-BY-NC": "CC-BY-NC-1.0", "CC-BY-NC-1": "CC-BY-NC-1.0", "CC-BY-NC-2": "CC-BY-NC-2.0", "CC-BY-NC-3": "CC-BY-NC-3.0", "CC-BY-NC-4": "CC-BY-NC-4.0", "CC-BY-NC-ND": "CC-BY-NC-ND-1.0", "CC-BY-NC-ND-1": "CC-BY-NC-ND-1.0", "CC-BY-NC-ND-2": "CC-BY-NC-ND-2.0", "CC-BY-NC-ND-3": "CC-BY-NC-ND-3.0", "CC-BY-NC-ND-4": "CC-BY-NC-ND-4.0", "CC-BY-NC-SA": "CC-BY-NC-SA-1.0", "CC-BY-NC-SA-1": "CC-BY-NC-SA-1.0", "CC-BY-NC-SA-2": "CC-BY-NC-SA-2.0", "CC-BY-NC-SA-3": "CC-BY-NC-SA-3.0", "CC-BY-NC-SA-4": "CC-BY-NC-SA-4.0", "CC-BY-SA": "CC-BY-SA-1.0", "CC-BY-SA-1": "CC-BY-SA-1.0", "CC-BY-SA-2": "CC-BY-SA-2.0", "CC-BY-SA-3": "CC-BY-SA-3.0", "CC-BY-SA-4": "CC-BY-SA-4.0", "CC0": "CC0-1.0", "CC0-1": "CC0-1.0", "CDDL": "CDDL-1.0", "CDDL-1": "CDDL-1.0", "CPL": "CPL-1.0", "CPL-1": "CPL-1.0", "EFL": "EFL-1.0", "EFL-1": "EFL-1.0", "EFL-2": "EFL-2.0", "EPL": "EPL-1.0", "EPL-1": "EPL-1.0", "EPL-2": "EPL-2.0", "Expat": "MIT", "GFDL": "GFDL-1.1-only", "GFDL+": "GFDL-1.1-or-later", "GFDL-1.1": "GFDL-1.1-only", "GFDL-1.1+": "GFDL-1.1-or-later", "GFDL-1.2": "GFDL-1.2-only", "GFDL-1.2+": "GFDL-1.2-or-later", "GFDL-1.3": "GFDL-1.3-only", "GFDL-1.3+": "GFDL-1.3-or-later", "GFDL-NIV": "GFDL-1.1-no-invariants-only", "GFDL-NIV+": "GFDL-1.1-no-invariants-or-later", "GFDL-NIV-1.1": "GFDL-1.1-no-invariants-only", "GFDL-NIV-1.1+": "GFDL-1.1-no-invariants-or-later", "GFDL-NIV-1.2": "GFDL-1.2-no-invariants-only", "GFDL-NIV-1.2+": "GFDL-1.2-no-invariants-or-later", "GFDL-NIV-1.3": "GFDL-1.3-no-invariants-only", "GFDL-NIV-1.3+": "GFDL-1.3-no-invariants-or-later", "GPL": "GPL-1.0-only", "GPL+": "GPL-1.0-or-later", "GPL-1": "GPL-1.0-only", "GPL-1+": "GPL-1.0-or-later", "GPL-2": "GPL-2.0-only", "GPL-2+": "GPL-2.0-or-later", "GPL-3": "GPL-3.0-only", "GPL-3+": "GPL-3.0-or-later", "GPL3+": "GPL-3.0-or-later", "LGPL": "LGPL-1.0-only", "LGPL+": "LGPL-1.0-or-later", "LGPL-1": "LGPL-1.0-only", "LGPL-1+": "LGPL-1.0-or-later", "LGPL-2": "LGPL-2.0-only", "LGPL-2+": "LGPL-2.0-or-later", "LGPL-3": "LGPL-3.0-only", "LGPL-3+": "LGPL-3.0-or-later", "LPGL-2.1": "LGPL-2.1-only", "LPGL-2.1+": "LGPL-2.1-or-later", "LPPL": "LPPL-1.0", "LPPL-1": "LPPL-1.0", "MPL": "MPL-1.0", "MPL-1": "MPL-1.0", "MPL-2": "MPL-2.0", "PSF-2": "PSF-2.0", "Perl": "Artistic-1.0-Perl", "Python": "Python-2.0", "QPL": "QPL-1.0", "QPL-1": "QPL-1.0", "RubyLicense": "Ruby", "SIL-1": "OFL-1.0", "SIL-1.1": "OFL-1.1", "SIL-OFL-1": "OFL-1.0", "SIL-OFL-1.1": "OFL-1.1", "Zope": "ZPL-1.0", "Zope-1": "ZPL-1.0", "Zope-2": "ZPL-2.0", } WELL_KNOWN_EXCEPTIONS = { "GPL-3+ with Bison exception": "GPL-3.0-or-later WITH Bison-exception-2.2", "GPL-3+ with Bison-2.2 exception": "GPL-3.0-or-later WITH Bison-exception-2.2", "GPL-2 with Linux-syscall-note exception": "GPL-2.0-only WITH Linux-syscall-note", "GPL-3+ with texinfo exception": "GPL-3.0-or-later WITH Texinfo-exception", "GPL-3.0-or-later-WITH-Autoconf-exception-macro": ( "GPL-3.0-or-later WITH Autoconf-exception-macro" ), "BSD-3-clause-Cambridge with BINARY LIBRARY-LIKE PACKAGES exception": ( "BSD-3-Clause WITH PCRE2-exception" ), "GPL-3+-WITH-BISON-EXCEPTION": "GPL-3.0-or-later WITH Bison-exception-2.2", } ``` -------------------------------- ### Generate CycloneDX SBOM Source: https://siemens.github.io/debsbom/examples.html Generate a CycloneDX SBOM from a root filesystem. ```bash debsbom --progress generate -r path/to/the/rootfs -t cdx -o sbom # output in sbom.cdx.json ``` -------------------------------- ### Identify and process new components Source: https://siemens.github.io/debsbom/_sources/examples.rst.txt Extract new PURLs from SBOMs and use them to download, merge, or repack source packages. ```bash PURLS=$( \ diff -U0 \ <(jq -r --sort-keys '.components[].purl' sbom.old.cdx.json) \ <(jq -r --sort-keys '.components[].purl' sbom.cdx.json) \ | grep ^+pkg | sed 's/^+//' \ ) ``` ```bash echo "$PURLS" | debsbom download --sources --binaries ``` ```bash echo "$PURLS" | debsbom source-merge --apply-patches ``` ```bash echo "$PURLS" | debsbom repack \ --apply-patches sbom.cdx.json \ sbom.cdx.repacked.json ``` -------------------------------- ### debsbom download Source: https://siemens.github.io/debsbom/commands/download.html Processes a SBOM and downloads the referenced packages. If no SBOM is provided, it reads line separated entries (name version arch) from stdin. ```APIDOC ## Command: debsbom download ### Description Processes a SBOM and downloads the referenced packages. If no SBOM is provided, it reads line separated entries (name version arch) from stdin to define what shall be downloaded. ### Usage `debsbom download [-h] [-t {cdx,spdx}] [--outdir OUTDIR] [--sources] [--binaries] [--skip-pkgs SKIP] [--resolver {debian-snapshot}] [bomin]` ### Positional Arguments - **bomin** (string) - Optional - sbom file(s) to process. Use '-' to read from stdin. ### Named Arguments - **-t, --sbom-type** (string) - Optional - SBOM type to process (cdx, spdx). Required when reading from stdin. - **--outdir** (string) - Optional - Directory to store downloaded files. Default: 'downloads'. - **--sources** (boolean) - Optional - Operate only on source packages (skip binaries). - **--binaries** (boolean) - Optional - Operate only on binary packages (skip sources). - **--skip-pkgs** (string) - Optional - Packages to exclude from the download, in package-list format. - **--resolver** (string) - Optional - Resolver to use to find upstream packages (default: 'debian-snapshot'). ### JSON Output Schema When run with JSON output enabled, the command emits objects with the following fields: - **status** (string) - The status of the download operation ('ok', 'not_found', 'checksum_mismatch'). - **package** (object) - Object identifying the package (name, version, purl). - **filename** (string) - The name of the processed file. - **path** (string) - The absolute path to the downloaded file. ``` -------------------------------- ### Interact with Binary Packages Source: https://siemens.github.io/debsbom/_modules/debsbom/snapshot/client.html Class for representing binary packages and retrieving their associated files. ```python class BinaryPackage: """ Binary package in a specific version """ def __init__( self, sdl: "SnapshotDataLake", binname: str, binversion: str, srcname: str | None, srcversion: str | None, ): self.sdl = sdl self.binname = binname self.binversion = binversion self.srcname = srcname self.srcversion = srcversion [docs] def files(self, arch: str | None = None) -> Iterable["SnapshotRemoteFile"]: """ All files associated with this binary package (e.g. per-architecture) If no architecture is specified, all packages are returned. Otherwise, only the packages with the matching architecture are returned. If we have information about the source package as well, we precisely resolve the binary package including the original path on the debian mirror. If not, we just resolve the file. The difference is only in the metadata, the file itself is the same in both cases. ``` -------------------------------- ### trace-path command Source: https://siemens.github.io/debsbom/commands/trace-path.html Processes an SBOM and a package identifier and emits paths from this package to the root. ```APIDOC ## trace-path ### Description Processes an SBOM and a package identifier and emits paths from this package to the root. ### Usage `debsbom trace-path [-h] [-t {cdx,spdx}] [--format {text,json,ref,dot}] [--mode {shortest,all-shortest,all}] [bomin] source` ### Positional Arguments - **bomin** (string) - Required - SBOM file(s) to process. Use '-' to read from stdin. - **source** (string) - Required - Source node (PURL). ### Named Arguments - **-t, --sbom-type** (choice) - Optional - SBOM type to process (cdx, spdx). Required when reading from stdin. - **--format** (choice) - Optional - Path output format (text, json, ref, dot). Default: 'text'. - **--mode** (choice) - Optional - Path finding mode (shortest, all-shortest, all). Default: 'shortest'. ### JSON Output Schema When using `--format json`, output adheres to the following schema: - **name** (string) - Name of the component - **ref** (string) - Reference identifier for the component - **version** (string|null) - Version of the component - **maintainer** (string|null) - Maintainer of the component - **purl** (string|null) - Package URL of the component ``` -------------------------------- ### Activate the virtual environment Source: https://siemens.github.io/debsbom/_sources/starting.rst.txt Activates the created virtual environment for the current shell session. ```bash source /bin/activate ``` -------------------------------- ### Implement PackageDownloader Source: https://siemens.github.io/debsbom/_modules/debsbom/download/download.html Manages the retrieval of package artifacts, ensuring files are only downloaded once by checking the local directory. ```python class PackageDownloader: """ Retrieve package artifacts from upstream. Files are only retrieved once by comparison with the data in the local downloads directory. """ def __init__( self, outdir: Path | str = "downloads", session: requests.Session = requests.Session(), ): self.outdir = Path(outdir) self.sources_dir = self.outdir / "sources" self.binaries_dir = self.outdir / "binaries" self.to_download: list[tuple[package.Package, RemoteFile]] = [] self.rs = session self.known_hashes = {} self.outdir.mkdir(exist_ok=True) for p in [self.sources_dir, self.binaries_dir]: p.mkdir(exist_ok=True) def _target_path(self, pkg: package.Package, f: RemoteFile): if pkg.is_source(): base = self.sources_dir else: base = self.binaries_dir base = base.resolve() target = (base / f.archive_name / f.filename).resolve() if not target.is_relative_to(base): purl = pkg.purl() raise ValueError( f"{purl}: malicious path in remote file detected: '{f.archive_name}/{f.filename}'" ) else: return target def register(self, files: list[RemoteFile], package: package.Package) -> None: """Register a list of files corresponding to a package for download.""" self.to_download.extend([(package, f) for f in files]) def stat(self) -> StatisticsType: """ Returns a tuple (files to download, total size, cached files, cached bytes) """ unique_dl = list( {frozenset(v.checksums.items()): (pkg, v) for pkg, v in self.to_download}.values() ) nbytes = reduce(lambda acc, x: acc + x[1].size if x[1].size else 0, unique_dl, 0) cfiles = list(filter(lambda x: self._target_path(x[0], x[1]).is_file(), unique_dl)) cbytes = reduce(lambda acc, x: acc + x[1].size if x[1].size else 0, cfiles, 0) return StatisticsType(len(unique_dl), nbytes, len(cfiles), cbytes) ``` -------------------------------- ### trace-path command usage Source: https://siemens.github.io/debsbom/commands/trace-path.html Displays the command-line syntax for the trace-path command, including positional and named arguments. ```bash usage: debsbom trace-path [-h] [-t {cdx,spdx}] [--format {text,json,ref,dot}] [--mode {shortest,all-shortest,all}] [bomin] source ``` -------------------------------- ### Create SourcePackage from deb822 Source: https://siemens.github.io/debsbom/_modules/debsbom/dpkg/package.html Parses a deb822 representation to instantiate a SourcePackage, handling VCS information and checksums. ```python @staticmethod def from_deb822(package) -> "SourcePackage": """ Create a package from a deb822 representation. If the deb822 input is a .dsc file, the name is read from the source property. """ name = package.get("Source") or package["Package"] version = Version(package.get("Version")) maintainer = package.get("Maintainer") binary_entry = package.get("Binary") if binary_entry is not None: binaries = [b.strip() for b in binary_entry.split(",")] else: binaries = [] homepage = package.get("Homepage") vcs = None for type in VcsType: locator = package.get(f"Vcs-{type.name}") if locator: if vcs: logger.warning( f"Multiple VCS types found for package {name}: {vcs.type} and {type}" ) # remove additional details (e.g., branch, path) locator = locator.split()[0] vcs = VcsInfo(type, locator) # Checksums according to Debian policy 5.6.24 pkg_chksums = {} chksums = checksums_from_dsc(package) for c in filter(lambda c_: c_.endswith(".dsc"), chksums): pkg_chksums = chksums[c] return SourcePackage( name=name, version=version, maintainer=maintainer, binaries=binaries, homepage=homepage, vcs=vcs, checksums=pkg_chksums, ) ``` -------------------------------- ### Export command usage Source: https://siemens.github.io/debsbom/commands/export.html Displays the command-line syntax for the export operation. ```bash usage: debsbom export [-h] [-t {cdx,spdx}] [--format {graphml}] [bomin] [out] ``` -------------------------------- ### PackageResolver.create Source: https://siemens.github.io/debsbom/_modules/debsbom/resolver/resolver.html Factory method to create a PackageResolver instance from a file path. It automatically detects the SBOM type based on the file extension. ```APIDOC ## PackageResolver.create(filename: Path, bomtype: SBOMType | None = None) ### Description Creates a PackageResolver instance for the given SBOM file. The resolver is used to iteratively process packages contained within the SBOM. ### Parameters - **filename** (Path) - Required - The path to the SBOM file. - **bomtype** (SBOMType | None) - Optional - The specific SBOM type if not detectable from the file extension. ``` -------------------------------- ### debsbom.resolver.resolver.PackageStreamResolver Source: https://siemens.github.io/debsbom/api.html Handles universal package ingress and emits partial package instances. ```APIDOC ## PackageStreamResolver(pkgstream: IOBase) ### Description Initializes the resolver to handle universal package ingress from the provided stream. ``` -------------------------------- ### source-merge command usage Source: https://siemens.github.io/debsbom/commands/source-merge.html Displays the command-line interface syntax and available arguments for the source-merge command. ```text usage: debsbom source-merge [-h] [-t {cdx,spdx}] [--compress {no,bzip2,gzip,xz,zstd,lz4}] [--apply-patches] [--mtime MTIME] [--pkgdir PKGDIR] [--outdir OUTDIR] [bomin] ```