← 输入框|github.com|刷新
Skip to content

[ci] Add the npm packages of the web-console to the image SBOM - #7883

Merged
Karakatiza666 merged 1 commit into
mainfrom
js-sbom
Nov 6, 2026
Merged

Karakatiza666 merged 1 commit into
mainfrom
js-sbom

Conversation

@Karakatiza666

@Karakatiza666 Karakatiza666 commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The pipeline-manager binary contains the web-console, but syft cannot see the npm packages in the binary. So the image SBOM listed none of them. This PR adds them.

  1. bun run sbom builds the web-console and profiler-app with a fork of rollup-plugin-sbom. The plugin lists only the npm packages that are in the bundles. The fork also detects vendored packages: packages whose code another package copies into its own files, e.g. fflate in posthog-js or dompurify@3.2.7 in monaco-editor. It writes CycloneDX SBOMs to build/sbom/.
  2. scripts/sbom-merge.mts dedupes the packages of both SBOMs by purl and adds them to the SPDX image SBOM. The image root CONTAINS each package, as for the packages that syft finds.

The release attaches no new files. The CycloneDX SBOMs stay as the feldera-sbom-js workflow artifact.

Changes

File Change
scripts/sbom.ts Builds both apps with JS_SBOM=1. Sets the root component to pkg:generic/feldera/<app>@<release version>
scripts/sbom-merge.mts Adds the npm packages to an SPDX image SBOM. Runs on Node.js 24 and on Bun
js-packages/*/vite.config.ts, package.json build:sbom scripts. The plugin runs only when JS_SBOM is set, so the shipped bundles do not contain SBOM files
.github/workflows/generate-sbom.yml New job generate-js-sbom. The image SBOM job waits for it and merges its output
.github/workflows/test-unit.yml Runs bun test scripts/tests

Effects on the source SBOM

The source SBOM step pins syft to v1.55.0 (the version in the feldera-dev image) and runs with --select-catalogers "-javascript". The action default was v1.14.1, which cannot read bun.lock.

Package type Before After Cause
npm 1371 0 All 1371 came from docs.feldera.com/package-lock.json. The docs site does not ship in the product
python 10 153 Newer syft
java-archive 36 35 Newer syft
rust 1290 1290

To keep the docs packages, use --exclude './bun.lock' in place of -javascript.

Dependencies

  • rollup-plugin-sbom is github:Karakatiza666/rollup-plugin-sbom#4487e0e. This is a temporary solution until upstream merges the vendored package detection. The ref is a 7-character SHA because Bun 1.3.14 --frozen-lockfile rejects a branch name, a full SHA and a tag for this dependency in this monorepo.
  • apache-arrow changes from github:Karakatiza666/arrow-js#dist to the npm release 21.2.0. This release contains the LargeList support that the fork carried.

Testing

  • scripts/tests/: 57 unit tests for both scripts. The merge tests run on Node.js and Bun. Each tested behavior was broken once to confirm that a test fails.
  • Merged the SBOMs into the v0.378.0 image SBOM: 110 npm packages added, no duplicate SPDX IDs. syft 1.55 reads the result.
  • Frozen install with Bun 1.3.14 on this branch.
  • web-console: svelte-check and the ad-hoc query tests pass with apache-arrow@21.2.0.

ci.yml runs generate-sbom.yml only in the merge queue, so the PR checks do not run the new SBOM jobs.

syft cannot see the npm packages that the pipeline-manager binary
contains. A fork of rollup-plugin-sbom now lists the packages in the
web-console and profiler-app bundles, including the packages that other
packages vendor (e.g. fflate in posthog-js). scripts/sbom-merge.mts adds
them to the SPDX image SBOM.

- `bun run sbom` writes CycloneDX SBOMs of both bundles to build/sbom/.
- The source SBOM skips npm packages, and syft is pinned to v1.55.0.
- apache-arrow uses the npm release 21.2.0 in place of a fork.

Signed-off-by: Karakatiza666 <bulakh.96@gmail.com>

# syft cannot see the npm packages that the pipeline-manager binary contains.
- name: Add npm packages to image SBOM
run: node scripts/sbom-merge.mts "feldera-sbom-image-${GITHUB_SHA}.spdx.json" js-sbom/*.cdx.json

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pitfall (medium): This job runs on ubuntu-latest-amd64 with whatever node the runner image ships, and nothing sets up Node. Running a .mts file with type annotations needs Node ≥ 22.18, where type stripping is on by default. On an image whose default is still Node 20, this step fails, and it fails only in the merge queue because PR checks do not run this workflow. Please pin Node with actions/setup-node (node-version: 24), or run this step in the feldera-dev container like generate-js-sbom and use bun.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked that the runner has at least Node 22

@mythical-fred-oss

Copy link
Copy Markdown
Contributor

What I ran

Check Result
bun install --frozen-lockfile (Bun 1.3.14) ✅
bun test scripts/tests ✅ 57/57 (Node 24 and Bun)
bun run sbom ✅ wrote both .cdx.json files (version 0.379.0)
sbom-merge.mts on a synthetic SPDX image ✅ 110 packages added, 110 unique SPDX IDs; a second run adds 0
apache-arrow@21.2.0: LargeList IPC round-trip ✅
web-console test-unit 497/497 client/server tests pass. Browser tests (including Query.svelte.spec.ts) did not run: no Playwright browser here
pre-commit run --files <changed> ✅

The licenses of the merged packages contain no GPL or AGPL. One package is EPL-2.0 (weak copyleft, already a dependency before this PR). 5 packages get NOASSERTION, including posthog-js.

  • tech-debt (medium): the build now depends on github:Karakatiza666/rollup-plugin-sbom, a fork in a personal account. It is MIT, a devDependency with no install scripts, and pinned by hash in bun.lock. Please move the fork to the feldera org, or open an issue to track the upstream merge, so that the dependency does not rely on one person's account.

The PR's tests cover the merge script well: dedup, qualifiers, vendored packages, idempotence and error paths. I found no gaps that need new tests. The only blocking item is the Node runtime in the inline comment.

@jyotshnayaparla-00 jyotshnayaparla-00 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Workflow changes look good.

@Karakatiza666
Karakatiza666 added this pull request to the merge queue Nov 6, 2026
Merged via the queue into main with commit 6b35396 Nov 6, 2026
3 checks passed
@Karakatiza666
Karakatiza666 deleted the js-sbom branch November 6, 2026 23:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants