Repository navigation
[ci] Add the npm packages of the web-console to the image SBOM - #7883
Conversation
syft cannot see the npm packages that the pipeline-manager binary contains. A fork of rollup-plugin-sbom now lists the packages in the web-console and profiler-app bundles, including the packages that other packages vendor (e.g. fflate in posthog-js). scripts/sbom-merge.mts adds them to the SPDX image SBOM. - `bun run sbom` writes CycloneDX SBOMs of both bundles to build/sbom/. - The source SBOM skips npm packages, and syft is pinned to v1.55.0. - apache-arrow uses the npm release 21.2.0 in place of a fork. Signed-off-by: Karakatiza666 <bulakh.96@gmail.com>
|
|
||
| # syft cannot see the npm packages that the pipeline-manager binary contains. | ||
| - name: Add npm packages to image SBOM | ||
| run: node scripts/sbom-merge.mts "feldera-sbom-image-${GITHUB_SHA}.spdx.json" js-sbom/*.cdx.json |
There was a problem hiding this comment.
pitfall (medium): This job runs on ubuntu-latest-amd64 with whatever node the runner image ships, and nothing sets up Node. Running a .mts file with type annotations needs Node ≥ 22.18, where type stripping is on by default. On an image whose default is still Node 20, this step fails, and it fails only in the merge queue because PR checks do not run this workflow. Please pin Node with actions/setup-node (node-version: 24), or run this step in the feldera-dev container like generate-js-sbom and use bun.
There was a problem hiding this comment.
I checked that the runner has at least Node 22
|
What I ran
The licenses of the merged packages contain no GPL or AGPL. One package is EPL-2.0 (weak copyleft, already a dependency before this PR). 5 packages get
The PR's tests cover the merge script well: dedup, qualifiers, vendored packages, idempotence and error paths. I found no gaps that need new tests. The only blocking item is the Node runtime in the inline comment. |
jyotshnayaparla-00
left a comment
There was a problem hiding this comment.
Workflow changes look good.
Summary
The pipeline-manager binary contains the web-console, but syft cannot see the npm packages in the binary. So the image SBOM listed none of them. This PR adds them.
bun run sbombuilds the web-console and profiler-app with a fork of rollup-plugin-sbom. The plugin lists only the npm packages that are in the bundles. The fork also detects vendored packages: packages whose code another package copies into its own files, e.g.fflateinposthog-jsordompurify@3.2.7inmonaco-editor. It writes CycloneDX SBOMs tobuild/sbom/.scripts/sbom-merge.mtsdedupes the packages of both SBOMs by purl and adds them to the SPDX image SBOM. The image rootCONTAINSeach package, as for the packages that syft finds.The release attaches no new files. The CycloneDX SBOMs stay as the
feldera-sbom-jsworkflow artifact.Changes
scripts/sbom.tsJS_SBOM=1. Sets the root component topkg:generic/feldera/<app>@<release version>scripts/sbom-merge.mtsjs-packages/*/vite.config.ts,package.jsonbuild:sbomscripts. The plugin runs only whenJS_SBOMis set, so the shipped bundles do not contain SBOM files.github/workflows/generate-sbom.ymlgenerate-js-sbom. The image SBOM job waits for it and merges its output.github/workflows/test-unit.ymlbun test scripts/testsEffects on the source SBOM
The source SBOM step pins syft to v1.55.0 (the version in the feldera-dev image) and runs with
--select-catalogers "-javascript". The action default was v1.14.1, which cannot readbun.lock.docs.feldera.com/package-lock.json. The docs site does not ship in the productTo keep the docs packages, use
--exclude './bun.lock'in place of-javascript.Dependencies
rollup-plugin-sbomisgithub:Karakatiza666/rollup-plugin-sbom#4487e0e. This is a temporary solution until upstream merges the vendored package detection. The ref is a 7-character SHA because Bun 1.3.14--frozen-lockfilerejects a branch name, a full SHA and a tag for this dependency in this monorepo.apache-arrowchanges fromgithub:Karakatiza666/arrow-js#distto the npm release21.2.0. This release contains the LargeList support that the fork carried.Testing
scripts/tests/: 57 unit tests for both scripts. The merge tests run on Node.js and Bun. Each tested behavior was broken once to confirm that a test fails.svelte-checkand the ad-hoc query tests pass withapache-arrow@21.2.0.ci.ymlrunsgenerate-sbom.ymlonly in the merge queue, so the PR checks do not run the new SBOM jobs.