Repository navigation
ci: run CodSpeed benchmarks in parallel with pytest-xdist - #668
shenxianpeng merged 1 commit into
Conversation
Split the benchmark run across processes with pytest-xdist (-n auto) so the single CodSpeed job stops serializing every benchmark under Valgrind. - codspeed.yml: run 'pytest tests/ --codspeed -n auto' (simulation mode). - pyproject.toml: add pytest-xdist to the [test] extra (the benchmark job installs .[test]); uv.lock regenerated (adds pytest-xdist, execnet). CodSpeed documents this exact command for parallel runs in one job. Full suite passes locally under the command.
|
No actionable comments were generated in the recent review. Walkthrough
Merge Risk:
|
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #668 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 14 14
Lines 2495 2495
Branches 484 484
=========================================
Hits 2495 2495 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Merging this PR will degrade performance by 11.1%3 warnings
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|




Closes #663.
The
Run benchmarksCodSpeed job runs the whole benchmark set serially under Valgrind (mode: simulation), which is why it dominates PR time (~54 min on the #657 run). This splits that one job across processes with pytest-xdist, which CodSpeed documents for exactly this case.Change
.github/workflows/codspeed.yml:run: pytest tests/ --codspeed -n auto.pyproject.toml: addpytest-xdistto the[test]extra — the benchmark job installs.[test], so the plugin has to be declared there.uv.lockregenerated (addspytest-xdist,execnet).Every benchmark still runs; CodSpeed aggregates the workers into one report. The CodSpeed Python docs give this exact command (
pytest tests/ --codspeed -n auto) withmode: simulationfor "running benchmarks in parallel processes."Verification
pytest tests/ --codspeed -n autoruns green locally (full suite, exit 0) — the suite is parallel-safe, so no benchmark depends on serial ordering.README pin: unchanged — it already pins
v2.20.0, which matches the current PyPI release, so nothing to bump here.The bigger lever (your call). The CodSpeed docs actually present process-parallelism as the fallback; their first-choice is sharding across CI jobs with
pytest-test-groups(amatrix: shard: [1..N]with--test-group/--test-group-count), which parallelizes across runners rather than within one. That's a larger change tocodspeed.ymland it's your CI infra, so I've kept this PR to the minimal in-job-n auto. Happy to follow up with the sharded-matrix version (or swap this PR to it) if you'd prefer that route.Summary by CodeRabbit