← 输入框|github.com|刷新
Skip to content

Fix TwitterBenchmark which only measured a validation error - #4499

Open
Meemaw wants to merge 1 commit into
graphql-java:masterfrom
Meemaw:fix/twitter-benchmark-depth-limit
Open

Meemaw wants to merge 1 commit into
graphql-java:masterfrom
Meemaw:fix/twitter-benchmark-depth-limit

Conversation

@Meemaw

@Meemaw Meemaw commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Problem

TwitterBenchmark builds a query that is 150 levels deep. Since e4c287c ("Add query depth and field count limits to Validator") validation enforces QueryComplexityLimits with a default maxDepth of 100, so every benchmark execution fails with:

MaxQueryDepthExceeded: Query depth 101 exceeds maximum allowed depth 100

and returns data == null. Because the benchmark uses a persisted query cache, it has been measuring the return of a cached validation error (~8M ops/s) rather than execution, and those numbers flow into the performance results history.

Fix

  • Pass a QueryComplexityLimits with maxDepth(DEPTH + 1) through the request's GraphQLContext (QueryComplexityLimits.KEY) rather than changing the global default, so other benchmarks in the same JVM are unaffected.
  • Execute the query once during class initialisation and fail fast if it returns errors, so the benchmark cannot silently degrade into measuring an error path again.

Verification

  • Before: query returns MaxQueryDepthExceeded, data == null; benchmark reports ~8,090,000 ops/s.
  • After: query returns no errors and fully nested data (150 levels, 150 leaves at the bottom); benchmark reports 359 ± 12 ops/s (~2.8 ms per ~22.6k field query, Apple Silicon, JDK 21).
  • Temporarily lowering the limit to DEPTH - 1 makes class initialisation throw ExceptionInInitializerError, confirming the fail-fast check works.
  • graphql.archunit.* tests pass (JMH fork rule).

Note: the historical TwitterBenchmark numbers recorded since the limits were introduced are not comparable with results after this change.

The benchmark query is 150 levels deep, but since validation started
enforcing QueryComplexityLimits (default maxDepth=100) every execution
failed with MaxQueryDepthExceeded and returned no data. The benchmark
was therefore measuring a cached validation error (~8M ops/s) rather
than execution.

Raise the max depth for the benchmark requests via the GraphQLContext
and fail fast at class initialisation if the query does not execute
cleanly, so the benchmark cannot silently regress like this again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Meemaw
Meemaw marked this pull request as ready for review October 17, 2026 19:55
@github-actions

Copy link
Copy Markdown
Contributor

Test Report

Test Results

Java Version Total Passed Failed Errors Skipped
Java 11 5962 (-20 🔴) 5906 (-20 🔴) 0 (±0) 0 (±0) 56 (±0)
Java 17 5962 (-20 🔴) 5905 (-20 🔴) 0 (±0) 0 (±0) 57 (±0)
Java 21 5962 (-20 🔴) 5905 (-20 🔴) 0 (±0) 0 (±0) 57 (±0)
Java 25 5962 (-20 🔴) 5905 (-20 🔴) 0 (±0) 0 (±0) 57 (±0)
jcstress 32 (±0) 32 (±0) 0 (±0) 0 (±0) 0 (±0)
Total 23880 (-80 🔴) 23653 (-80 🔴) 0 (±0) 0 (±0) 227 (±0)

Code Coverage (Java 25)

Metric Covered Missed Coverage vs Master
Lines 30140 3118 90.6% ±0.0%
Branches 8838 1520 85.3% ±0.0%
Methods 8048 1207 87.0% ±0.0%

Changed Class Coverage (2 classes)

Class Line Branch Method
g.e.ExecutionStrategy ±0.0% -0.8% 🔴 ±0.0%
g.s.i.TypeDefinitionRegistry -0.3% 🔴 ±0.0% -1.3% 🔴
ExecutionStrategy — method details
Method Line Branch
lambda$buildFieldValueMap$0 100.0% 90.0% (-10.0% 🔴)
TypeDefinitionRegistry — method details
Method Line Branch
getType 0.0% (-100.0% 🔴) —

Full HTML report: build artifact jacoco-html-report

Updated: 2026-10-19 07:19:35 UTC

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant