Repository navigation
[pipeline-manager] Check memory and storage autoscaling limits in the api-server - #7819
Conversation
|
Review summary
(true, "POST", "/pipelines", with_env("e"), 400),
+ (true, "PUT", "/pipelines/p", pipeline("p", json!({"memory_max": "1000M"})), 200), // boundary
+ (true, "PATCH", "/pipelines/p", patch(json!({"memory_max": "1001M"})), 400),
+ (true, "PATCH", "/pipelines/p", patch(json!({"memory_mb_max": 2000})), 400), // legacy field
+ (false, "PATCH", "/pipelines/p", patch(json!({"memory_min": "2G", "memory_max": "1G"})), 400),
+ (true, "PATCH", "/pipelines/p", json!({"runtime_config": {"env": {"A": "1"}}}), 400),
+ (true, "PUT", "/pipelines/p", with_env("p"), 400),
+ (true, "PATCH", "/pipelines/p", json!({"runtime_config": {"env": {}}}), 200), // empty env allowedPlatform tests were not run here (no k8s). Otherwise the logic, the actionable error messages, and the docs look good. |
dbc9d3c to
e9013fb
Compare
e9013fb to
607d9d1
Compare
|
LGTM. What I ran (debug, merge commit):
Gates: tests cover the changed behaviour, nothing flaky (no sleeps, and each case builds its own state), docs and changelog are updated, the Missing tests (low, optional): at the endpoint level, the storage-autoscaling refusal is only tested for |
607d9d1 to
17c8b4f
Compare
17c8b4f to
1b2fb3f
Compare
… api-server POST, PUT, and PATCH return 400 InvalidRuntimeConfig for a runtime_config with memory above the Kubernetes runner's maximum, or with both storage_min and storage_max when the runner does not enable storage autoscaling. A PATCH of deployment_config refuses memory above the maximum. Before, the next start failed, and a refused resize only went to the pipeline log. Both flags are unset by default and then not checked. On all installations, memory_min above memory_max is now refused.
1b2fb3f to
3bc6736
Compare
POST,PUT, andPATCH /v0/pipelinesnow refuse a runtime config with memory above the runner's maximum, or with storage autoscaling when the runner does not enable it, with a 400InvalidRuntimeConfig. APATCHofdeployment_configthat resizes a running pipeline refuses memory above the maximum, with a 400InvalidDeploymentPatch. Before, a resize the runner refused returned 200, went only to the pipeline log, and failed the next start.resources.memory_min,memory_max--upper-limit-memory-megabytekubernetesRunner.upperLimitMemoryMegabytestorage_minandstorage_max--enable-storage-autoscalingkubernetesRunner.enableStorageAutoscalingWith #7712 (storage size and class allowlists) and #7656 (the resize
PATCHchecks them), every field that can change while a pipeline runs is checked against the runner's limits. CPU has no runner limit.memory_minabovememory_maxis refused, asstorage_minabovestorage_maxalready is. A memory refusal reads "unsupported memory: ...".Settings that cannot change while a pipeline runs, such as
resources.namespace,service_account_name, orenv, are left to the runner, which refuses them at start with a message that names the Helm value.The Helm side is feldera/cloud#2165, which must merge after this.
Testing: unit tests for each refusal and for parsing the flags as the chart passes them, and endpoint tests for
POST,PUT,PATCHofruntime_config(including the deprecatedmemory_mb_maxand an installation without limits), andPATCHofdeployment_config. Each check was removed in turn, and a test failed. Platform tests were not run.