Repository navigation
fix: Bound Arrow Flight batch size in the offline server - #6896
Merged
ntkathole merged 2 commits intoOct 1, 2026
Merged
Conversation
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #6896 +/- ##
==========================================
+ Coverage 48.53% 48.56% +0.03%
==========================================
Files 427 427
Lines 53774 53787 +13
Branches 7832 7834 +2
==========================================
+ Hits 26097 26120 +23
+ Misses 25814 25803 -11
- Partials 1863 1864 +1
Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
5 tasks done
ntkathole
approved these changes
Oct 1, 2026
The offline server's do_get and do_exchange handlers each returned a single unbounded Arrow Table/RecordBatch to the Flight client. Once a result approaches the IPC layer's 2GiB-per-batch limit, Flight raises "Cannot send record batches exceeding 2GiB yet" and the request fails outright instead of degrading gracefully. Add module-level _split_oversized/_bounded_reader helpers that re-batch any Table or RecordBatchReader so no single RecordBatch exceeds 1.5GB, and route both do_get's RecordBatchStream and do_exchange's writer.write_batch loop through them. do_exchange is the newer HPA-safe read path (bae07fc) and was still writing one unbounded table via write_table, so it needed the same fix as do_get. Signed-off-by: dbbvitor <vitor.diniz@gympass.com>
cosmic-ray's mutation run over the diff found one surviving mutant: `half = batch.num_rows // 2` mutated to `/`. A real pyarrow RecordBatch.slice() silently truncates a float offset/length to the same int, so no test built on real batches can observe the difference. Add a duck-typed fake batch that records the exact slice() call arguments and asserts they stay int, which does distinguish the two operators. Signed-off-by: dbbvitor <vitor.diniz@gympass.com>
ntkathole
force-pushed
the
fix/offline-server-bounded-batches
branch
from
October 1, 2026 13:36
ff4d4ef to
1de36b2
Compare
ntkathole
pushed a commit
that referenced
this pull request
Oct 13, 2026
# [0.67.0](v0.66.0...v0.67.0) (2026-10-13) ### Bug Fixes * Allow dill 0.4 so UDF serialization works on Python 3.14 ([#6858](#6858)) ([139704f](139704f)) * Allow serving features while a feature view is MATERIALIZING ([#6789](#6789)) ([e2d7fe2](e2d7fe2)), closes [#6780](#6780) * Allowlist the pinned commit SHA flagged by detect-secrets ([4b81d18](4b81d18)) * Apply point-in-time filter correctly with the Polars backend ([#6992](#6992)) ([01224d0](01224d0)) * Authorize registry server writes before deserializing UDFs ([e0ef16e](e0ef16e)) * Automate versioned documentation publication ([#6763](#6763)) ([5ad5592](5ad5592)), closes [#6762](#6762) * Avoid NumPy generic-unit deprecation when importing feast.type_map ([5fea698](5fea698)) * Bind metrics, REST registry, ui, and lineage servers dual-stack ([#6886](#6886)) ([8d2773c](8d2773c)) * Bound Arrow Flight batch size in the offline server ([#6896](#6896)) ([d169238](d169238)) * Compute std and var correctly for tiled aggregations ([213a130](213a130)) * Convert PdfBytes values to protos like ImageBytes ([#6968](#6968)) ([af20fe4](af20fe4)) * Decode fixed_size_list as the matching Feast list value type ([#6944](#6944)) ([8cfe891](8cfe891)) * Do not log /metrics requests from the pre-fork metrics server thread ([7087f1f](7087f1f)), closes [#6928](#6928) * Don't return from finally blocks in REST registry helpers ([d7cb4d0](d7cb4d0)) * Encode non-finite floats in feature server JSON responses ([#6930](#6930)) ([211ecb8](211ecb8)) * Feature freshness logic updated ([#6765](#6765)) ([f0bc070](f0bc070)) * Fix monitoring auth ([015cc84](015cc84)) * Handle non-nanosecond timestamps in tiled aggregations ([b8989cb](b8989cb)) * Improve feature list column sizing ([#6793](#6793)) ([2573693](2573693)) * Include vector attributes in Field equality ([#6934](#6934)) ([298c3f6](298c3f6)) * Increase feast container probe timeouts to avoid false restarts ([e8dbd50](e8dbd50)) * Infer the tensor type from retrieved values in to_tensor ([a03e323](a03e323)) * Infer timestamps for datetime columns in any unit or timezone ([810391f](810391f)) * Infer vector width when vector_length is undeclared ([0056565](0056565)), closes [#6907](#6907) * Initialize auth managers in standalone lineage server ([#6873](#6873)) ([ecd0d38](ecd0d38)) * Keep a zero watermark delay threshold on a Kafka source ([e2e28c7](e2e28c7)) * Keep entity rows apart in the ibis point-in-time join ([a87c070](a87c070)) * Keep the BatchFeatureView type and settings when copying it ([#6989](#6989)) ([013fc57](013fc57)) * Keep tiling, offline and stream engine settings when copying a StreamFeatureView ([fac0f88](fac0f88)) * Limit tiled max and min aggregations to the window ([2583c45](2583c45)) * Load Milvus collections once instead of on every query ([#6882](#6882)) ([f9e91fa](f9e91fa)) * Make vector length validation reachable, schema-driven and vectorized ([#6909](#6909)) ([29af8c0](29af8c0)), closes [#6907](#6907) * Map INT32_LIST, PDF_BYTES and IMAGE_BYTES type names to their ValueType ([8c1e825](8c1e825)) * Map narrow and unsigned Arrow integer types to Feast value types ([#6960](#6960)) ([ae74360](ae74360)) * Map Spark smallint and tinyint columns to INT32 ([9d42729](9d42729)) * Map SQL Server bigint to INT64 and float to DOUBLE ([6d15533](6d15533)) * Map Trino DECIMAL columns to pyarrow decimal128 ([155bc24](155bc24)) * Normalize Arrow-backed dtypes from Ray Dataset.to_pandas() for pyarrow >=25 compatibility ([#6916](#6916)) ([374f1d7](374f1d7)) * **operator:** Set grpc appProtocol on the offline Service port ([f9dcfb5](f9dcfb5)) * Pin expected tensor to same device in online retrieval test ([#6777](#6777)) ([1f2584d](1f2584d)) * Preserve an unset aggregation time window across a proto round-trip ([2000379](2000379)) * Preserve entity row order in precomputed reads ([ce37029](ce37029)) * Pull the MinIO test image from pgsty/minio ([fe27230](fe27230)) * Pull the MinIO test image from quay ([#6838](#6838)) ([367001b](367001b)) * Raise Trino monitoring query errors other than a missing table ([6161766](6161766)) * Read Kafka and Kinesis sources without a batch source from proto ([a620f5c](a620f5c)) * Read naive datetimes as UTC in UnixTimestamp values ([bc703d9](bc703d9)) * Record which storage class wrote a custom_storage payload ([#6988](#6988)) ([c0545f7](c0545f7)) * Refresh the SQL registry cache after deleting a data source ([66c55e3](66c55e3)) * Refresh the SQL registry cache after deleting a permission ([6b212f5](6b212f5)) * Reject unknown resource types and actions in the REST permission API ([a5eaa36](a5eaa36)) * Remove pip install command from Redis GLIDE docstring ([66ff990](66ff990)) * Remove runtime pip install hints from mlflow errors ([dd9f099](dd9f099)) * Remove unsupported vector_len from PostgreSQL tutorial config ([fcc9a85](fcc9a85)) * Repair Chronon operator validation and Pixi CI setup ([#6947](#6947)) ([3710f78](3710f78)) * Report Dask monitoring read errors instead of no data ([81673d9](81673d9)) * Resolve feature view batch source name against registry ([f8883d0](f8883d0)) * Restrict UI server CORS to trusted origins ([#6986](#6986)) ([5e7ef3e](5e7ef3e)) * Return feature_view_metadata from the REST feature server ([#6927](#6927)) ([bc5aeef](bc5aeef)) * Serve aliased on demand feature views in feature services ([823b92e](823b92e)) * Snapshot tag filters in registry cache keys ([81e1546](81e1546)) * Stabilize materialization checks across hour boundaries ([#6948](#6948)) ([acadb17](acadb17)) * Stop the Couchbase online store reporting failed writes as written ([b3f3c98](b3f3c98)) * Support SQL registry dump ([b60f9c7](b60f9c7)) * Treat datetimes with a None utcoffset as naive ([687bee8](687bee8)) * Unwrap SecretStr tokens when building Trino auth objects ([#6771](#6771)) ([922db5c](922db5c)) * Use a valid ShuffleStrategy for the Ray DatasetContext ([6259f5c](6259f5c)) * Use do_exchange for offline server reads to support HPA ([bae07fc](bae07fc)) * Use group_by in Polars backend aggregation ([#6965](#6965)) ([7b9fad2](7b9fad2)) * Use upstream kubectl image as feast-operator CronJob default ([d6cc1dc](d6cc1dc)) * Use UTC end bound and correct docs for disable_event_timestamp ([#6956](#6956)) ([cc07c82](cc07c82)), closes [#6936](#6936) ### Features * Add a non-JVM read path for Lance data sources ([#6943](#6943)) ([e2a53bb](e2a53bb)), closes [#6925](#6925) [#5782](#5782) [#6909](#6909) * Add a non-JVM write path for Lance data sources ([#6945](#6945)) ([f754323](f754323)), closes [#6943](#6943) [#6909](#6909) * Add Chronon online and offline store integrations ([#6188](#6188)) ([dd059c6](dd059c6)) * Add consistency_level to the Milvus online store config ([97296c1](97296c1)) * Add dualStack server option to the Feast operator ([#6887](#6887)) ([725a147](725a147)), closes [#6977](#6977) * Add Lance optional dependencies ([#6958](#6958)) ([20bc993](20bc993)) * Add LanceFormat table format with version/tag pinning ([1196e22](1196e22)), closes [#5650](#5650) * Add MLflow as a first-class offline DataSource for Feast ([#6702](#6702)) ([3667772](3667772)) * Add optional GLIDE client for Redis online reads ([#6870](#6870)) ([6e9a532](6e9a532)) * Add quality monitoring features for Trino offline store ([#6779](#6779)) ([9340545](9340545)) * Add SavedDatasetLanceStorage ([1118d3e](1118d3e)) * Add standalone MCP server as a first-class component ([0efab5c](0efab5c)), closes [python-keycloak#623](https://github.com/python-keycloak/issues/623) * Add structured audit logging for MCP feature server ([#6456](#6456)) ([640ac70](640ac70)) * Add Trino compute engine for batch retrieval and materialization ([#6898](#6898)) ([11a47b4](11a47b4)) * Batch Postgres online reads across feature views ([#6832](#6832)) ([514bee7](514bee7)), closes [#3259](#3259) * **bigtable:** Make write batch size and concurrency configurable ([#6847](#6847)) ([5388522](5388522)) * Bucket feature count labels in latency metrics ([#6811](#6811)) ([d9ea120](d9ea120)) * Expose runtime version in Feast UI ([#6843](#6843)) ([fa8f06b](fa8f06b)) * Make Milvus index and search params configurable ([ef743ff](ef743ff)) * Offline write batch for IcebergSource ([5be5ed0](5be5ed0)) * OpenLineage consumer improvements - dataset versioning, column lineage, ownership ([69f294e](69f294e)) * **operator:** Add tolerations and nodeSelector to the FeatureStore CR ([#6741](#6741)) ([03dac6f](03dac6f)) * Serve the MCP server dual-stack on --host :: ([4fc9726](4fc9726)) * Support Milvus partition keys in the Milvus online store ([0ed3285](0ed3285)) * Support token, uri and db_name in the Milvus online store ([c5efecd](c5efecd)) * Version-pinning for FeatureService (online + offline) ([#6718](#6718)) ([787845a](787845a)) ### Performance Improvements * Avoid pandas copy in Spark reads ([#6878](#6878)) ([4771156](4771156)) * Decouple metrics HTTP server into a dedicated child process ([#6855](#6855)) ([0b621de](0b621de)) * Reduce online serving hot-path overhead from metrics and audit logging ([5ee7ee3](5ee7ee3))
ntkathole
added a commit
to opendatahub-io/feast
that referenced
this pull request
Oct 14, 2026
* fix: Increase feast container probe timeouts to avoid false restarts Signed-off-by: Srihari <svenkata@redhat.com> * chore: Remove unused hermeto offline tree and sdist requirement locks (feast-dev#6987) * chore: Remove unused multicloud hermeto offline build tree Drop sdk/python/feast/infra/feature_servers/multicloud/offline/, which only supported local hermeto/arrow-from-source and RPM lock offline image builds and is unused by Makefile CI, GitHub Actions, or the standard feature-server Dockerfiles. Also clear related .secrets.baseline entries. Signed-off-by: ntkathole <nikhilkathole2683@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> * chore: Drop unused minimal-sdist requirement locks Remove py3.10–3.12 minimal-sdist(-build) lockfiles, stop generating them in lock-python-dependencies-all, and delete the minimal-sdist-build extra. They only supported hermeto build-from-sdist offline image builds. Signed-off-by: ntkathole <nikhilkathole2683@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> * chore: Refresh pixi.lock after removing minimal-sdist-build Keep pixi install --locked in sync with the pyproject.toml optional dependency change so ray/duckdb CI environments can install. Signed-off-by: ntkathole <nikhilkathole2683@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> --------- Signed-off-by: ntkathole <nikhilkathole2683@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> * chore(release): release 0.67.0 # [0.67.0](feast-dev/feast@v0.66.0...v0.67.0) (2026-10-13) ### Bug Fixes * Allow dill 0.4 so UDF serialization works on Python 3.14 ([feast-dev#6858](feast-dev#6858)) ([139704f](feast-dev@139704f)) * Allow serving features while a feature view is MATERIALIZING ([feast-dev#6789](feast-dev#6789)) ([e2d7fe2](feast-dev@e2d7fe2)), closes [feast-dev#6780](feast-dev#6780) * Allowlist the pinned commit SHA flagged by detect-secrets ([4b81d18](feast-dev@4b81d18)) * Apply point-in-time filter correctly with the Polars backend ([feast-dev#6992](feast-dev#6992)) ([01224d0](feast-dev@01224d0)) * Authorize registry server writes before deserializing UDFs ([e0ef16e](feast-dev@e0ef16e)) * Automate versioned documentation publication ([feast-dev#6763](feast-dev#6763)) ([5ad5592](feast-dev@5ad5592)), closes [feast-dev#6762](feast-dev#6762) * Avoid NumPy generic-unit deprecation when importing feast.type_map ([5fea698](feast-dev@5fea698)) * Bind metrics, REST registry, ui, and lineage servers dual-stack ([feast-dev#6886](feast-dev#6886)) ([8d2773c](feast-dev@8d2773c)) * Bound Arrow Flight batch size in the offline server ([feast-dev#6896](feast-dev#6896)) ([d169238](feast-dev@d169238)) * Compute std and var correctly for tiled aggregations ([213a130](feast-dev@213a130)) * Convert PdfBytes values to protos like ImageBytes ([feast-dev#6968](feast-dev#6968)) ([af20fe4](feast-dev@af20fe4)) * Decode fixed_size_list as the matching Feast list value type ([feast-dev#6944](feast-dev#6944)) ([8cfe891](feast-dev@8cfe891)) * Do not log /metrics requests from the pre-fork metrics server thread ([7087f1f](feast-dev@7087f1f)), closes [feast-dev#6928](feast-dev#6928) * Don't return from finally blocks in REST registry helpers ([d7cb4d0](feast-dev@d7cb4d0)) * Encode non-finite floats in feature server JSON responses ([feast-dev#6930](feast-dev#6930)) ([211ecb8](feast-dev@211ecb8)) * Feature freshness logic updated ([feast-dev#6765](feast-dev#6765)) ([f0bc070](feast-dev@f0bc070)) * Fix monitoring auth ([015cc84](feast-dev@015cc84)) * Handle non-nanosecond timestamps in tiled aggregations ([b8989cb](feast-dev@b8989cb)) * Improve feature list column sizing ([feast-dev#6793](feast-dev#6793)) ([2573693](feast-dev@2573693)) * Include vector attributes in Field equality ([feast-dev#6934](feast-dev#6934)) ([298c3f6](feast-dev@298c3f6)) * Increase feast container probe timeouts to avoid false restarts ([e8dbd50](feast-dev@e8dbd50)) * Infer the tensor type from retrieved values in to_tensor ([a03e323](feast-dev@a03e323)) * Infer timestamps for datetime columns in any unit or timezone ([810391f](feast-dev@810391f)) * Infer vector width when vector_length is undeclared ([0056565](feast-dev@0056565)), closes [feast-dev#6907](feast-dev#6907) * Initialize auth managers in standalone lineage server ([feast-dev#6873](feast-dev#6873)) ([ecd0d38](feast-dev@ecd0d38)) * Keep a zero watermark delay threshold on a Kafka source ([e2e28c7](feast-dev@e2e28c7)) * Keep entity rows apart in the ibis point-in-time join ([a87c070](feast-dev@a87c070)) * Keep the BatchFeatureView type and settings when copying it ([feast-dev#6989](feast-dev#6989)) ([013fc57](feast-dev@013fc57)) * Keep tiling, offline and stream engine settings when copying a StreamFeatureView ([fac0f88](feast-dev@fac0f88)) * Limit tiled max and min aggregations to the window ([2583c45](feast-dev@2583c45)) * Load Milvus collections once instead of on every query ([feast-dev#6882](feast-dev#6882)) ([f9e91fa](feast-dev@f9e91fa)) * Make vector length validation reachable, schema-driven and vectorized ([feast-dev#6909](feast-dev#6909)) ([29af8c0](feast-dev@29af8c0)), closes [feast-dev#6907](feast-dev#6907) * Map INT32_LIST, PDF_BYTES and IMAGE_BYTES type names to their ValueType ([8c1e825](feast-dev@8c1e825)) * Map narrow and unsigned Arrow integer types to Feast value types ([feast-dev#6960](feast-dev#6960)) ([ae74360](feast-dev@ae74360)) * Map Spark smallint and tinyint columns to INT32 ([9d42729](feast-dev@9d42729)) * Map SQL Server bigint to INT64 and float to DOUBLE ([6d15533](feast-dev@6d15533)) * Map Trino DECIMAL columns to pyarrow decimal128 ([155bc24](feast-dev@155bc24)) * Normalize Arrow-backed dtypes from Ray Dataset.to_pandas() for pyarrow >=25 compatibility ([feast-dev#6916](feast-dev#6916)) ([374f1d7](feast-dev@374f1d7)) * **operator:** Set grpc appProtocol on the offline Service port ([f9dcfb5](feast-dev@f9dcfb5)) * Pin expected tensor to same device in online retrieval test ([feast-dev#6777](feast-dev#6777)) ([1f2584d](feast-dev@1f2584d)) * Preserve an unset aggregation time window across a proto round-trip ([2000379](feast-dev@2000379)) * Preserve entity row order in precomputed reads ([ce37029](feast-dev@ce37029)) * Pull the MinIO test image from pgsty/minio ([fe27230](feast-dev@fe27230)) * Pull the MinIO test image from quay ([feast-dev#6838](feast-dev#6838)) ([367001b](feast-dev@367001b)) * Raise Trino monitoring query errors other than a missing table ([6161766](feast-dev@6161766)) * Read Kafka and Kinesis sources without a batch source from proto ([a620f5c](feast-dev@a620f5c)) * Read naive datetimes as UTC in UnixTimestamp values ([bc703d9](feast-dev@bc703d9)) * Record which storage class wrote a custom_storage payload ([feast-dev#6988](feast-dev#6988)) ([c0545f7](feast-dev@c0545f7)) * Refresh the SQL registry cache after deleting a data source ([66c55e3](feast-dev@66c55e3)) * Refresh the SQL registry cache after deleting a permission ([6b212f5](feast-dev@6b212f5)) * Reject unknown resource types and actions in the REST permission API ([a5eaa36](feast-dev@a5eaa36)) * Remove pip install command from Redis GLIDE docstring ([66ff990](feast-dev@66ff990)) * Remove runtime pip install hints from mlflow errors ([dd9f099](feast-dev@dd9f099)) * Remove unsupported vector_len from PostgreSQL tutorial config ([fcc9a85](feast-dev@fcc9a85)) * Repair Chronon operator validation and Pixi CI setup ([feast-dev#6947](feast-dev#6947)) ([3710f78](feast-dev@3710f78)) * Report Dask monitoring read errors instead of no data ([81673d9](feast-dev@81673d9)) * Resolve feature view batch source name against registry ([f8883d0](feast-dev@f8883d0)) * Restrict UI server CORS to trusted origins ([feast-dev#6986](feast-dev#6986)) ([5e7ef3e](feast-dev@5e7ef3e)) * Return feature_view_metadata from the REST feature server ([feast-dev#6927](feast-dev#6927)) ([bc5aeef](feast-dev@bc5aeef)) * Serve aliased on demand feature views in feature services ([823b92e](feast-dev@823b92e)) * Snapshot tag filters in registry cache keys ([81e1546](feast-dev@81e1546)) * Stabilize materialization checks across hour boundaries ([feast-dev#6948](feast-dev#6948)) ([acadb17](feast-dev@acadb17)) * Stop the Couchbase online store reporting failed writes as written ([b3f3c98](feast-dev@b3f3c98)) * Support SQL registry dump ([b60f9c7](feast-dev@b60f9c7)) * Treat datetimes with a None utcoffset as naive ([687bee8](feast-dev@687bee8)) * Unwrap SecretStr tokens when building Trino auth objects ([feast-dev#6771](feast-dev#6771)) ([922db5c](feast-dev@922db5c)) * Use a valid ShuffleStrategy for the Ray DatasetContext ([6259f5c](feast-dev@6259f5c)) * Use do_exchange for offline server reads to support HPA ([bae07fc](feast-dev@bae07fc)) * Use group_by in Polars backend aggregation ([feast-dev#6965](feast-dev#6965)) ([7b9fad2](feast-dev@7b9fad2)) * Use upstream kubectl image as feast-operator CronJob default ([d6cc1dc](feast-dev@d6cc1dc)) * Use UTC end bound and correct docs for disable_event_timestamp ([feast-dev#6956](feast-dev#6956)) ([cc07c82](feast-dev@cc07c82)), closes [feast-dev#6936](feast-dev#6936) ### Features * Add a non-JVM read path for Lance data sources ([feast-dev#6943](feast-dev#6943)) ([e2a53bb](feast-dev@e2a53bb)), closes [feast-dev#6925](feast-dev#6925) [feast-dev#5782](feast-dev#5782) [feast-dev#6909](feast-dev#6909) * Add a non-JVM write path for Lance data sources ([feast-dev#6945](feast-dev#6945)) ([f754323](feast-dev@f754323)), closes [feast-dev#6943](feast-dev#6943) [feast-dev#6909](feast-dev#6909) * Add Chronon online and offline store integrations ([feast-dev#6188](feast-dev#6188)) ([dd059c6](feast-dev@dd059c6)) * Add consistency_level to the Milvus online store config ([97296c1](feast-dev@97296c1)) * Add dualStack server option to the Feast operator ([feast-dev#6887](feast-dev#6887)) ([725a147](feast-dev@725a147)), closes [feast-dev#6977](feast-dev#6977) * Add Lance optional dependencies ([feast-dev#6958](feast-dev#6958)) ([20bc993](feast-dev@20bc993)) * Add LanceFormat table format with version/tag pinning ([1196e22](feast-dev@1196e22)), closes [feast-dev#5650](feast-dev#5650) * Add MLflow as a first-class offline DataSource for Feast ([feast-dev#6702](feast-dev#6702)) ([3667772](feast-dev@3667772)) * Add optional GLIDE client for Redis online reads ([feast-dev#6870](feast-dev#6870)) ([6e9a532](feast-dev@6e9a532)) * Add quality monitoring features for Trino offline store ([feast-dev#6779](feast-dev#6779)) ([9340545](feast-dev@9340545)) * Add SavedDatasetLanceStorage ([1118d3e](feast-dev@1118d3e)) * Add standalone MCP server as a first-class component ([0efab5c](feast-dev@0efab5c)), closes [python-keycloak#623](https://github.com/python-keycloak/issues/623) * Add structured audit logging for MCP feature server ([feast-dev#6456](feast-dev#6456)) ([640ac70](feast-dev@640ac70)) * Add Trino compute engine for batch retrieval and materialization ([feast-dev#6898](feast-dev#6898)) ([11a47b4](feast-dev@11a47b4)) * Batch Postgres online reads across feature views ([feast-dev#6832](feast-dev#6832)) ([514bee7](feast-dev@514bee7)), closes [feast-dev#3259](feast-dev#3259) * **bigtable:** Make write batch size and concurrency configurable ([feast-dev#6847](feast-dev#6847)) ([5388522](feast-dev@5388522)) * Bucket feature count labels in latency metrics ([feast-dev#6811](feast-dev#6811)) ([d9ea120](feast-dev@d9ea120)) * Expose runtime version in Feast UI ([feast-dev#6843](feast-dev#6843)) ([fa8f06b](feast-dev@fa8f06b)) * Make Milvus index and search params configurable ([ef743ff](feast-dev@ef743ff)) * Offline write batch for IcebergSource ([5be5ed0](feast-dev@5be5ed0)) * OpenLineage consumer improvements - dataset versioning, column lineage, ownership ([69f294e](feast-dev@69f294e)) * **operator:** Add tolerations and nodeSelector to the FeatureStore CR ([feast-dev#6741](feast-dev#6741)) ([03dac6f](feast-dev@03dac6f)) * Serve the MCP server dual-stack on --host :: ([4fc9726](feast-dev@4fc9726)) * Support Milvus partition keys in the Milvus online store ([0ed3285](feast-dev@0ed3285)) * Support token, uri and db_name in the Milvus online store ([c5efecd](feast-dev@c5efecd)) * Version-pinning for FeatureService (online + offline) ([feast-dev#6718](feast-dev#6718)) ([787845a](feast-dev@787845a)) ### Performance Improvements * Avoid pandas copy in Spark reads ([feast-dev#6878](feast-dev#6878)) ([4771156](feast-dev@4771156)) * Decouple metrics HTTP server into a dedicated child process ([feast-dev#6855](feast-dev#6855)) ([0b621de](feast-dev@0b621de)) * Reduce online serving hot-path overhead from metrics and audit logging ([5ee7ee3](feast-dev@5ee7ee3)) --------- Signed-off-by: Srihari <svenkata@redhat.com> Signed-off-by: ntkathole <nikhilkathole2683@gmail.com> Co-authored-by: Srihari <svenkata@redhat.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: feast-ci-bot <feast-ci-bot@willem.co>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
The offline server's
do_get()anddo_exchange()Flight handlers each returned a single, unbounded Arrow Table/RecordBatch to the client. Once a result approaches Arrow IPC's per-batch limit, sending it raises:and the whole request fails instead of degrading gracefully.
This adds two module-level helpers in
offline_server.py:_split_oversized(batch)recursively halves anyRecordBatchwhose IPC-encoded size exceeds_MAX_BATCH_BYTES(1.5GB, comfortably under the 2GiB cap). It stops recursing at a single row and yields it as-is, so Flight still raises its own clear error for a genuinely oversized row instead of looping forever._bounded_reader(data)accepts either apa.Tableor apa.RecordBatchReaderand re-batches it through_split_oversized.Both read paths are routed through
_bounded_reader:do_getreturnsfl.RecordBatchStream(_bounded_reader(table))instead of streaming the raw table directly.do_exchange(the newer HPA-safe single-stream read path added inbae07fc4c) now writes batches in a loop over the bounded reader instead of onewriter.write_table(table)call, it was still writing one unbounded table and needed the same fix asdo_get.Which issue(s) this PR fixes:
Part of #6863
Checks
git commit -s)Testing Strategy
New/updated unit tests in
sdk/python/tests/unit/test_offline_server.pycover_split_oversized/_bounded_readerdirectly (oversized splitting, exact-cap boundary, single-oversized-row pass-through, empty batch) and bothdo_getanddo_exchangerouting through the bounded reader. The two existingdo_exchangeunit tests were updated from assertingwrite_tableto asserting the reassembledwrite_batchcalls.Quality bar for this PR:
upstream/master: 100% (17/17 changed lines).offline_server.py, filtered to lines changed vsupstream/master): 35 mutants generated on the diff, 35 killed, 0 survivors, 0 timeouts, all worker outcomes NORMAL/SKIPPED. One initial survivor (//->/in the halving arithmetic) was closed with an added test, since a realpyarrow.RecordBatch.slice()silently truncates a float length/offset to the same int and can't otherwise distinguish the two operators.sdk/python/tests/integration/offline_server/test_offline_server.py -v --integrationpasses locally (4/4) against the real Flight client/server round trip.