Repository navigation
Conversation
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
…ame on inference failure (feast-dev#7040) Signed-off-by: Mrun25 <165297479+Mrun25@users.noreply.github.com>
b2cc986 to
2f38847
Compare
luc-pimentel
left a comment
There was a problem hiding this comment.
Thanks for picking up the issue I opened. I ran its reproduction against this PR's current head: the Arrow type strings as FileSource reads them from a pandas-written Parquet file, a schema-less feast apply on a file holding one such column, and the CLI repro from the issue.
| pandas column | Arrow type read | master | this PR |
|---|---|---|---|
datetime.date values |
date32[day] |
KeyError |
applied as UnixTimestamp |
category dtype |
dictionary<values=string, indices=int8, ordered=0> |
KeyError |
applied as String |
Decimal values |
decimal128(3, 2) |
KeyError |
applied as Decimal |
| int64, float64, str, bytes, list of int (controls) | int64, double, string, binary, list<element: int64> |
applied | applied, same types |
The issue's feast apply repro exits 0 here and registers trips: Int64 and signup_date: UnixTimestamp; on master it exits 1 with KeyError: 'date32[day]'. decimal256, large_binary and halffloat columns map as well. time32, time64, duration and a polars-written large_list column now fail with the new error naming the column and the Arrow type instead of the bare KeyError, which is what the issue asked for as the fallback. The DataFrame inference path gives the same types as before.
Tested with Python 3.11, pandas 2.3.3, pyarrow 25.0.1 and polars 2.1.0 on 2026-10-26.
Looks good to me. Curious what the maintainers think.
Fixes #7040.
A Parquet file written by pandas fails as a batch source: pa_to_feast_value_type raises a bare KeyError for Arrow types pandas commonly writes (date32/date64, decimal128/256, dictionary-encoded categories, halffloat, large_binary), even though the DataFrame inference path maps the equivalent pandas dtypes fine.
Changes:
Tests: parametrized mapping tests and a descriptive-error test in tests/unit/test_type_map.py. ruff format and check clean.