Python SDK #48

Activity

1. woop

   assigned

   zhilingc

   ,

   pradithya

   and

   tims

   and unassigned

   zhilingc

   on Jan 8, 2019
2. zhilingc commented on Jan 8, 2019

   

   zhilingc

   on Jan 8, 2019

   Last edited by zhilingc

   Collaborator

   More actions

   Outstanding tasks:

   • Importer from BQ should pull descriptions from bq table
   • CI should run py sdk unit tests

   Enhancements:

   • More intuitive way of updating stores for features
   • Feature exploration api
3. tims commented on Jan 11, 2019

   

   tims

   on Jan 11, 2019

   Contributor

   More actions

   Further feedback:

   The main thing that I think we'll need to think more about, is how the notion of a dataset will work in an environment without BigQuery. One way to do this, would be rather saving query results back to BQ tables, we could read them from the warehouse is in beam and export them again using beam. Then the warehouse wouldn't have to be BQ. Eg, exporting them as Avro files would be pretty neat.
4. woop commented on Jan 12, 2019

   

   woop

   on Jan 12, 2019

   MemberAuthor

   More actions

   Further feedback:

   The main thing that I think we'll need to think more about, is how the notion of a dataset will work in an environment without BigQuery. One way to do this, would be rather saving query results back to BQ tables, we could read them from the warehouse is in beam and export them again using beam. Then the warehouse wouldn't have to be BQ. Eg, exporting them as Avro files would be pretty neat.

   Is there anything that couples us to BQ right now? The way I see it is we will have a feature warehouse where we will create these datasets (that will remain the source of truth). When the user wants to retrieve it, the dataset table is exported in file format to some staging location, and then the user can download it through through our client. That process can happen in any way, Beam or otherwise.

   We should definitely support structured data as the default, with my preference being Parquet. Avro is also a strong contender.
5. tims commented on Jan 18, 2019

   

   tims

   on Jan 18, 2019

   Contributor

   More actions

   There is, in that there isn't a way to create training datasets from the warehouse if the warehouse is just files (currently our only other supported warehouse sink). Which suggests flat files aren't suitable warehouse sinks.

   I think we can close this now that the SDK is merged in. Alternative open source warehouse stores are a future issue.
6. tims

   closed this as completedon Jan 18, 2019
7. Yanson

   added a commit that references this issue on Jul 29, 2020

   Merge pull request feast-dev#48 from farfetch-external/KE-799_fix_dat…

   ...

   2b7e047
8. lokeshrangineni

   added a commit that references this issue on Jul 19, 2024

   Merge pull request feast-dev#48 from tmihalac/fix-missing-auth-calls

   ...

   b416c15
9. aniketpalu

   added a commit that references this issue on Nov 24, 2025

   Merge pull request feast-dev#48 from odh-on-pz/stable-2.x-revert-pr46

   ...

   22517f4