Repository navigation
Ingestion should fail immediately when there are no valid stores #12
Description
Expected Behavior
Starting a job with invalid stores (e.g. using redis as a warehouse store) should not be allowed, and should fail quickly - ideally at the graph building step of ingestion.
Current Behavior
Starting a job with invalid stores will successfully send the job to the runner, which will run to completion (or indefinitely, in the case of streaming jobs). The errors will be logged, but the pipeline will run with no problems.
Steps to reproduce
- Register a feature with its warehouse sink pointing to a serving store (e.g. redis)
- Run a job (direct runner is the best way to view errors)
- Pipeline runs successfully, a successful response is returned to the caller
Possible Solution
PR #11 is a band-aid solution to this problem: it checks the store types at registration, ensuring that a feature is unable to use a serving store for warehousing, but ideally ingestion error out properly during graph building.
Metadata
Metadata
Assignees
Labels
No labels
Activity
tims commentedon Dec 24, 2018
@zhilingc I think you misunderstood the code it's not a "bandaid" :)
It does fail at graph build time if the feature references a store that is not found in the specs service.
See Specs.validate(). Am I missing something?
tims commentedon Dec 24, 2018
Sorry I misread this issue.
The fix is in PR #15.
zhilingc commentedon Dec 24, 2018
It's a band-aid because if someone alters the DB the job will fail since the checks are only at feature registration time.
And nice! If it works on remote flink we're golden :)
Merge #12: prefetch specs and validate on job expansion
Merge #12: prefetch specs and validate on job expansion
tims commentedon Dec 31, 2018
This should stop those self imposed DoS events every 10 minutes too.
tims commentedon Dec 31, 2018
We can close this now?
zhilingc commentedon Jan 3, 2019
Yep. Thanks for the hard work :)
Merge pull request feast-dev#12 from farfetch-external/algattik/merge…
fix: Java CI #12
chore(release): release 0.49.0
fix: Java CI feast-dev#12
chore(release): release 0.49.0