You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat: Add feature_service label to Feature Server RED metrics #6639
Follow-up from #5920 (Improve Feature Server Observability).
Problem
sdk/python/feast/metrics.py currently breaks out online-feature metrics by feature_view (see online_features_status_total's feature_view label in track_feature_statuses), but there is no equivalent feature_service label anywhere in the metrics, even though requests to /get-online-features are frequently made via a named Feature Service.
Proposed solution
Add a feature_service label to the relevant RED metrics (at minimum request_latency/request_count in the track_request_latency context, and consider online_features_status_total).
Resolve the feature service name in the /get-online-features handler in sdk/python/feast/feature_server.py (available from the parsed request body) and pass it through via RequestMetricsContext, following the existing pattern used for feature_count/feature_view_count.
Add cardinality safeguards per the original issue: make this label opt-in (config flag, default off) and/or support an allowlist of feature service names, since unbounded feature-service cardinality could blow up metric storage in large deployments.
Wire the new config flag into MetricsConfig / _MetricsFlags alongside the existing per-category toggles.
@jyejare@ntkathole I'd like to pick this up (following on from #6709). I read through metrics.py / feature_server.py and the existing feature_count/feature_view_count plumbing, and the pieces line up nicely — GetOnlineFeaturesRequest already carries feature_service, and the /get-online-features handler already owns a mutable RequestMetricsContext. Before I write it, I'd like to confirm the design so I build the version you want:
Proposed implementation (mirrors the existing pattern):
Add feature_service to RequestMetricsContext.__slots__ and to the track_request_latency(...) signature (default ""), exactly like feature_count/feature_view_count.
Add a feature_service label to request_latency (and request_count) — gated so it's only populated when the opt-in flag is on; "" otherwise.
In the /get-online-features handler, set metrics_ctx.feature_service = request.feature_service or "" (the name is already on the parsed request body — no extra lookup needed).
Add an opt-in flag to MetricsConfig + _MetricsFlags (build_metrics_flags), default off, since a per-service label on the request_latencyHistogram multiplies series and unbounded service cardinality could hurt large deployments.
Design questions I'd like your call on before coding:
Flag name / shape: a dedicated request_feature_service boolean under MetricsConfig, or a sub-toggle nested under the existing request category? Happy to match whatever convention you prefer.
Which metrics get the label:request_latency for sure. Also add it to request_count? And do you want it on online_features_status_total (the track_feature_statuses path) in this same PR, or keep that out of scope for now?
Allowlist: the issue mentions an optional feature-service allowlist for cardinality control. I'd suggest shipping the opt-in boolean first (smaller, reviewable PR) and doing the allowlist as a follow-up — unless you'd rather have it in v1.
Once you confirm the flag shape + metric scope, I'll open a PR with tests. Thanks!
Activity
saket3395 commentedon Aug 14, 2026
@jyejare @ntkathole I'd like to pick this up (following on from #6709). I read through
metrics.py/feature_server.pyand the existingfeature_count/feature_view_countplumbing, and the pieces line up nicely —GetOnlineFeaturesRequestalready carriesfeature_service, and the/get-online-featureshandler already owns a mutableRequestMetricsContext. Before I write it, I'd like to confirm the design so I build the version you want:Proposed implementation (mirrors the existing pattern):
feature_servicetoRequestMetricsContext.__slots__and to thetrack_request_latency(...)signature (default""), exactly likefeature_count/feature_view_count.feature_servicelabel torequest_latency(andrequest_count) — gated so it's only populated when the opt-in flag is on;""otherwise./get-online-featureshandler, setmetrics_ctx.feature_service = request.feature_service or ""(the name is already on the parsed request body — no extra lookup needed).MetricsConfig+_MetricsFlags(build_metrics_flags), default off, since a per-service label on therequest_latencyHistogram multiplies series and unbounded service cardinality could hurt large deployments.Design questions I'd like your call on before coding:
request_feature_serviceboolean underMetricsConfig, or a sub-toggle nested under the existingrequestcategory? Happy to match whatever convention you prefer.request_latencyfor sure. Also add it torequest_count? And do you want it ononline_features_status_total(thetrack_feature_statusespath) in this same PR, or keep that out of scope for now?Once you confirm the flag shape + metric scope, I'll open a PR with tests. Thanks!