feat: Add version tracking to FeatureView#6101
feat: Add version tracking to FeatureView#6101franciscojavierarceo wants to merge 32 commits intomasterfrom
Conversation
…emandFeatureView Every `feast apply` now creates a version snapshot. Users can pin a feature view to a specific historical version declaratively via `version="v2"`. By default, the latest version is always served. - New proto: FeatureViewVersion.proto with version record/history - Added `version` field to FeatureViewSpec, StreamFeatureViewSpec, OnDemandFeatureViewSpec and version metadata to their Meta messages - New version_utils module for parsing/normalizing version strings - Version-aware apply_feature_view in both SQL and file registries - New `list_feature_view_versions` API on FeatureStore and registries - CLI: `feast feature-views versions <name>` subcommand - Updated all 14 templates with explicit `version="latest"` - Unit tests (28) and integration tests (7) for versioning Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix current_version_number=0 being silently dropped during proto deserialization in FeatureView, OnDemandFeatureView (proto3 int32 default 0 is falsy in Python); use spec.version to disambiguate - Add current_version_number restoration in StreamFeatureView.from_proto (was missing entirely) - Use timezone-aware UTC datetime in SqlRegistry.list_feature_view_versions for consistency with the rest of the codebase - Add test for v0 proto roundtrip Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add Versioning section to feature-view.md concept page covering automatic snapshots, version pinning, version string formats, CLI usage, and Python SDK API - Add `feast feature-views versions` command to CLI reference Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix current_version_number roundtrip bug: version="latest" (always truthy) caused None to become 0 after proto roundtrip; now check that spec.version is not "latest" before treating 0 as intentional - Use write_engine (not read_engine) for pre/post apply reads in SqlRegistry to avoid read replica lag causing missed version snapshots - Remove redundant version check in StreamFeatureView.__eq__ (parent FeatureView.__eq__ already checks it) - Add else clause to StreamFeatureView.from_proto for consistency - Add test for latest/None roundtrip preservation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ntly - delete_feature_view now also deletes version history records, preventing IntegrityError when re-creating a previously deleted FV - _get_next_version_number uses write_engine instead of read_engine to avoid stale version numbers with read replicas Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add step-by-step walkthrough showing how versions auto-increment on changes and skip on identical re-applies - Add CLI example showing the apply/change/apply cycle - Clarify that pinning ignores constructor params and uses the snapshot - Explain how to return to auto-incrementing after a pin/revert Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Devin Review found 1 new potential issue.
🐛 1 issue in files not directly in the diff
🐛 BatchFeatureView missing version parameter — incomplete transformation (sdk/python/feast/batch_feature_view.py:80-102)
FeatureView, StreamFeatureView, and OnDemandFeatureView all accept a version parameter in their constructors, but BatchFeatureView.__init__ (sdk/python/feast/batch_feature_view.py:80-102) does not. Furthermore, BatchFeatureView.__init__ does not pass version to super().__init__() at sdk/python/feast/batch_feature_view.py:144-158. This means users cannot construct a BatchFeatureView with a pinned version (e.g., BatchFeatureView(..., version="v2") raises TypeError), and any BatchFeatureView created directly always defaults to version="latest". The proto round-trip path works because FeatureView.from_proto sets feature_view.version after construction, but the constructor API is inconsistent with the other feature view types.
View 19 additional findings in Devin Review.
Raises FeatureViewPinConflict when a user pins to an older version while also modifying the feature view definition (schema, source, etc.). Fixes FeatureView.__copy__() to include description and owner fields, which was causing false positive conflict detection. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Devin Review found 1 new potential issue.
🐛 1 issue in files not directly in the diff
🐛 BatchFeatureView missing version parameter — incomplete transformation (sdk/python/feast/batch_feature_view.py:80-102)
FeatureView, StreamFeatureView, and OnDemandFeatureView all accept a version parameter in their constructors, but BatchFeatureView.__init__ (sdk/python/feast/batch_feature_view.py:80-102) does not. Furthermore, BatchFeatureView.__init__ does not pass version to super().__init__() at sdk/python/feast/batch_feature_view.py:144-158. This means users cannot construct a BatchFeatureView with a pinned version (e.g., BatchFeatureView(..., version="v2") raises TypeError), and any BatchFeatureView created directly always defaults to version="latest". The proto round-trip path works because FeatureView.from_proto sets feature_view.version after construction, but the constructor API is inconsistent with the other feature view types.
View 23 additional findings in Devin Review.
- Add version parameter to BatchFeatureView constructor for consistency with FeatureView, StreamFeatureView, and OnDemandFeatureView - Clean up version history records in file registry delete_feature_view to prevent orphaned records on re-creation - Fix current_version_number proto roundtrip: preserve 0 when version="latest" (after first apply) instead of incorrectly returning None Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Clarify that versioning provides definition management and rollback, not concurrent multi-version serving. Document recommended approaches (separate projects or distinct FV names) for A/B testing scenarios. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extends feature view versioning with support for reading features from specific
versions at query time using the syntax: "driver_stats@v2:trips_today"
Core changes:
- Add _parse_feature_ref() to parse version-qualified feature references
- Update all feature reference parsing to use _parse_feature_ref()
- Add get_feature_view_by_version() to BaseRegistry and all implementations
- Add FeatureViewProjection.version_tag for multi-version query support
- Add version-aware _table_id() in SQLite online store (v0→unversioned, v1+→_v{N})
- Add VersionedOnlineReadNotSupported error for unsupported stores
Features:
- "driver_stats:trips" = "driver_stats@latest:trips" (backward compatible)
- "driver_stats@v2:trips" reads from v2 snapshot using _v2 table suffix
- Multiple versions in same query: ["driver@v1:trips", "driver@v2:daily"]
- Version parameter added to all decorator functions for consistency
Backward compatibility:
- Unversioned table serves as v0, only v1+ get _v{N} suffix
- All existing queries work unchanged
- SQLite-only for now, other stores raise clear error
Documentation:
- Updated feature-view.md with @Version syntax examples
- Updated feature-retrieval.md reference format
- Added version examples to how-to guides
Tests: 47 unit + 11 integration tests pass, no regressions
Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>
- Fix type inference issues in get_feature_view_by_version() - Use distinct variable names for different proto types - Ensure proper type annotations for BaseFeatureView subclasses
|
I haven't been able to go through the whole thing yet, but iiuc the expectation is that online stores would be expected to handle this by essentially treating fv1@v1 and fv1@v2 as two different feature views, right? (sqlite example infers table name from the version). Just wondering, maybe that's to high a bat for online store implementations. for example, if you have fv1@v1 and fv1@v2 both containing 100 features that differ by a single feature only, the online store would have to keep redundant copies of the other 99 features or do some complicated logic to diff values and deduplicate. wdyt about concentrating on introducing versioning on feature level instead? tbh, that makes more intuitive sense to me. there's a feature called |
I spent some time reasoning about feature-level versioning with Claude. My initial reaction was that it's too large of a change and it only works today in a broken sense. By "in a broken sense" I mean that today we don't really version feature views or features, if someone changes the feature view or the feature, we just overwrite it and lose history. Moreover, we essentially force the materialization to be out of sync. Today, that just ignores the behavior silently. I like how we've implemented it here (i.e., creating a new table and storing the history of the metadata but allowing for callers to specify exact versions and defaulting to the existing behavior) because it is far more explicit about the challenges of materialization consistency issues when you change feature versions. So, I don't recommend we do feature-level versioning as I worry it makes materialization very unreliable. We can, in fact, declare feature view level versioning because transformations are a collection of features mapped one-to-one with a table. |
Implement optional feature view version metadata in API responses to address the issue where internal @v2 version syntax was leaking into client responses. Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>
Add missing include_feature_view_version_metadata parameter to: - EmbeddedGoOnlineFeaturesService.get_online_features() - FooProvider.get_online_features() and get_online_features_async() - FooProvider.retrieve_online_documents() and retrieve_online_documents_v2() This resolves CI failures where provider implementations were not updated with the new parameter from the abstract Provider interface. Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>
- Fix version-qualified features dropped with full_feature_names=True: use _parse_feature_ref to build clean requested_result_row_names - Fix retrieve_online_documents breaking with @vn refs: use _parse_feature_ref instead of split(":") for FV name extraction - Fix metadata-only updates not committed: add self.commit() after _update_metadata_fields in file registry - Fix ODFV transforms broken by version-qualified refs: use _parse_feature_ref in _augment_response_with_on_demand_transforms - Fix _update_metadata_fields not updating spec.version: add version field update so pinned-to-latest transitions persist - Fix _resolve_feature_counts inflating FV count: strip @vn from feature view names in metrics - Fix version snapshots storing stale current_version_number: set version number before serializing snapshot in both file and SQL registries Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Set spec.project on snapshot protos in SqlRegistry before serializing, so version snapshots include the correct project field - Fix _check_versioned_read_support to check projection.version_tag instead of current_version_number, so explicitly version-qualified reads (@v0) are correctly rejected on non-SQLite stores Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Versioning was always active on every `feast apply`. This adds an `enable_feature_view_versioning` boolean (default False) to RegistryConfig so version history, version pins, and version-qualified refs are only available when explicitly enabled. Existing behaviour is fully preserved when the flag is set to true. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix SQLite _table_id to prefer projection.version_tag over current_version_number so @v2 refs read from the correct table - Detect feature name collisions for multi-version queries with full_feature_names=True (e.g. fv@v1:feat vs fv@v2:feat) - Remove unused include_feature_view_version_metadata parameter from retrieve_online_documents (v1) across all providers and online stores - Remove redundant name check from _schema_or_udf_changed since callers always match by name first Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…queries When version-qualified refs (e.g. fv@v1:feat, fv@v2:feat) are used, include the version tag in full_feature_names output so multi-version queries produce distinct columns (fv@v1__feat vs fv@v2__feat). Also fix proto roundtrip test to match -1 sentinel behavior. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…d declaration Auto-increment path (version="latest") now retries up to 3 times on IntegrityError in the SQL registry when concurrent applies race on the same version number. Explicit version path (version="v<N>") now creates the version if it doesn't exist (forward declaration) instead of raising FeatureViewVersionNotFound, with ConcurrentVersionConflict on race. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fail fast at apply time and retrieval time when a feature service references a versioned FV (current_version_number > 0) and enable_online_feature_view_versioning is off. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rename enable_feature_view_versioning -> enable_online_feature_view_versioning to clarify that it controls online reads, not version history tracking. Fix mypy type narrowing issues with current_version_number (int | None) and variable redefinition in feature_store.py. Remove -1 sentinel from proto serialization in favor of treating proto default 0 without spec.version as None. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When enable_online_feature_view_versioning is on and a FeatureService references a versioned FV, set version_tag on the projection so that online reads resolve to the correct versioned table. Previously the FeatureService path never set version_tag, causing reads from the wrong (unversioned) online store table. Changes: - _get_features(): version-qualify feature refs for FeatureService projections - _get_feature_views_to_use(): capture current_version_number before with_projection() discards it, then set version_tag on the projection - feature_store.py: fix mypy type narrowing for the gate check - Add integration tests for both code paths Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
| # Re-serialize with updated version number | ||
| with self.write_engine.begin() as conn: | ||
| update_stmt = ( | ||
| update(fv_table) | ||
| .where( | ||
| fv_table.c.feature_view_name == feature_view.name, | ||
| fv_table.c.project_id == project, | ||
| ) | ||
| .values( | ||
| feature_view_proto=snapshot_proto_bytes, | ||
| ) | ||
| ) | ||
| conn.execute(update_stmt) |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
sdk/python/feast/infra/online_stores/elasticsearch_online_store/elasticsearch.py
Show resolved
Hide resolved
Response to Devin Review FindingsThanks for the thorough automated review! Here's a summary of where all the findings stand after the latest commits ( Already Resolved (confirmed by Devin itself in follow-up passes)The following were real issues that have been fixed in prior commits:
Not Bugs / By DesignThe remaining open findings are either false positives or by-design behavior:
|
Update the Feature Services section in the RFC to reflect that feature services now correctly serve versioned FVs when the online versioning flag is enabled (automatic version_tag resolution on projections). Rename CLI command from `feast feature-views versions` to `feast feature-views list-versions` for consistency, and update all docs references. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Show current version badge in feature view headers and listing table, and add a Versions tab with expandable version history across all feature view types (Regular, Stream, OnDemand). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Add --version flag to feast materialize/materialize-incremental CLI commands and corresponding Python SDK support. Gate versioned table IDs behind enable_online_feature_view_versioning config flag in SQLite online store. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1. Stop injecting version_tag on FeatureService projections in utils.py, which was causing non-SQLite stores to reject FeatureService reads when versioning was enabled. 2. Persist version_tag in FeatureViewProjection proto (field 10) so it survives registry round-trips. 3. Fix _update_metadata_fields() to reset current_version_number to 0 when unpinning a feature view back to version="latest". Update tests to match new behavior and add test_unpin_from_versioned_to_latest. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add --no-promote flag that saves new version snapshots without promoting them to active, enabling phased rollouts without a transition window where unversioned consumers briefly see the new schema. Also audit and fix brittle feature reference parsing across the codebase to properly handle @v<N> version-qualified syntax: - Fix ibis offline store prefix matching to use name_to_use() - Fix Go ParseFeatureReference to strip @v<N> from view name - Fix passthrough_provider saved dataset column naming Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Summary
Adds automatic version tracking to Feast feature views (
FeatureView,StreamFeatureView,OnDemandFeatureView). Everyfeast applythat changes a feature view's schema or UDF creates a versioned snapshot in the registry. Users can list version history, pin serving to a prior version, query specific versions via@v<N>syntax, and stage new versions without promoting them.Resolves #2728
Key Features
FeatureView(name="driver_stats", version="v2")driver_stats@v2:trips_today(SQLite online store)--no-promote) —feast apply --no-promotesaves a new version without promoting it to active, enabling phased rollouts without a transition windowfeast materialize --views fv --version v2targets a specific version's online store tableenable_online_feature_view_versioningis enabledversionis optional; omitting it is identical toversion="latest"Changes
Proto layer:
FeatureViewVersion.protowithFeatureViewVersionRecord/FeatureViewVersionHistoryversionfield toFeatureViewSpec,StreamFeatureViewSpec,OnDemandFeatureViewSpeccurrent_version_numberto meta messages,version_tagtoFeatureViewProjectionfeature_view_version_historytoRegistryprotoPython SDK — core versioning:
version_utils.py— parses"latest","v2","version2"(case-insensitive)versionparameter onFeatureView,StreamFeatureView,OnDemandFeatureViewapply_feature_viewin file registry, SQL registry (with concurrent apply retry)list_feature_view_versions(name, project)on registries andFeatureStoreget_feature_view_by_version(name, project, version_number)for explicit version lookup_schema_or_udf_changed()to distinguish version-significant changes from metadata updatesPython SDK —
--no-promoteflag:feast apply --no-promoteCLI option threaded throughapply_total()→apply_total_with_repo_instance()→store.apply()/store._apply_diffs()→registry.apply_feature_view()store.apply([...], no_promote=True)Python SDK — version-qualified reads:
_parse_feature_ref()and_strip_version_from_ref()utilities forfv@v<N>:featureparsing@v<N>syntaxversion_tagon projectionsPython SDK — feature ref parsing audit & fixes:
name_to_use()for versioned refspassthrough_provider.pysaved dataset column naming to strip@v<N>ParseFeatureReferenceto strip@v<N>from feature view nameCLI:
feast feature-views versions <name>— list version historyfeast apply --no-promote— stage versions without promotingfeast materialize --views <name> --version v<N>— version-aware materializationUI:
Documentation:
docs/rfcs/feature-view-versioning.md— full design, concurrency, feature services, staged publishingdocs/reference/alpha-feature-view-versioning.md[Alpha] VersioningsectionTest plan
--no-promote(5 tests)UI Preview
🤖 Generated with Claude Code