fixes

Author: Alexandru Ormenisan

docs/user_guides/fs/provenance/provenance.md

@@ -2,7 +2,7 @@
 
 ## Introduction 
 
-Hopsworks feature store allows users to track provenance (lineage) between storage connectors, feature groups, feature views, training datasets and models. Tracking lineage allows users to determine where/if a feature group is being used. You can track if feature groups are being used to create additional (derived) feature groups or feature views.
+Hopsworks feature store allows users to track provenance (lineage) between storage connectors, feature groups, feature views, training datasets and models. Tracking lineage allows users to determine where/if a feature group is being used. You can track if feature groups are being used to create additional (derived) feature groups or feature views, or to train models.
 
 You can interact with the provenance graph using the UI and the APIs.
 
@@ -262,55 +262,3 @@ In the feature view overview UI you can explore the provenance graph of the feat
     <figcaption>Feature view provenance graph</figcaption>
   </figure>
 </p>
-
-## Step 3: Model lineage
-
-The relationship between feature views and models is captured automatically when you create a model. You can inspect the relationship between feature views and models using the APIs or the UI.
-=== "Python"
-
-    ```python
-    lineage = model.get_feature_view_provenance()
-
-    # List all accessible parent feature views
-    lineage.accessible
-
-    # List all deleted parent feature views
-    lineage.deleted
-
-    # List all the inaccessible parent feature views
-    lineage.inaccessible
-    ```
-
-You can also retrieve the training dataset provenance object.
-=== "Python"
-
-    ```python
-    lineage = model.get_training_dataset_provenance()
-
-    # List all accessible parent training datasets
-    lineage.accessible
-
-    # List all deleted parent training datasets
-    lineage.deleted
-
-    # List all the inaccessible parent training datasets
-    lineage.inaccessible
-    ```
-
-You can also retrieve directly the parent feature view object, without the need to extract them from the provenance links object
-=== "Python"
-
-    ```python
-    feature_view = model.get_feature_view()
-    ```
-This utility method also has the options to initialize the required components for batch or online retrieval of feature vectors. 
-=== "Python"
-
-    ```python
-    model.get_feature_view(init: bool = True, online: Optional[bool]: None)
-    ```
-
-By default, the base init for feature vector retrieval is enabled. In case you have a workflow that requires more particular options, you can disable this base init by setting the `init` to `false`.
-The method detects if it is running within a deployment and will initialize the feature vector retrieval for the serving.
-If the `online` argument is provided and `true` it will initialize for online feature vector retrieval.
-If the `online` argument is provided and `false` it will initialize the feature vector retrieval for batch scoring.

docs/user_guides/mlops/provenance/provenance.md

@@ -0,0 +1,92 @@
+# Provenance 
+
+## Introduction 
+
+Hopsworks feature store allows users to track provenance (lineage) between storage connectors, feature groups, feature views, training datasets and models. Tracking lineage allows users to determine where/if a feature group is being used. You can track if feature groups are being used to create additional (derived) feature groups or feature views, or to train models.
+
+You can interact with the provenance graph using the UI and the APIs.
+
+## Model provenance
+
+The relationship between feature views and models is captured when you create a model. If you do not provide at least the feature view object to the constructor, the provenance will not capture this relation and you will not be able to navigate from model to the feature view it used or from the feature view to the models that were created from it.
+
+You can provide the feature view object and have the training dataset version be inferred.
+=== "Python"
+    ```python
+    # this object will be provided to the model constructor
+    feature_view = hsfs.get_feature_view(...) 
+
+    # when calling this method, the training dataset version is cached in the feature view and is implicitly provided to the model constructor 
+    X_train, X_test, y_train, y_test = feature_view.train_test_split(...) 
+    
+    # provide the feature_view object in the model constructor
+    hsml.model_registry.ModelRegistry.python.create_model(..., feature_view = feature_view)
+    ```
+
+You can of course explicitly provide the training dataset version.
+=== "Python"
+    ```python
+    # this object will be provided to the model constructor
+    feature_view = hsfs.get_feature_view(...) 
+    
+    # this training dataset version will be provided to the model constructor
+    X_train, X_test, y_train, y_test = feature_view.get_train_test_split(training_dataset_version=1)
+    
+    # provide the feature_view object in the model constructor
+    hsml.model_registry.ModelRegistry.python.create_model(..., feature_view = feature_view, training_dataset_version = training_dataset_version)
+    ```
+
+Once the relation is stored in the provenance graph, you can navigate the graph from model to feature view and the other way around.
+
+Users can call the provenance method which will return a Link object containing the parent feature view in either the `accessible`, `deleted` or `inaccessible` list. 
+* If the user has access to both the model and the feature view (including shared featurestores), the feature view will be present in the `accessible` list. 
+* If the user had access to the feature view at some point, through a shared feature store, it used it to generate the model, but after that the sharing feature store access was restricted, the relation is still maintained in the provenance, but the user only has access to limited metadata for the feature view and the provenanance method with return it in the `inaccessible` list. 
+* If the feature view was deleted after the model creation, the provenance will retain the relation, with a minimum amount of metadata for the feature view and provenance method will return the feature view in the `deleted` list.
+
+=== "Python"
+    ```python
+    lineage = model.get_feature_view_provenance()
+
+    # List accessible parent feature view
+    lineage.accessible
+
+    # List deleted parent feature view
+    lineage.deleted
+
+    # List inaccessible parent feature view
+    lineage.inaccessible
+    ```
+
+You can also retrieve the training dataset provenance object.
+=== "Python"
+
+    ```python
+    lineage = model.get_training_dataset_provenance()
+
+    # List accessible parent training dataset
+    lineage.accessible
+
+    # List deleted parent training dataset
+    lineage.deleted
+
+    # List inaccessible parent training dataset
+    lineage.inaccessible
+    ```
+
+You can also retrieve directly the parent feature view object, without the need to extract them from the provenance links object
+=== "Python"
+
+    ```python
+    feature_view = model.get_feature_view()
+    ```
+This utility method also has the options to initialize the required components for batch or online retrieval of feature vectors. 
+=== "Python"
+
+    ```python
+    model.get_feature_view(init: bool = True, online: Optional[bool]: None)
+    ```
+
+By default, the base init for feature vector retrieval is enabled. In case you have a workflow that requires more particular options, you can disable this base init by setting the `init` to `false`.
+The method detects if it is running within a deployment and will initialize the feature vector retrieval for the serving.
+If the `online` argument is provided and `true` it will initialize for online feature vector retrieval.
+If the `online` argument is provided and `false` it will initialize the feature vector retrieval for batch scoring.

mkdocs.yml

@@ -195,6 +195,7 @@ nav:
               - API Protocol: user_guides/mlops/serving/api-protocol.md
               - Troubleshooting: user_guides/mlops/serving/troubleshooting.md
           - Vector Database: user_guides/mlops/vector_database/index.md
+          - Provenance: user_guides/mlops/provenance/provenance.md
       - Migration:
           - 3.X to 4.0: user_guides/migration/40_migration.md
   - Setup and Administration: