diff --git a/docs/docs/core/flow_methods.mdx b/docs/docs/core/flow_methods.mdx index 63a45e7f..c52d6782 100644 --- a/docs/docs/core/flow_methods.mdx +++ b/docs/docs/core/flow_methods.mdx @@ -1,5 +1,6 @@ --- title: Flow Running +toc_max_heading_level: 4 description: Run a CocoIndex Flow, including build / update data in the target storage and evaluate the flow without changing the target storage. --- @@ -70,8 +71,7 @@ This is to achieve best efficiency. ### One time update - - +#### CLI The `cocoindex update` subcommand creates/updates data in the target storage. @@ -81,7 +81,9 @@ Once it's done, the target data is fresh up to the moment when the function is c python main.py cocoindex update ``` - +#### Library API + + The `update()` async method creates/updates data in the target storage. @@ -108,8 +110,7 @@ A data source may enable one or multiple *change capture mechanisms*: Change capture mechanisms enable CocoIndex to continuously capture changes from the source data and update the target data accordingly, under live update mode. - - +#### CLI To perform live update, run the `cocoindex update` subcommand with `-L` option: @@ -120,7 +121,9 @@ python main.py cocoindex update -L If there's at least one data source with change capture mechanism enabled, it will keep running until the aborted (e.g. by `Ctrl-C`). Otherwise, it falls back to the same behavior as one time update, and will finish after a one-time update is done. - +#### Library API + + To perform live update, you need to create a `cocoindex.FlowLiveUpdater` object using the `cocoindex.Flow` object. @@ -145,8 +148,10 @@ my_updater = cocoindex.FlowLiveUpdater( A `FlowLiveUpdater` object supports the following methods: +* `start()`: Start the updater. + CocoIndex will continuously capture changes from the source data and update the target data accordingly in background threads managed by the engine. * `abort()`: Abort the updater. -* `wait()` (async): Wait for the updater to finish. It only unblocks in one of the following cases: +* `wait()`: Wait for the updater to finish. It only unblocks in one of the following cases: * The updater was aborted. * A one time update is done, and live update is not enabled: either `live_mode` is `False`, or all data sources have no change capture mechanisms enabled. @@ -154,6 +159,8 @@ A `FlowLiveUpdater` object supports the following methods: ```python my_updater = cocoindex.FlowLiveUpdater(demo_flow) +# Start the updater. +my_updater.start() # Perform your own logic (e.g. a query loop). ... @@ -163,26 +170,47 @@ print(my_updater.update_stats()) # Abort the updater. my_updater.abort() # Wait for the updater to finish. -await my_updater.wait() +my_updater.wait() ``` Python SDK also allows you to use the updater as a context manager. -It will abort and wait for the updater to finish automatically when the context is exited. -The following code is equivalent to the code above: +It will automatically start the updater during the context entry, and abort and wait for the updater to finish automatically when the context is exited. +The following code is equivalent to the code above (if no early return happens): ```python -async with cocoindex.FlowLiveUpdater(demo_flow) as my_updater: +with cocoindex.FlowLiveUpdater(demo_flow) as my_updater: # Perform your own logic (e.g. a query loop). ... print(my_updater.update_stats()) ``` -Within a synchronous function, remove `async` before `with`, like this: +CocoIndex also provides asynchronous versions of APIs for blocking operations, including: -```python -with cocoindex.FlowLiveUpdater(demo_flow) as my_updater: +* `start_async()` and `wait_async()`, e.g. + + ```python + my_updater = cocoindex.FlowLiveUpdater(demo_flow) + # Start the updater. + await my_updater.start_async() + + # Perform your own logic (e.g. a query loop). ... -``` + + # Print the update stats. + print(my_updater.update_stats()) + # Abort the updater. + my_updater.abort() + # Wait for the updater to finish. + await my_updater.wait_async() + ``` +* Async context manager, e.g. + + ```python + async with cocoindex.FlowLiveUpdater(demo_flow) as my_updater: + # Perform your own logic (e.g. a query loop). + ... + print(my_updater.update_stats()) + ``` @@ -191,8 +219,7 @@ with cocoindex.FlowLiveUpdater(demo_flow) as my_updater: CocoIndex allows you to run the transformations defined by the flow without updating the target storage. - - +### CLI The `cocoindex evaluate` subcommand runs the transformation and dumps flow outputs. It takes the following options: @@ -208,7 +235,9 @@ Example: python main.py cocoindex evaluate --output-dir ./eval_output ``` - +### Library API + + The `evaluate_and_dump()` method runs the transformation and dumps flow outputs to files.