Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Link to docs tutorial from main readme #519

Merged
merged 6 commits into from
Feb 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ Getting started

The best way to get started is to take the quickstart tutorial:

- `Quickstart PyTorch <https://github.com/scaleoutsystems/fedn/tree/master/examples/mnist-pytorch>`__
- `Quickstart PyTorch <https://fedn.readthedocs.io/en/latest/quickstart.html>`__

Documentation
=============
Expand All @@ -57,9 +57,9 @@ You will find more details about the architecture, compute package and how to de

FEDn Studio
===============
Scaleout develops a Django Application, FEDn Studio, that provides a UI, authentication/authorization, client identity management, project-based multitenancy for manging multiple projects, and integration with your MLOps pipelines.
There are also additional tooling and charts for deployments on Kubernetes including integration with several projects from the cloud native landscape. See `FEDn Framework <https://www.scaleoutsystems.com/framework>`__
for more information.
Scaleout also develops FEDn Studio, a web application that extends the FEDn SDK with a UI, production-grade deployment of the FEDn server side on Kubernetes, user authentication/authorization, client identity/API-token management, and project-based multitenancy for segmenting work and resources into collaboration workspaces. FEDn Studio is available as a fully managed service.
There is also additional tooling and charts for self-managed deployment on Kubernetes including integration with several projects from the cloud native landscape.
See `FEDn Framework <https://www.scaleoutsystems.com/framework>`__ for more information.


Making contributions
Expand Down
6 changes: 3 additions & 3 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@
:caption: Table of Contents

introduction
architecture
quickstart
apiclient
tutorial
architecture
aggregators
helpers
tutorial
faq
modules

Expand All @@ -18,4 +18,4 @@ Indices and tables

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
* :ref:`search`
43 changes: 37 additions & 6 deletions docs/quickstart.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,16 @@ Prerequisites
- `Docker <https://docs.docker.com/get-docker>`__
- `Docker Compose <https://docs.docker.com/compose/install>`__

Launch a pseudo-distributed FEDn Network
-------------


Clone this repository, locate into it and start a pseudo-distributed FEDn network using docker-compose:

.. code-block::

docker-compose up


This starts up the needed backend services MongoDB and Minio, the API Server and one Combiner.
You can verify the deployment using these urls:

Expand All @@ -30,6 +33,9 @@ You can verify the deployment using these urls:
.. warning::
The FEDn network is configured to use a local Minio and MongoDB instances for storage. This is not suitable for production, but is fine for testing.

Install the FEDn SDK
-------------

.. note::
To programmatically interact with the FEDn network use the APIClient.
Install the FEDn via pip:
Expand All @@ -38,9 +44,13 @@ You can verify the deployment using these urls:

$ pip install fedn
# or from source
$ cd fedn
$ cd fedn/fedn
$ pip install .


Prepare the compute package and seed the FEDn network
-------------

Next, we will prepare the client. A key concept in FEDn is the compute package -
a code bundle that contains entrypoints for training and (optionally) validating a model update on the client.

Expand Down Expand Up @@ -68,7 +78,10 @@ Upload the compute package and seed model to FEDn:
>>> from fedn import APIClient
>>> client = APIClient(host="localhost", port=8092)
>>> client.set_package("package.tgz", helper="numpyhelper")
>>> client.set_initial_model("seed.npz")
>>> client.set_initial_model("seed.npz")

Configure and attach clients
-------------

The next step is to configure and attach clients. For this we need to download data and make data partitions:

Expand Down Expand Up @@ -117,6 +130,10 @@ To connect a client that uses the data partition ``data/clients/1/mnist.pt`` and
--network=fedn_default \
ghcr.io/scaleoutsystems/fedn/fedn:0.8.0-mnist-pytorch run client -in client.yaml --name client1


Start a training session
-------------

Observe the API Server logs and combiner logs, you should see the client connecting.
You are now ready to start training the model. In the python enviroment you installed FEDn:

Expand All @@ -135,10 +152,18 @@ Please see :py:mod:`fedn.network.api` for more details on the APIClient.

There is also a Jupyter `Notebook <https://github.com/scaleoutsystems/fedn/blob/master/examples/mnist-pytorch/API_Example.ipynb>`_ version of this tutorial including examples of how to fetch and visualize model validations.

Automate and scale up experimentation with several clients
Automate experimentation with several clients
----------------------------------------------------------
Now that you have an understanding of the main components of FEDn, you can use the provided docker-compose templates to automate deployment of FEDn and clients.
To start the network and attach 4 clients. Standing in ``examples/mnist-pytorch``, run the following docker compose command:
You can use the provided docker-compose templates to automate deployment of FEDn and clients.

Split the dataset in 4 partitions:

.. code-block::

bin/split_data --num_splits=4


To start 4 clients, standing in ``examples/mnist-pytorch``, run the following docker compose command:

.. code-block::

Expand Down Expand Up @@ -170,3 +195,9 @@ You can clean up by running

docker-compose down

Where to go from here?
--------
With you first FEDn federation deployed, we suggest that you take a close look at how a FEDn project is structured
and how you develop your own compute package:

- Compute package: :ref:`tutorial-label`
2 changes: 1 addition & 1 deletion docs/tutorial.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.. _tutorial-label:

Tutorial: Compute Package
Compute Package
================================================

This tutorial walks you through the design of a *compute package* for a FEDn client. The compute package is a tar.gz bundle of the code to be executed by each data-provider/client.
Expand Down
2 changes: 1 addition & 1 deletion examples/mnist-pytorch/README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Clone this repository, locate into this directory:
.. code-block::

git clone https://github.com/scaleoutsystems/fedn.git
cd fedn/examples/mnist-keras
cd fedn/examples/mnist-pytorch

Start a pseudo-distributed FEDn network using docker-compose:

Expand Down
Loading