Skip to content

Commit

Permalink
slim down
Browse files Browse the repository at this point in the history
  • Loading branch information
viktorvaladi committed Mar 15, 2024
1 parent ad9d396 commit cecc892
Show file tree
Hide file tree
Showing 2 changed files with 17 additions and 108 deletions.
2 changes: 1 addition & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.organizeImports": true
"source.organizeImports": "explicit"
},
"python.linting.enabled": true,
"python.linting.flake8Enabled": true,
Expand Down
123 changes: 16 additions & 107 deletions docs/distributed.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ Distributed Deployment
===================================

This tutorial outlines the steps for deploying the FEDn framework on a local network, using a workstation as
the host and Raspberry Pi 5 devices as clients. We'll demonstrate this setup using the PyTorch MNIST example model
and compute package.
the host and different devices as clients. For general steps on how to run FEDn, see one of the quickstart tutorials.


.. note::
For deployment over a public network, ensure you have:
Expand All @@ -25,97 +25,16 @@ Launch a distributed FEDn Network
-------------


Note the local IP address of your host device; it will be needed later. Clone the FEDn repository and navigate to the cloned directory.
Start the FEDn network using Docker Compose:

.. code-block::
docker-compose up
Install the FEDn SDK
-------------

.. note::
To programmatically interact with the FEDn network use the APIClient.
Install the FEDn via pip:

.. code-block:: bash
$ pip install fedn
# or from source
$ cd fedn/fedn
$ pip install .
Prepare the compute package and seed for the FEDn network
-------------

From your host device, prepare the compute package and model seed. In this tutorial we will use the
``examples/mnist-pytorch`` example.

Locate into ``examples/mnist-pytorch`` folder and initialize a virtual environment.

.. code-block:: python
bin/init_venv.sh
Now create the compute package and an initial model:

.. code-block::
bin/build.sh
Upload the compute package and seed model to FEDn:

.. code:: python
>>> from fedn import APIClient
>>> client = APIClient(host="localhost", port=8092)
>>> client.set_package("package.tgz", helper="numpyhelper")
>>> client.set_initial_model("seed.npz")
Building a Custom Image for Client Devices (Optional)
-------------
Note the local IP address of your host device; it will be needed later. Follow the standard procedure
to initiate a FEDn network. Once the network is active, upload your compute package and seed (for comprehensive details,
see the quickstart tutorials).

The prebuilt Docker images may not be compatible with Raspberry Pi due to system architecture differences. To
build a custom Docker image:

#. **Clone the Repository on Your Client Device**: Ensure you're in the root directory of the repository.

#. **Build Your Custom Image**: Substitute <custom image name> with your desired image name:

.. code-block::
docker build —build-arg REQUIREMENTS=examples/mnist-pytorch/requirements.txt -t <custom image name> .

Configuring and Attaching Clients
-------------

The next step is to configure and attach clients. Begin with navigating to ``examples/mnist-pytorch`` and
execute the initializion script:

.. code-block:: python
bin/init_venv.sh
Download the data:

.. code-block::
bin/get_data
Split the data in 2 parts for the clients:

.. code-block::
bin/split_data
Data partitions will be generated in the folder 'data/clients'.


FEDn relies on a configuration file for the client to connect to the server. Create a file called 'client.yaml' with
the follwing content:
On your client device, continue with initializing your client. To connect to the host machine we need to ensure we are
routing the correct DNS to our hosts local IP adress. We can do this using the standard FEDn `client.yaml`:

.. code-block::
Expand All @@ -124,35 +43,25 @@ the follwing content:
discover_port: 8092
Make sure to move the file ``client.yaml`` to the root of the examples/mnist-pytorch folder and connect the client to
the network. Note that we are routing `api-server` and `combiner` to our hosts local ip adress:
We can then run using docker by adding the hosts to the docker run command:

.. code-block::
docker run \
-v $PWD/client.yaml:/app/client.yaml \
-v $PWD/data/clients/1:/var/data \
-e ENTRYPOINT_OPTS=--data_path=/var/data/mnist.pt \
—add-host=api-server:<host local ip> \
—add-host=combiner:<host local ip> \
<custom image name> run client -in client.yaml --name client1
<image name> run client -in client.yaml --name client1
Start a training session
-------------
On the host device, observe the API Server logs and combiner logs, you should see the client connecting.
You are now ready to start training the model. In the python enviroment you installed FEDn:
Alternatively updating the `/etc/hosts` file, appending the following lines for running naitively:

.. code:: python
.. code-block::
>>> ...
>>> client.start_session(session_id="test-session", rounds=3)
<host local ip> api-server
<host local ip> combiner
Clean up
--------
You can clean up by running
.. code-block::
Start a training session
-------------

docker-compose down
After connecting with your clients, you are ready to start training sessions from the host machine.

0 comments on commit cecc892

Please sign in to comment.