Skip to content

Commit

Permalink
correct readme
Browse files Browse the repository at this point in the history
  • Loading branch information
mattiasakesson committed Jun 19, 2024
1 parent 23c5f2a commit 5f324b4
Showing 1 changed file with 42 additions and 21 deletions.
63 changes: 42 additions & 21 deletions examples/monai-2D-mednist/README.rst
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
FEDn Project: MNIST (PyTorch)
-----------------------------
FEDn Project: MonAI 2D Classification with the MedNIST Dataset (PyTorch)
------------------------------------------------------------------------

This is an example FEDn Project based on the classic hand-written text recognition dataset MNIST.
This is an example FEDn Project based on the MonAI 2D Classification with the MedNIST Dataset.
The example is intented as a minimalistic quickstart and automates the handling of training data
by letting the client download and create its partition of the dataset as it starts up.

**Note: These instructions are geared towards users seeking to learn how to work
with FEDn in local development mode using Docker/docker-compose. We recommend all new users
to start by following the Quickstart Tutorial: https://fedn.readthedocs.io/en/stable/quickstart.html**
Links:

- MonAI: https://monai.io/
- Base example notebook: https://github.com/Project-MONAI/tutorials/blob/main/2d_classification/mednist_tutorial.ipynb
- MedNIST dataset: https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/MedNIST.tar.gz

Prerequisites
-------------
Expand All @@ -22,6 +24,7 @@ If using pseudo-distributed mode with docker-compose:
- `Docker <https://docs.docker.com/get-docker>`__
- `Docker Compose <https://docs.docker.com/compose/install>`__


Creating the compute package and seed model
-------------------------------------------

Expand All @@ -36,7 +39,7 @@ Clone this repository, then locate into this directory:
.. code-block::
git clone https://github.com/scaleoutsystems/fedn.git
cd fedn/examples/mnist-pytorch
cd fedn/examples/monai-2D-mednist
Create the compute package:

Expand All @@ -54,29 +57,43 @@ Next, generate a seed model (the first model in a global model trail):
This will create a seed model called 'seed.npz' in the root of the project. This step will take a few minutes, depending on hardware and internet connection (builds a virtualenv).

Download and Prepare the data
-------------------------------------------

Install monai

.. code-block::
pip install -r requirements.txt
Download and divide the data into parts. Set the number of
data parts as an arguments python prepare_data.py NR-OF-DATAPARTS. In the
below command we divide the dataset into 10 parts.
.. code-block::
python prepare_data.py 10
Using FEDn Studio
-----------------

Follow the guide here to set up your FEDn Studio project and learn how to connect clients (using token authentication): `Studio guide <https://fedn.readthedocs.io/en/stable/studio.html>`__.
On the step "Upload Files", upload 'package.tgz' and 'seed.npz' created above.

Connecting clients:
===================

Modifing the data split:
========================

The default traning and test data for this example is downloaded and split direcly by the client when it starts up (see 'startup' entrypoint).
The number of splits and which split used by a client can be controlled via the environment variables ``FEDN_NUM_DATA_SPLITS`` and ``FEDN_DATA_PATH``.
For example, to split the data in 10 parts and start a client using the 8th partiton:
**NOTE: In case a different data path needs to be set, use the env variable FEDN_DATA_PATH.**

.. code-block::
export FEDN_PACKAGE_EXTRACT_DIR=package
export FEDN_NUM_DATA_SPLITS=10
export FEDN_DATA_PATH=./data/clients/8/mnist.pt
fedn client start -in client.yaml --secure=True --force-ssl
The default is to split the data into 2 partitions and use the first partition.
export FEDN_DATA_PATH=<full_path_to_the_dir>/data/
export FEDN_CLIENT_SETTINGS_PATH=<full_path_to_the_dir>/client_settings.yaml
export export FEDN_DATA_SPLIT_INDEX=0
fedn client start -in client.yaml --secure=True --force-ssl
Connecting clients using Docker:
================================
Expand All @@ -87,12 +104,16 @@ For convenience, there is a Docker image hosted on ghrc.io with fedn preinstalle
docker run \
-v $PWD/client.yaml:/app/client.yaml \
-v $PWD/client_settings.yaml:/app/client_settings.yaml \
-e FEDN_PACKAGE_EXTRACT_DIR=package \
-e FEDN_NUM_DATA_SPLITS=2 \
-e FEDN_DATA_PATH=/app/package/data/clients/1/mnist.pt \
-e FEDN_DATA_PATH=./data/ \
-e FEDN_CLIENT_SETTINGS_PATH=/app/client_settings.yaml \
ghcr.io/scaleoutsystems/fedn/fedn:0.9.0 run client -in client.yaml --force-ssl --secure=True
**NOTE: The following instructions are only for SDK-based client communication and for local development environments using Docker.**


Local development mode using Docker/docker compose
--------------------------------------------------

Expand Down Expand Up @@ -134,7 +155,7 @@ Automate experimentation with several clients

If you want to scale the number of clients, you can do so by modifying ``docker-compose.override.yaml``. For example,
in order to run with 3 clients, change the environment variable ``FEDN_NUM_DATA_SPLITS`` to 3, and add one more client
by copying ``client1`` and setting ``FEDN_DATA_PATH`` to ``/app/package/data/clients/3/mnist.pt``
by copying ``client1`` and setting ``FEDN_DATA_PATH`` to ``/app/package/data3/``


Access message logs and validation data from MongoDB
Expand Down

0 comments on commit 5f324b4

Please sign in to comment.