Commit 289a0ff 1 parent ec6f5a8 commit 289a0ff Copy full SHA for 289a0ff
File tree 1 file changed +9
-1
lines changed
examples/pytorch/custom_handler_fp8_fsdp1n2_compile
1 file changed +9
-1
lines changed Original file line number Diff line number Diff line change @@ -15,13 +15,21 @@ sh setup.sh
15
15
In this example we present
16
16
17
17
``` bash
18
+ # # config the PYTHONPATH if needed
19
+ # export PYTHONPATH=/teamspace/studios/this_studio/pytorch-lightning/examples/pytorch/custom_handler_fp8_fsdp1n2_compile:$PYTHONPATH
20
+ cd pytorch-lightning/examples/pytorch/custom_handler_fp8_fsdp1n2_compile
21
+
22
+ # fsdp1 + fp8 + torch compile + gradient checkpointing + cpu offloading
23
+ python train.py --enable_fp8 --enable_torch_compile --enable_gradient_checkpointing --enable_cpu_offload
18
24
25
+ # fsdp2 + fp8 + torch compile + gradient checkpointing (the example does not implement fsdp2 cpu offloading currently)
26
+ python train.py --enable_fsdp2 --enable_fp8 --enable_torch_compile --enable_gradient_checkpointing
19
27
```
20
28
21
29
## Test the handlers
22
30
23
31
``` bash
24
- # config the PYTHONPATH if needed
32
+ # # config the PYTHONPATH if needed
25
33
# export PYTHONPATH=/teamspace/studios/this_studio/pytorch-lightning/examples/pytorch/custom_handler_fp8_fsdp1n2_compile:$PYTHONPATH
26
34
cd pytorch-lightning/examples/pytorch/custom_handler_fp8_fsdp1n2_compile
27
35
pytest tests/*
You can’t perform that action at this time.
0 commit comments