Skip to content

Commit ced5624

Browse files
gibchikafagibchikafa
committedNov 8, 2024
[HWORKS-1670] Ray documentation
1 parent dd321b5 commit ced5624

18 files changed

+355
-2
lines changed
 

‎docs/assets/images/guides/jobs/create_new_job.png

136 KB
Loading

‎docs/assets/images/guides/jobs/jobs_overview.png

22.2 KB
Loading

‎docs/assets/images/guides/jobs/ray_application_logs.png

145 KB
Loading

‎docs/assets/images/guides/jobs/ray_dashboard.png

152 KB
Loading

‎docs/assets/images/guides/jobs/ray_execution.png

261 KB
Loading

‎docs/assets/images/guides/jobs/ray_runtime_env_and_additional_files.png

153 KB
Loading

‎docs/assets/images/guides/jobs/resource_configuration_ray.png

135 KB
Loading

‎docs/assets/images/guides/jobs/start_ray_job.png

337 KB
Loading

‎docs/assets/images/guides/jobs/upload_ray_script.png

250 KB
Loading

‎docs/assets/images/guides/jupyter/environment_list.png

227 KB
Loading

‎docs/assets/images/guides/jupyter/jupyter_overview.png

140 KB
Loading

‎docs/assets/images/guides/jupyter/ray_jupyter_notebook_session.png

232 KB
Loading

‎docs/assets/images/guides/jupyter/ray_kernel.png

239 KB
Loading

‎docs/assets/images/guides/jupyter/ray_resource_and_compute.png

116 KB
Loading

‎docs/user_guides/projects/jobs/ray_job.md

+211
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,211 @@
1+
---
2+
description: Documentation on how to configure and execute a Ray job on Hopsworks.
3+
---
4+
5+
# How To Run A Ray Job
6+
7+
## Introduction
8+
9+
All members of a project in Hopsworks can launch the following types of applications through a project's Jobs service:
10+
11+
- Python (*Hopsworks Enterprise only*)
12+
- Apache Spark
13+
- Ray
14+
15+
Launching a job of any type is very similar process, what mostly differs between job types is
16+
the various configuration parameters each job type comes with. Hopsworks support scheduling to run jobs on a regular basis,
17+
e.g backfilling a Feature Group by running your feature engineering pipeline nightly. Scheduling can be done both through the UI and the python API,
18+
checkout [our Scheduling guide](schedule_job.md).
19+
20+
21+
## UI
22+
23+
### Step 1: Jobs overview
24+
25+
The image below shows the Jobs overview page in Hopsworks and is accessed by clicking `Jobs` in the sidebar.
26+
27+
<p align="center">
28+
<figure>
29+
<img src="../../../../assets/images/guides/jobs/jobs_overview.png" alt="Jobs overview">
30+
<figcaption>Jobs overview</figcaption>
31+
</figure>
32+
</p>
33+
34+
### Step 2: Create new job dialog
35+
36+
Click `New Job` and the following dialog will appear.
37+
38+
<p align="center">
39+
<figure>
40+
<img src="../../../../assets/images/guides/jobs/create_new_job.png" alt="Create new job dialog">
41+
<figcaption>Create new job dialog</figcaption>
42+
</figure>
43+
</p>
44+
45+
### Step 3: Set the job type
46+
47+
By default, the dialog will create a Spark job. Make sure `RAY` is chosen.
48+
49+
### Step 4: Set the script
50+
51+
Next step is to select the program to run. You can either select `From project`, if the file was previously uploaded to Hopsworks, or `Upload new file` which lets you select a file from your local filesystem as demonstrated below. After that set the name for the job. By default, the job name is the same as the file name, but you can customize it here.
52+
53+
<p align="center">
54+
<figure>
55+
<img src="../../../../assets/images/guides/jobs/upload_ray_script.png" alt="Configure program">
56+
<figcaption>Configure program</figcaption>
57+
</figure>
58+
</p>
59+
60+
### Step 5 (optional): Advanced configuration
61+
62+
Resource allocation for the Driver and Workers can be configured.
63+
64+
!!! notice "Using the resources in the Ray script"
65+
The resources you will configure are for the cluster that will be created when you start a Ray job. You can still
66+
provide extra configuration later in your script to use the resources in the created cluster using `ScalingConfig` i.e.,
67+
`ScalingConfig(num_workers=4, trainer_resources={"CPU": 1}, use_gpu=True)`.
68+
69+
* `Driver memory`: Number of MBs to allocate for Driver
70+
71+
* `Driver virtual cores`: Number of cores to allocate for the Driver
72+
73+
* `Worker memory`: Number of MBs to allocate for each worker
74+
75+
* `Worker cores`: Number of cores to allocate for each worker
76+
77+
* `Min workers`: Minimum number of workers to start with
78+
79+
* `Max workers`: Maximum number of workers to scale up to
80+
81+
82+
<p align="center">
83+
<figure>
84+
<img src="../../../../assets/images/guides/jobs/resource_configuration_ray.png" alt="Resource configuration
85+
for Ray Job">
86+
<figcaption>Resource configuration for the Ray Job</figcaption>
87+
</figure>
88+
</p>
89+
90+
Runtime environment and Additional files required for the Ray job can also be provided.
91+
92+
* `Runtime Environment (Optional)`: A runtime environment describes the dependencies required for the Ray job
93+
including files, packages, environment variables, and more. This is useful when you need to install specific
94+
packages and set environment variables for this particular Ray job. It should be provided as a YAML file. You can
95+
select the file from the project or upload a new one.
96+
97+
* `Additional files`: List of other files required for the Ray job. These files will be placed in `/srv/hops/ray/job`.
98+
99+
<p align="center">
100+
<figure>
101+
<img src="../../../../assets/images/guides/jobs/ray_runtime_env_and_additional_files.png" alt="Runtime
102+
environment and additional files">
103+
<figcaption>Runtime configuration and additional files for Ray job</figcaption>
104+
</figure>
105+
</p>
106+
107+
### Step 6: Execute the job
108+
109+
Now click the `Run` button to start the execution of the job, and then click on `Executions` to see the list of all executions.
110+
111+
112+
<p align="center">
113+
<figure>
114+
<img src="../../../../assets/images/guides/jobs/start_ray_job.png" alt="Start job execution">
115+
<figcaption>Start job execution</figcaption>
116+
</figure>
117+
</p>
118+
119+
## Ray Dashboard
120+
When the Ray job is running, you can access the Ray dashboard to monitor the job. To open the Ray Dashboard click on
121+
the `Ray Dashboard` link under the execution and a new tab will be opened. The Ray dashboard is accessible from the
122+
`Executions` page. The Ray dashboard is only available when the job execution is running. In the Ray Dashboard,
123+
you can monitor the resources used by the job, the number of workers, logs, and the tasks that are running.
124+
<p align="center">
125+
<figure>
126+
<img src="../../../../assets/images/guides/jobs/ray_dashboard.png" alt="Access Ray Dashboard">
127+
<figcaption>Access Ray Dashboard</figcaption>
128+
</figure>
129+
</p>
130+
131+
### Step 9: Application logs
132+
Once the execution is finished, you can click on `Logs` to see the full logs for execution.
133+
134+
<p align="center">
135+
<figure>
136+
<img src="../../../../assets/images/guides/jobs/ray_application_logs.png" alt="Access Ray logs">
137+
<figcaption>Access Ray job execution logs</figcaption>
138+
</figure>
139+
</p>
140+
141+
## Code
142+
143+
### Step 1: Upload the Ray script
144+
145+
This snippet assumes the Ray program is in the current working directory and named `ray_job.py`. If the file is already in the project, you can skip this step.
146+
147+
It will upload the jar to the `Resources` dataset in your project.
148+
149+
```python
150+
151+
import hopsworks
152+
153+
project = hopsworks.login()
154+
155+
dataset_api = project.get_dataset_api()
156+
157+
uploaded_file_path = dataset_api.upload("ray_job.py", "Resources")
158+
159+
```
160+
161+
162+
### Step 2: Create Ray job
163+
164+
In this snippet we get the `JobsApi` object to get the default job configuration for a `RAY` job, set the python script to run and create the `Job` object.
165+
166+
```python
167+
168+
jobs_api = project.get_jobs_api()
169+
170+
ray_config = jobs_api.get_configuration("RAY")
171+
172+
ray_config['appPath'] = uploaded_file_path
173+
ray_config['environmentName'] = "ray-training-pipeline"
174+
ray_config['driverCores'] = 2
175+
ray_config['driverMemory'] = 2048
176+
ray_config['workerCores'] = 2
177+
ray_config['workerMemory'] = 4096
178+
ray_config['minWorkers'] = 1
179+
ray_config['maxWorkers'] = 4
180+
181+
job = jobs_api.create_job("ray_job", ray_config)
182+
183+
```
184+
185+
### Step 3: Execute the job
186+
187+
In this snippet we execute the job synchronously, that is wait until it reaches a terminal state, and then download and print the logs.
188+
189+
```python
190+
191+
execution = job.run(await_termination=True)
192+
193+
out, err = execution.download_logs()
194+
195+
f_out = open(out, "r")
196+
print(f_out.read())
197+
198+
f_err = open(err, "r")
199+
print(f_err.read())
200+
201+
```
202+
203+
### API Reference
204+
205+
[Jobs](https://docs.hopsworks.ai/hopsworks-api/{{{ hopsworks_version }}}/generated/api/jobs/)
206+
207+
[Executions](https://docs.hopsworks.ai/hopsworks-api/{{{ hopsworks_version }}}/generated/api/executions/)
208+
209+
## Conclusion
210+
211+
In this guide you learned how to create and run a Ray job.

‎docs/user_guides/projects/jupyter/ray_notebook.md

+139
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
# How To Run A Ray Notebook
2+
3+
### Introduction
4+
5+
Jupyter is provided as a service in Hopsworks, providing the same user experience and features as if run on your laptop.
6+
7+
* Supports JupyterLab and the classic Jupyter front-end
8+
* Configured with Python3, PySpark and Ray kernels
9+
10+
11+
## Step 1: Jupyter dashboard
12+
13+
The image below shows the Jupyter service page in Hopsworks and is accessed by clicking `Jupyter` in the sidebar.
14+
15+
<p align="center">
16+
<figure>
17+
<img src="../../../../assets/images/guides/jupyter/jupyter_overview.png" alt="Jupyter dashboard in Hopsworks">
18+
<figcaption>Jupyter dashboard in Hopsworks</figcaption>
19+
</figure>
20+
</p>
21+
22+
From this page, you can configure various options and settings to start Jupyter with as described in the sections below.
23+
24+
## Step 2 (Optional): Configure Ray
25+
26+
Next step is to configure the Ray cluster configuration that will be created when you start Ray session later in
27+
Jupyter. Click `edit configuration` to get to the configuration page and select `Ray`.
28+
29+
### Resource and compute
30+
31+
Resource allocation for the Driver and Workers can be configured.
32+
33+
!!! important
34+
The resources you will configure are for the cluster that will be created when you start a Ray session inside the
35+
Jupyter notebook. You can still provide extra configuration later in your code specify the resources you would
36+
like to use in the created cluster for a particular task (i.e., model training) using `ScalingConfig` i.e.,
37+
`ScalingConfig(num_workers=4, trainer_resources={"CPU": 1}, use_gpu=True)`.
38+
39+
* `Driver memory`: Number of MBs to allocate for Driver
40+
41+
* `Driver virtual cores`: Number of cores to allocate for the Driver
42+
43+
* `Worker memory`: Number of MBs to allocate for each worker
44+
45+
* `Worker cores`: Number of cores to allocate for each worker
46+
47+
* `Min workers`: Minimum number of workers to start with
48+
49+
* `Max workers`: Maximum number of workers to scale up to
50+
51+
<p align="center">
52+
<figure>
53+
<img src="../../../../assets/images/guides/jupyter/ray_resource_and_compute.png" alt="Resource configuration for
54+
the Ray kernels">
55+
<figcaption>Resource configuration for the Ray kernels</figcaption>
56+
</figure>
57+
</p>
58+
59+
### Runtime environment and Additional files required for the Ray job can also be provided.
60+
61+
* `Runtime Environment (Optional)`: A runtime environment describes the dependencies required for the Ray session
62+
including files, packages, environment variables, and more. This is useful when you need to install specific
63+
packages and set environment variables for this particular Ray session. It should be provided as a YAML file. You can
64+
select the file from the project or upload a new one.
65+
66+
* `Additional files`: List of other files required for the Ray job. These files will be placed in `/srv/hops/ray/job`.
67+
68+
<p align="center">
69+
<figure>
70+
<img src="../../../../assets/images/guides/jobs/ray_runtime_env_and_additional_files.png" alt="Runtime
71+
environment and additional files">
72+
<figcaption>Runtime configuration and additional files for Ray jupyter session</figcaption>
73+
</figure>
74+
</p>
75+
76+
77+
Click `Save` to save the new configuration.
78+
79+
## Step 3 (Optional): Configure max runtime and root path
80+
81+
Before starting the server there are two additional configurations that can be set next to the `Run Jupyter` button.
82+
83+
The runtime of the Jupyter instance can be configured, this is useful to ensure that idle instances will not be hanging around and keep allocating resources. If a limited runtime is not desirable, this can be disabled by setting `no limit`.
84+
85+
<p align="center">
86+
<figure>
87+
<img src="../../../../assets/images/guides/jupyter/configure_shutdown.png" alt="Configure maximum runtime">
88+
<figcaption>Configure maximum runtime</figcaption>
89+
</figure>
90+
</p>
91+
92+
The root path from which to start the Jupyter instance can be configured. By default it starts by setting the `/Jupyter` folder as the root.
93+
94+
<p align="center">
95+
<figure>
96+
<img src="../../../../assets/images/guides/jupyter/start_from_folder.png" alt="Configure root folder">
97+
<figcaption>Configure root folder</figcaption>
98+
</figure>
99+
</p>
100+
101+
## Step 4: Select the environment
102+
Hopsworks provides a variety of environments to run Jupyter notebooks. Select the environment you want to use by clicking on the dropdown menu.
103+
In order to be able to run a Ray notebook, you need to select the environment that has the Ray kernel installed.
104+
Environment with Ray kernel have a `Ray Enabled` label next to them.
105+
106+
## Step 5: Start Jupyter
107+
108+
Start the Jupyter instance by clicking the `Run Jupyter` button.
109+
110+
## Running Ray code in Jupyter
111+
Once the Jupyter instance is started, you can create a new notebook by clicking on the `New` button and selecting
112+
`Ray` kernel. You can now write and run Ray code in the notebook. When you first run a cell with Ray code, a Ray session will be started and you can monitor the resources used by the job in the Ray dashboard.
113+
114+
<p align="center">
115+
<figure>
116+
<img src="../../../../assets/images/guides/jupyter/ray_kernel.png" alt="Ray Kernel">
117+
<figcaption>Ray Kernel</figcaption>
118+
</figure>
119+
</p>
120+
121+
## Step 6: Access Ray Dashboard
122+
123+
When you start a Ray session in Jupyter, a new application will appear in the Jupyter page. A notebook name from
124+
which the session was started is displayed. You can access the Ray UI by clicking on the `Ray Dashboard` and a new
125+
tab will be opened. The Ray dashboard is only available the notebook or the kernel it was started from is running.
126+
You can kill the Ray session to free up resources by shutting down the kernel in Jupyter. In the Ray Dashboard, you can monitor
127+
the resources used by code you are running, the number of workers, logs, and the tasks that are running.
128+
129+
<p align="center">
130+
<figure>
131+
<img src="../../../../assets/images/guides/jupyter/ray_jupyter_notebook_session.png" alt="Access Ray Dashboard">
132+
<figcaption>Access Ray Dashboard for Jupyter Ray session</figcaption>
133+
</figure>
134+
</p>
135+
136+
## Conclusion
137+
138+
In this guide you learned how to configure and run a Ray application in Jupyter. You can now follow this [guide](..
139+
/python/python_install.md) to install a library that can be used in a notebook.

‎docs/user_guides/projects/jupyter/spark_notebook.md

+3-2
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
Jupyter is provided as a service in Hopsworks, providing the same user experience and features as if run on your laptop.
66

77
* Supports JupyterLab and the classic Jupyter front-end
8-
* Configured with Python3, Spark, PySpark and SparkR kernels
8+
* Configured with Python3, PySpark and Ray kernels
99

1010

1111
## Step 1: Jupyter dashboard
@@ -23,7 +23,8 @@ From this page, you can configure various options and settings to start Jupyter
2323

2424
## Step 2 (Optional): Configure spark
2525

26-
Next step is to configure the Spark properties to be used in Jupyter, Click `edit configuration` to get to the configuration page and select `Spark`.
26+
Next step is to configure the Ray properties to be used in Jupyter, Click `edit configuration` to get to the
27+
configuration page and select `Ray`.
2728

2829
### Resource and compute
2930

‎mkdocs.yml

+2
Original file line numberDiff line numberDiff line change
@@ -153,11 +153,13 @@ nav:
153153
- Jupyter:
154154
- Run Spark Notebook: user_guides/projects/jupyter/spark_notebook.md
155155
- Run Python Notebook: user_guides/projects/jupyter/python_notebook.md
156+
- Run Ray Notebook: user_guides/projects/jupyter/ray_notebook.md
156157
- Remote Filesystem Driver: user_guides/projects/jupyter/remote_filesystem_driver.md
157158
- Jobs:
158159
- Run PySpark Job: user_guides/projects/jobs/pyspark_job.md
159160
- Run Spark Job: user_guides/projects/jobs/spark_job.md
160161
- Run Python Job: user_guides/projects/jobs/python_job.md
162+
- Run Ray Job: user_guides/projects/jobs/ray_job.md
161163
- Run Jupyter Notebook Job: user_guides/projects/jobs/notebook_job.md
162164
- Scheduling: user_guides/projects/jobs/schedule_job.md
163165
- Airflow: user_guides/projects/airflow/airflow.md

0 commit comments

Comments
 (0)
Failed to load comments.