You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: docs/user_guides/projects/python/custom_commands.md
+2-2
Original file line number
Diff line number
Diff line change
@@ -23,11 +23,11 @@ To use the UI, navigate to the Python environment in the Project settings. In th
23
23
</p>
24
24
25
25
## Code
26
-
You can also run the custom commands using the REST API. From the REST API, you should provide the path, in HOPSFS, to the bash script and the artifacts(comma separated string of paths in HopsFs). The REST API endpoint for running custom commands is: `hopsworks-api/api/project/<projectId>/python/environments/<pythonVersion>/commands/custom` and the body should look like this:
26
+
You can also run the custom commands using the REST API. From the REST API, you should provide the path, in HOPSFS, to the bash script and the artifacts(comma separated string of paths in HopsFs). The REST API endpoint for running custom commands is: `hopsworks-api/api/project/<projectId>/python/environments/<environmentName>/commands/custom` and the body should look like this:
Copy file name to clipboardexpand all lines: docs/user_guides/projects/python/python_env_clone.md
+17-7
Original file line number
Diff line number
Diff line change
@@ -2,7 +2,7 @@
2
2
3
3
### Introduction
4
4
5
-
Cloning an environment in Hopsworks means creating a snapshot of one of our provided base environments. The base environments are immutable, meaning that it is required to clone an environment before you can make any change to it, such as installing your own libraries. This is to ensure that the project maintains a set of stable environments that are tested with the capabilities of the platform and allows users to build pipelines using supported versions of different python packages.
5
+
Cloning an environment in Hopsworks means creating a copy of one of the base environments. The base environments are immutable, meaning that it is required to clone an environment before you can make any change to it, such as installing your own libraries. This ensures that the project maintains a set of stable environments that are tested with the capabilities of the platform, meanwhile through cloning, allowing users to further customize an environment without affecting the base environments.
6
6
7
7
In this guide, you will learn how to clone an environment.
8
8
@@ -15,30 +15,40 @@ First select an environment, for example the `python-feature-pipeline`.
<figcaption>Select a base environment</figcaption>
19
19
</figure>
20
20
</p>
21
21
22
22
## Step 2: Clone environment
23
23
24
-
The environment can now be cloned by clicking `Clone environment` and entering a name and description. The interface will show `Syncing packages` while creating the environment.
24
+
The environment can now be cloned by clicking `Clone env` and entering a name and description. The interface will show `Syncing packages` while creating the environment.
<figcaption>Environment is now cloned</figcaption>
39
+
</figure>
40
+
</p>
41
+
42
+
!!! notice "What does the CUSTOM mean?"
43
+
Notice that the cloned environment is tagged as `CUSTOM`, it means that it is a base environment which has been cloned.
44
+
45
+
!!! notice "Base environment also marked"
46
+
When you select a `CUSTOM` environment the base environment it was cloned from is also shown.
37
47
38
48
## Concerning upgrades
39
49
40
50
!!! warning "Please note"
41
-
The base environments are automatically upgraded when Hopsworks is upgraded and application code should keep functioning provided that no breaking changes were made in the upgraded version of the environment. However a `CUSTOM` environment is not automatically upgraded and the user will need to manually apply the required changes if they encounter issues.
51
+
The base environments are automatically upgraded when Hopsworks is upgraded and application code should keep functioning provided that no breaking changes were made in the upgraded version of the environment. A `CUSTOM` environment is not automatically upgraded and the users is recommended to reapply the modifications to a base environment if they encounter issues after an upgrade.
Copy file name to clipboardexpand all lines: docs/user_guides/projects/python/python_install.md
+26-10
Original file line number
Diff line number
Diff line change
@@ -8,17 +8,25 @@ In this guide, you will learn how to install Python packages using these differe
8
8
9
9
* PyPi, using pip package manager
10
10
* A conda channel, using conda package manager
11
-
* Packages saved in certain file formats, currently we support .whl or .egg
11
+
* Packages contained in .whl format
12
12
* A public or private git repository
13
13
* A requirements.txt file to install multiple libraries at the same time using pip
14
-
* An environment.yml file to install multiple libraries at the same time using conda and pip
15
-
16
-
Under the `Project settings` section you can find the `Python environment` setting.
17
14
18
15
!!! notice "Notice"
19
16
If your libraries require installing some extra OS-Level packages, refer to the guide custom commands guide on how to install OS-Level packages.
20
17
21
-
### Name and version
18
+
19
+
### Step 1: Go to environments page
20
+
21
+
Under the `Project settings` section select the `Python environment` setting.
22
+
23
+
### Step 2: Select a CUSTOM environment
24
+
25
+
Select the environment that you have previously cloned and want to modify.
26
+
27
+
### Step 3: Installation options
28
+
29
+
#### Name and version
22
30
23
31
Enter the name and, optionally, the desired version to install.
24
32
@@ -29,7 +37,7 @@ Enter the name and, optionally, the desired version to install.
29
37
</figure>
30
38
</p>
31
39
32
-
### Search
40
+
####Search
33
41
34
42
Enter the search term and select a library and version to install.
35
43
@@ -40,7 +48,7 @@ Enter the search term and select a library and version to install.
40
48
</figure>
41
49
</p>
42
50
43
-
### Distribution (.whl, .egg..)
51
+
####Distribution (.whl, .egg..)
44
52
45
53
Install a python package by uploading the corresponding package file and selecting it in the file browser.
46
54
@@ -51,12 +59,20 @@ Install a python package by uploading the corresponding package file and selecti
51
59
</figure>
52
60
</p>
53
61
54
-
### Git source
62
+
#### Git source
63
+
64
+
The URL you should provide is the same as you would enter on the command line using `pip install git+{repo_url}`, where `repo_url` is the part that you enter in `Git URL`.
65
+
66
+
For example to install matplotlib 3.7.2, the following are correct inputs:
To install from a git repository simply provide the repository URL. The URL you should provide is the same as you would enter on the command line using `pip install git+{repo_url}`.
57
72
In the case of a private git repository, also select whether it is a GitHub or GitLab repository and the preconfigured access token for the repository.
58
73
59
-
**Note**: If you are installing from a git repository which is not GitHub or GitLab simply supply the access token in the URL. Keep in mind that in this case the access token may be visible in logs for other users in the same project to see.
74
+
!!! notice "Keep your secrets safe"
75
+
If you are installing from a git repository which is not GitHub or GitLab simply supply the access token in the URL. Keep in mind that in this case the access token may be visible in logs for other users in the same project to see.
0 commit comments