You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: CONTRIBUTING.md
+59-57
Original file line number
Diff line number
Diff line change
@@ -1,17 +1,16 @@
1
1
## Python development setup
2
-
3
2
---
4
3
5
4
- Fork and clone the repository
6
5
7
-
- Create a new Python environment with your favourite environment manager (e.g. virtualenv or conda) and Python 3.9 (newer versions will return a library conflict in `auto_doc.py`)
6
+
- Create a new Python environment with your favourite environment manager, e.g. virtualenv or conda
8
7
9
8
- Install repository in editable mode with development dependencies:
10
9
11
-
```bash
12
-
cd python
13
-
pip install -e ".[dev]"
14
-
```
10
+
```bash
11
+
cd python
12
+
pip install -e ".[dev]"
13
+
```
15
14
16
15
- Install [pre-commit](https://pre-commit.com/) and then activate its hooks. pre-commit is a framework for managing and maintaining multi-language pre-commit hooks. The library uses pre-commit to ensure code-style and code formatting through [ruff](https://docs.astral.sh/ruff/). Run the following commands from the `python` directory:
17
16
@@ -38,104 +37,110 @@ We follow a few best practices for writing the Python documentation:
38
37
39
38
1. Use the google docstring style:
40
39
41
-
```python
42
-
"""[One Line Summary]
40
+
```python
41
+
"""[One Line Summary]
43
42
44
-
[Extended Summary]
43
+
[Extended Summary]
45
44
46
-
[!!! example
47
-
import xyz
48
-
]
45
+
[!!! example
46
+
import xyz
47
+
]
49
48
50
-
# Arguments
51
-
arg1: Type[, optional]. Description[, defaults to `default`]
52
-
arg2: Type[, optional]. Description[, defaults to `default`]
49
+
# Arguments
50
+
arg1: Type[, optional]. Description[, defaults to `default`]
51
+
arg2: Type[, optional]. Description[, defaults to `default`]
53
52
54
-
# Returns
55
-
Type. Description.
53
+
# Returns
54
+
Type. Description.
56
55
57
-
# Raises
58
-
Exception. Description.
59
-
"""
60
-
```
56
+
# Raises
57
+
Exception. Description.
58
+
"""
59
+
```
60
+
61
+
If Python 3 type annotations are used, they are inserted automatically.
61
62
62
-
If Python 3 type annotations are used, they are inserted automatically.
63
63
64
64
2. Hopsworks entity engine methods (e.g. ExecutionEngine etc.) only require a single line docstring.
65
-
3. Private REST API implementations (e.g. FeatureGroupApi etc.) should be fully documented with docstrings without defaults.
66
-
4. Public API such as metadata objects and public REST API implementations should be fully documented with defaults.
65
+
3. Private REST Api implementations (e.g. GitRemoteApi etc.) should be fully documented with docstrings without defaults.
66
+
4. Public Api such as metadata objects and public REST Api implementations should be fully documented with defaults.
67
67
68
68
#### Setup and Build Documentation
69
69
70
70
We use `mkdocs` together with `mike` ([for versioning](https://github.com/jimporter/mike/)) to build the documentation and a plugin called `keras-autodoc` to auto generate Python API documentation from docstrings.
71
71
72
72
**Background about `mike`:**
73
-
`mike` builds the documentation and commits it as a new directory to the gh-pages branch. Each directory corresponds to one version of the documentation. Additionally, `mike` maintains a json in the root of gh-pages with the mappings of versions/aliases for each of the directories available. With aliases you can define extra names like `dev` or `latest`, to indicate stable and unstable releases.
73
+
`mike` builds the documentation and commits it as a new directory to the gh-pages branch. Each directory corresponds to one version of the documentation. Additionally, `mike` maintains a json in the root of gh-pages with the mappings of versions/aliases for each of the directories available. With aliases you can define extra names like `dev` or `latest`, to indicate stable and unstable releases.
74
74
75
-
1.Install Hopsworks with `dev-docs` extras:
75
+
1. Currently we are using our own version of `keras-autodoc`
2. To build the docs, first run the auto doc script:
81
+
2. Install HOPSWORKS with `docs` extras:
82
+
83
+
```bash
84
+
pip install -e .[dev,docs]
85
+
```
82
86
83
-
```bash
84
-
python auto_doc.py
85
-
```
87
+
3. To build the docs, first run the auto doc script:
88
+
89
+
```bash
90
+
cd ..
91
+
python auto_doc.py
92
+
```
86
93
87
94
##### Option 1: Build only current version of docs
88
95
89
-
3. Either build the docs, or serve them dynamically:
96
+
4. Either build the docs, or serve them dynamically:
90
97
91
-
Note: Links and pictures might not resolve properly later on when checking with this build.
92
-
The reason for that is that the docs are deployed with versioning on docs.hopsworks.ai and
93
-
therefore another level is added to all paths, e.g. `docs.hopsworks.ai/[version-or-alias]`.
94
-
Using relative links should not be affected by this, however, building the docs with version
95
-
(Option 2) is recommended.
98
+
Note: Links and pictures might not resolve properly later on when checking with this build.
99
+
The reason for that is that the docs are deployed with versioning on docs.hopsworks.ai and
100
+
therefore another level is added to all paths, e.g. `docs.hopsworks.ai/[version-or-alias]`.
101
+
Using relative links should not be affected by this, however, building the docs with version
102
+
(Option 2) is recommended.
96
103
97
-
```bash
98
-
mkdocs build
99
-
# or
100
-
mkdocs serve
101
-
```
104
+
```bash
105
+
mkdocs build
106
+
# or
107
+
mkdocs serve
108
+
```
102
109
103
110
##### Option 2 (Preferred): Build multi-version doc with `mike`
104
111
105
112
###### Versioning on docs.hopsworks.ai
106
113
107
114
On docs.hopsworks.ai we implement the following versioning scheme:
108
115
109
-
- current master branches (e.g. of hopsworks corresponding to master of Hopsworks): rendered as current Hopsworks snapshot version, e.g. **4.0.0-SNAPSHOT [dev]**, where `dev` is an alias to indicate that this is an unstable version.
110
-
- the latest release: rendered with full current version, e.g. **3.8.0[latest]** with `latest` alias to indicate that this is the latest stable release.
111
-
- previous stable releases: rendered without alias, e.g. **3.4.4**.
116
+
- current master branches (e.g. of hopsworks corresponding to master of Hopsworks): rendered as current Hopsworks snapshot version, e.g. **3.1.0-SNAPSHOT [dev]**, where `dev` is an alias to indicate that this is an unstable version.
117
+
- the latest release: rendered with full current version, e.g. **3.0.1 [latest]** with `latest`alias to indicate that this is the latest stable release.
118
+
- previous stable releases: rendered without alias, e.g. **3.0.0**.
112
119
113
120
###### Build Instructions
114
121
115
-
4. For this you can either checkout and make a local copy of the `upstream/gh-pages` branch, where `mike` maintains the current state of docs.hopsworks.ai, or just build documentation for the branch you are updating:
122
+
4. For this you can either checkout and make a local copy of the `upstream/gh-pages` branch, where
123
+
`mike` maintains the current state of docs.hopsworks.ai, or just build documentation for the branch you are updating:
116
124
117
125
Building *one* branch:
118
126
119
127
Checkout your dev branch with modified docs:
120
-
121
128
```bash
122
129
git checkout [dev-branch]
123
130
```
124
131
125
132
Generate API docs if necessary:
126
-
127
133
```bash
128
134
python auto_doc.py
129
135
```
130
136
131
137
Build docs with a version and alias
132
-
133
138
```bash
134
139
mike deploy [version] [alias] --update-alias
135
140
136
141
# for example, if you are updating documentation to be merged to master,
137
142
# which will become the new SNAPSHOT version:
138
-
mike deploy 4.0.0-SNAPSHOT dev --update-alias
143
+
mike deploy 3.1.0-SNAPSHOT dev --update-alias
139
144
140
145
# if you are updating docs of the latest stable release branch
141
146
mike deploy [version] latest --update-alias
@@ -153,20 +158,17 @@ On docs.hopsworks.ai we implement the following versioning scheme:
153
158
```
154
159
155
160
You can now checkout the gh-pages branch and serve:
156
-
157
161
```bash
158
162
git checkout gh-pages
159
163
mike serve
160
164
```
161
165
162
166
You can also list all available versions/aliases:
163
-
164
167
```bash
165
168
mike list
166
169
```
167
170
168
171
Delete and reset your local gh-pages branch:
169
-
170
172
```bash
171
173
mike delete --all
172
174
@@ -192,7 +194,7 @@ PAGES = {
192
194
193
195
Now you can add a template markdown file to the `docs/templates` directory with the name you specified in the auto-doc script. The `new_template.md` file should contain a tag to identify the place at which the API documentation should be inserted:
194
196
195
-
````
197
+
```
196
198
## The XYZ package
197
199
198
200
{{module}}
@@ -205,7 +207,7 @@ Some extra content here.
205
207
```
206
208
207
209
{{xyz.asd}}
208
-
````
210
+
```
209
211
210
212
Finally, run the `auto_doc.py` script, as decribed above, to update the documentation.
0 commit comments