Skip to content

Commit a52e5dc

Browse files
authored
k8s release docs (#410)
1 parent 9181305 commit a52e5dc

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

81 files changed

+797
-5967
lines changed

docs/index.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -185,7 +185,7 @@ pointer-events: initial;
185185
</a>
186186
</div>
187187
<div id="w-node-_4a479fbb-90c7-9f47-d439-20aa6a224339-46672785" class="infra">
188-
<a href="./setup_installation/on_prem/hopsworks_installer/">
188+
<a href="./setup_installation/on_prem/contact_hopsworks/">
189189
<img src="images/icons8-database.svg" loading="lazy" alt="" class="infra-icon">
190190
<div class="name_item small">On-premise</div>
191191
</a>
@@ -265,7 +265,7 @@ Hopsworks provides projects as a secure sandbox in which teams can collaborate a
265265
Hopsworks provides a FTI (feature/training/inference) pipeline architecture for ML systems. Each part of the pipeline is defined in a Hopsworks job which corresponds to a Jupyter notebook, a python script or a jar. The production pipelines are then orchestrated with Airflow which is bundled in Hopsworks. Hopsworks provides several python environments that can be used and customized for each part of the FTI pipeline, for example switching between using PyTorch or TensorFlow in the training pipeline. You can train models on as many GPUs as are installed in a Hopsworks cluster and easily share them among users. You can also run Spark, Spark Streaming, or Flink programs on Hopsworks. JupyterLab is also bundled which can be used to run Python and Spark interactively.
266266

267267
## Available on any Platform
268-
Hopsworks is available as a both managed platform in the cloud on AWS, Azure, and GCP, and can be installed on any Linux-based virtual machines (Ubuntu/Redhat compatible), even in air-gapped data centers. Hopsworks is also available as a serverless platform that manages and serves both your features and models.
268+
Hopsworks is available to be installed on a kubernetes cluster in the cloud on AWS, Azure, and GCP, and On-Prem (Ubuntu/Redhat compatible), even in air-gapped data centers. Hopsworks is also available as a serverless platform that manages and serves both your features and models.
269269

270270
## Join the community
271271
- Ask questions and give us feedback in the [Hopsworks Community](https://community.hopsworks.ai/)

docs/js/dropdown.js

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
1-
document.getElementsByClassName("md-tabs__link")[7].style.display = "none";
2-
document.getElementsByClassName("md-tabs__link")[9].style.display = "none";
1+
document.getElementsByClassName("md-tabs__link")[6].style.display = "none";
2+
document.getElementsByClassName("md-tabs__link")[8].style.display = "none";
33

docs/admin/alert.md docs/setup_installation/admin/alert.md

+9-9
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ Cluster Settings from the dropdown menu. In the Cluster Settings' Alerts tab you
1414
manager to send alerts via email, slack or pagerduty.
1515

1616
<figure>
17-
<img src="../../assets/images/alerts/configure-alerts.png" alt="Configure alerts"/>
17+
<img src="../../../assets/images/alerts/configure-alerts.png" alt="Configure alerts"/>
1818
<figcaption>Configure alerts</figcaption>
1919
</figure>
2020

@@ -23,7 +23,7 @@ To send alerts via email you need to configure an SMTP server. Click on the _Con
2323
button on the left side of the **email** row and fill out the form that pops up.
2424

2525
<figure>
26-
<img src="../../assets/images/alerts/smtp-config.png" alt="Configure Email Alerts"/>
26+
<img src="../../../assets/images/alerts/smtp-config.png" alt="Configure Email Alerts"/>
2727
<figcaption>Configure Email Alerts</figcaption>
2828
</figure>
2929

@@ -34,27 +34,27 @@ button on the left side of the **email** row and fill out the form that pops up.
3434
CRAM-MD5, LOGIN or PLAIN.
3535

3636
Optionally cluster wide Email alert receivers can be added in _Default receiver emails_.
37-
These receivers will be available to all users when they create event triggered [alerts](../../user_guides/fs/feature_group/data_validation_best_practices#setup-alerts).
37+
These receivers will be available to all users when they create event triggered [alerts](../../../user_guides/fs/feature_group/data_validation_best_practices#setup-alerts).
3838

3939
### Step 3: Configure Slack Alerts
4040
Alerts can also be sent via Slack messages. To be able to send Slack messages you first need to configure
4141
a Slack webhook. Click on the _Configure_ button on the left side of the **slack** row and past in your
4242
[Slack webhook](https://api.slack.com/messaging/webhooks) in _Webhook_.
4343

4444
<figure>
45-
<img src="../../assets/images/alerts/slack-config.png" alt="Configure slack Alerts"/>
45+
<img src="../../../assets/images/alerts/slack-config.png" alt="Configure slack Alerts"/>
4646
<figcaption>Configure slack Alerts</figcaption>
4747
</figure>
4848

4949
Optionally cluster wide Slack alert receivers can be added in _Slack channel/user_.
50-
These receivers will be available to all users when they create event triggered [alerts](../../user_guides/fs/feature_group/data_validation_best_practices/#setup-alerts).
50+
These receivers will be available to all users when they create event triggered [alerts](../../../user_guides/fs/feature_group/data_validation_best_practices/#setup-alerts).
5151

5252
### Step 4: Configure Pagerduty Alerts
5353
Pagerduty is another way you can send alerts from Hopsworks. Click on the _Configure_ button on the left side of
5454
the **pagerduty** row and fill out the form that pops up.
5555

5656
<figure>
57-
<img src="../../assets/images/alerts/pagerduty-config.png" alt="Configure Pagerduty Alerts"/>
57+
<img src="../../../assets/images/alerts/pagerduty-config.png" alt="Configure Pagerduty Alerts"/>
5858
<figcaption>Configure Pagerduty Alerts</figcaption>
5959
</figure>
6060

@@ -76,7 +76,7 @@ If you are familiar with Prometheus' [Alert manager](https://prometheus.io/docs/
7676
you can also configure alerts by editing the _yaml/json_ file directly.
7777

7878
<figure>
79-
<img src="../../assets/images/alerts/advanced-config.png" alt="Advanced configuration"/>
79+
<img src="../../../assets/images/alerts/advanced-config.png" alt="Advanced configuration"/>
8080
<figcaption>Advanced configuration</figcaption>
8181
</figure>
8282

@@ -93,7 +93,7 @@ global:
9393
...
9494
```
9595

96-
To test the alerts by creating triggers from Jobs and Feature group validations see [Alerts](../../user_guides/fs/feature_group/data_validation_best_practices/#setup-alerts).
96+
To test the alerts by creating triggers from Jobs and Feature group validations see [Alerts](../../../user_guides/fs/feature_group/data_validation_best_practices/#setup-alerts).
9797

9898
The yaml syntax in the UI is slightly different in that it does not allow double quotes (it will ignore the values but give no error).
9999
Below is an example configuration, that can be used in the UI, with both email and slack receivers configured for system alerts.
@@ -144,4 +144,4 @@ receivers:
144144
```
145145
146146
## Conclusion
147-
In this guide you learned how to configure alerts in Hopsworks.
147+
In this guide you learned how to configure alerts in Hopsworks.

docs/admin/audit/audit-logs.md docs/setup_installation/admin/audit/audit-logs.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ Audit logs can be configured from the _Cluster Settings_ Configuration tab.
1717
You can access the _Configuration_ page of your Hopsworks cluster by clicking on your name, in the top right corner, and choosing _Cluster Settings_ from the dropdown menu.
1818

1919
<figure>
20-
<img src="../../../assets/images/admin/audit/audit-log-vars.png" alt="Audit log configuration" />
20+
<img src="../../../../assets/images/admin/audit/audit-log-vars.png" alt="Audit log configuration" />
2121
<figcaption>Audit log configuration</figcaption>
2222
</figure>
2323

docs/admin/auth.md docs/setup_installation/admin/auth.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -35,12 +35,12 @@ In the **Cluster Settings** _Authentication_ tab you can configure how users aut
3535
[Configure LDAP](../ldap/configure-ldap) and [Configure Kerberos](../ldap/configure-krb).
3636

3737
<figure>
38-
<img src="../../assets/images/admin/auth-config.png" alt="Authentication config" />
38+
<img src="../../../assets/images/admin/auth-config.png" alt="Authentication config" />
3939
<figcaption>Setup Authentication Methods</figcaption>
4040
</figure>
4141

4242
In the figure above we see a cluster with Two-factor authentication disabled, OAuth enabled with one registered
4343
identity provider and LDAP authentication enabled.
4444

4545
## Conclusion
46-
In this guide you learned how to configure authentication methods in Hopsworks.
46+
In this guide you learned how to configure authentication methods in Hopsworks.
File renamed without changes.

docs/admin/ha-dr/ha.md docs/setup_installation/admin/ha-dr/ha.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ At a high level a Hopsworks cluster can be divided into 4 groups of nodes. Each
1010
Example deployment:
1111

1212
<figure>
13-
<img width="800px" src="../../../assets/images/admin/ha_dr/example_ha_cluster.svg" alt="Example HA deployment"/>
13+
<img width="800px" src="../../../../assets/images/admin/ha_dr/example_ha_cluster.svg" alt="Example HA deployment"/>
1414
<figcaption>Example High Available deployment</figcaption>
1515
</figure>
1616

File renamed without changes.
File renamed without changes.

docs/admin/ldap/configure-krb.md docs/setup_installation/admin/ldap/configure-krb.md

+4-4
Original file line numberDiff line numberDiff line change
@@ -19,15 +19,15 @@ If LDAP/Kerberos checkbox is not checked, make sure that you configured your app
1919
clicking on the checkbox.
2020

2121
<figure>
22-
<img src="../../../assets/images/admin/ldap/auth-config-krb.png" alt="Authentication config" />
22+
<img src="../../../../assets/images/admin/ldap/auth-config-krb.png" alt="Authentication config" />
2323
<figcaption>Setup Authentication Methods</figcaption>
2424
</figure>
2525

2626
### Step 2: Edit configuration
2727
Finally, click on edit configuration and fill in the attributes.
2828

2929
<figure>
30-
<img src="../../../assets/images/admin/ldap/configure-krb.png" alt="Kerberos config" />
30+
<img src="../../../../assets/images/admin/ldap/configure-krb.png" alt="Kerberos config" />
3131
<figcaption>Configure Kerberos</figcaption>
3232
</figure>
3333

@@ -59,7 +59,7 @@ All defaults are taken from [OpenLDAP](https://www.openldap.org/).
5959

6060
The login page will now have the choice to use Kerberos for authentication.
6161
<figure>
62-
<img width="400px" src="../../../assets/images/admin/ldap/login-using-krb.png" alt="Log in using Kerberos" />
62+
<img width="400px" src="../../../../assets/images/admin/ldap/login-using-krb.png" alt="Log in using Kerberos" />
6363
<figcaption>Log in using Kerberos</figcaption>
6464
</figure>
6565

@@ -69,4 +69,4 @@ The login page will now have the choice to use Kerberos for authentication.
6969
Kerberos support must also be configured on the browser to use Kerberos for authentication.
7070

7171
## Conclusion
72-
In this guide you learned how to configure Kerberos for authentication.
72+
In this guide you learned how to configure Kerberos for authentication.

docs/admin/ldap/configure-ldap.md docs/setup_installation/admin/ldap/configure-ldap.md

+4-4
Original file line numberDiff line numberDiff line change
@@ -20,15 +20,15 @@ If LDAP/Kerberos checkbox is not checked make sure that you configured your appl
2020
clicking on the checkbox.
2121

2222
<figure>
23-
<img src="../../../assets/images/admin/auth-config.png" alt="Authentication config" />
23+
<img src="../../../../assets/images/admin/auth-config.png" alt="Authentication config" />
2424
<figcaption>Setup Authentication Methods</figcaption>
2525
</figure>
2626

2727
### Step 2: Edit configuration
2828
Finally, click on edit configuration and fill in the attributes.
2929

3030
<figure>
31-
<img src="../../../assets/images/admin/ldap/configure-ldap.png" alt="LDAP config" />
31+
<img src="../../../../assets/images/admin/ldap/configure-ldap.png" alt="LDAP config" />
3232
<figcaption>Configure LDAP</figcaption>
3333
</figure>
3434

@@ -53,7 +53,7 @@ All defaults are taken from [OpenLDAP](https://www.openldap.org/).
5353

5454
The login page will now have the choice to use LDAP for authentication.
5555
<figure>
56-
<img width="400px" src="../../../assets/images/admin/ldap/login-ldap.png" alt="Log in using LDAP" />
56+
<img width="400px" src="../../../../assets/images/admin/ldap/login-ldap.png" alt="Log in using LDAP" />
5757
<figcaption>Log in using LDAP</figcaption>
5858
</figure>
5959

@@ -65,4 +65,4 @@ The login page will now have the choice to use LDAP for authentication.
6565

6666

6767
## Conclusion
68-
In this guide you learned how to configure LDAP for authentication.
68+
In this guide you learned how to configure LDAP for authentication.

docs/admin/ldap/configure-project-mapping.md docs/setup_installation/admin/ldap/configure-project-mapping.md

+9-9
Original file line numberDiff line numberDiff line change
@@ -21,16 +21,16 @@ corner of the navigation bar and choosing *Cluster Settings* from the dropdown m
2121
In the _Project mapping_ tab, you can create a new mapping by clicking on _Create new mapping_.
2222

2323
<figure>
24-
<a href="../../../assets/images/admin/project-mapping/project-mapping-empty.png">
25-
<img src="../../../assets/images/admin/project-mapping/project-mapping-empty.png" alt="Project mapping tab" />
24+
<a href="../../../../assets/images/admin/project-mapping/project-mapping-empty.png">
25+
<img src="../../../../assets/images/admin/project-mapping/project-mapping-empty.png" alt="Project mapping tab" />
2626
</a>
2727
<figcaption>Project mapping</figcaption>
2828
</figure>
2929

3030
This will take you to the create mapping page shown below
3131
<figure>
32-
<a href="../../../assets/images/admin/project-mapping/create-mapping.png">
33-
<img src="../../../assets/images/admin/project-mapping/create-mapping.png" alt="Create mapping" />
32+
<a href="../../../../assets/images/admin/project-mapping/create-mapping.png">
33+
<img src="../../../../assets/images/admin/project-mapping/create-mapping.png" alt="Create mapping" />
3434
</a>
3535
<figcaption>Create mapping</figcaption>
3636
</figure>
@@ -41,8 +41,8 @@ You can also choose the _Project role_ users will be assigned when they are adde
4141
Finally, click on _Create mapping_ and go back to mappings. You should see the newly created mapping(s) as shown below.
4242

4343
<figure>
44-
<a href="../../../assets/images/admin/project-mapping/project-mappings.png">
45-
<img src="../../../assets/images/admin/project-mapping/project-mappings.png" alt="Project mappings" />
44+
<a href="../../../../assets/images/admin/project-mapping/project-mappings.png">
45+
<img src="../../../../assets/images/admin/project-mapping/project-mappings.png" alt="Project mappings" />
4646
</a>
4747
<figcaption>Project mappings</figcaption>
4848
</figure>
@@ -63,8 +63,8 @@ From the list of mappings click on the edit button (:material-pencil:). This wil
6363
the _remote group_, _project name_, and _project role_ of a mapping.
6464

6565
<figure>
66-
<a href="../../../assets/images/admin/project-mapping/edit-mapping.png">
67-
<img src="../../../assets/images/admin/project-mapping/edit-mapping.png" alt="Edit mapping" />
66+
<a href="../../../../assets/images/admin/project-mapping/edit-mapping.png">
67+
<img src="../../../../assets/images/admin/project-mapping/edit-mapping.png" alt="Edit mapping" />
6868
</a>
6969
<figcaption>Edit mapping</figcaption>
7070
</figure>
@@ -88,4 +88,4 @@ add or remove users from projects.
8888

8989

9090
## Conclusion
91-
In this guide you learned how to configure LDAP group to project mapping.
91+
In this guide you learned how to configure LDAP group to project mapping.

docs/admin/ldap/configure-server.md docs/setup_installation/admin/ldap/configure-server.md

+4-5
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# Configure Server for LDAP and Kerberos
22

33
## Introduction
4-
LDAP and Kerberos integration need some configuration in the [Karamel](https://github.com/logicalclocks/karamel-chef)
4+
LDAP and Kerberos integration need some configuration in the helm charts for your
55
cluster definition used to deploy your Hopsworks cluster. This tutorial shows an administrator how to configure the application
66
server for LDAP and Kerberos integration.
77

@@ -34,12 +34,11 @@ ldap:
3434
- security_credentials: contains the password of the user that will be used to query LDAP.
3535
- referral: whether to follow or ignore an alternate location in which an LDAP Request may be processed.
3636
37-
### Without Karamel/Chef
38-
An already deployed instance can be configured to connect to LDAP without re-running Karamel/Chef.
37+
An already deployed instance can be configured to connect to LDAP.
3938
Go to the payara admin UI and create a new JNDI external resource. The name of the resource should be __ldap/LdapResource__.
4039
4140
<figure>
42-
<img src="../../../assets/images/admin/ldap/ldap-resource.png" alt="LDAP Resource" />
41+
<img src="../../../../assets/images/admin/ldap/ldap-resource.png" alt="LDAP Resource" />
4342
<figcaption>LDAP Resource</figcaption>
4443
</figure>
4544
@@ -95,4 +94,4 @@ Both Kerberos and LDAP attributes need to be specified to configure Kerberos. Th
9594
Initiator should be set to false.
9695
9796
## Conclusion
98-
In this guide you learned how to configure the application server for LDAP and Kerberos.
97+
In this guide you learned how to configure the application server for LDAP and Kerberos.

docs/admin/monitoring/grafana.md docs/setup_installation/admin/monitoring/grafana.md

+3-3
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ You can access the admin page of your Hopsworks cluster by clicking on your name
1717
You can then navigate to the _Monitoring_ tab. The _Monitoring_ tab gives you access to several of the observability tools that are already deployed to help you manage the health of the cluster.
1818

1919
<figure>
20-
<img src="../../../assets/images/admin/monitoring/monitoring_tab.png" alt="monitoring tab" />
20+
<img src="../../../../assets/images/admin/monitoring/monitoring_tab.png" alt="monitoring tab" />
2121
<figcaption>Monitoring tab</figcaption>
2222
</figure>
2323

@@ -36,7 +36,7 @@ Dashboards are organized into three folders:
3636
- **Kubernetes**: If you have integrated Hopsworks with a Kubernetes cluster, this folder contains the dashboards to monitor the health of the Kubernetes cluster.
3737

3838
<figure>
39-
<img src="../../../assets/images/admin/monitoring/grafana.png" alt="Grafana view" />
39+
<img src="../../../../assets/images/admin/monitoring/grafana.png" alt="Grafana view" />
4040
<figcaption>Grafana view</figcaption>
4141
</figure>
4242

@@ -50,4 +50,4 @@ The default dashboards are read only and cannot be edited. Additional dashboards
5050

5151
In this guide you learned how to access the Grafana dashboards to monitor the health and performance of the Hopsworks services.
5252

53-
You can find additional documentation on Grafana itself at: [https://grafana.com/docs/](https://grafana.com/docs/)
53+
You can find additional documentation on Grafana itself at: [https://grafana.com/docs/](https://grafana.com/docs/)

docs/admin/monitoring/services-logs.md docs/setup_installation/admin/monitoring/services-logs.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ You can access the admin page of your Hopsworks cluster by clicking on your name
1717
You can then navigate to the _Monitoring_ tab. The _Monitoring_ tab gives you access to several of the observability tools that are already deployed to help you manage the health of the cluster.
1818

1919
<figure>
20-
<img src="../../../assets/images/admin/monitoring/monitoring_tab.png" alt="monitoring tab" />
20+
<img src="../../../../assets/images/admin/monitoring/monitoring_tab.png" alt="monitoring tab" />
2121
<figcaption>Monitoring tab</figcaption>
2222
</figure>
2323

@@ -32,7 +32,7 @@ You can filter the logs of a specific service by searching for the term `service
3232
Currently only the logs of the following services are collected and indexed: Hopsworks web application (called `domain1` in the log entries), namenodes, resource managers, datanodes, nodemanagers, Kafka brokers, Hive services and RonDB. These are the core component of the platform, additional logs will be added in the future.
3333

3434
<figure>
35-
<img src="../../../assets/images/admin/monitoring/services_logs.png" alt="OpenSearch Dashboards with services logs" />
35+
<img src="../../../../assets/images/admin/monitoring/services_logs.png" alt="OpenSearch Dashboards with services logs" />
3636
<figcaption>OpenSearch Dashboards displaying the logs</figcaption>
3737
</figure>
3838

0 commit comments

Comments
 (0)