Skip to content

Commit 03f7319

Browse files
committed
support for clusteridentity
1 parent 39426d3 commit 03f7319

9 files changed

+109
-556
lines changed

README.md

Lines changed: 99 additions & 119 deletions
Original file line numberDiff line numberDiff line change
@@ -11,10 +11,9 @@ This Helm chart deploys a Kubernetes cluster on vSphere using Cluster API with K
1111
- [Cluster Autoscaler Integration](#cluster-autoscaler-integration)
1212
- [Prerequisites](#prerequisites)
1313
- [Installation](#installation)
14-
- [Secret Management](#secret-management)
15-
- [Create Cluster API Secret](#create-cluster-api-secret)
16-
- [Create Cloud Controller Manager Secret](#create-cloud-controller-manager-secret)
17-
- [Create CSI Controller Secret](#create-csi-controller-secret)
14+
- [Credentials Management](#credentials-management)
15+
- [Credentials through Secrets](#credentials-through-secrets)
16+
- [Credentials through VSphereClusterIdentity](#credentials-through-vsphereclusteridentity)
1817
- [Usage](#usage)
1918
- [Creating a cluster](#creating-a-cluster)
2019
- [Upgrading a cluster](#upgrading-a-cluster)
@@ -125,51 +124,97 @@ helm repo update
125124
helm install my-cluster clastix/capi-kamaji-vsphere -f my-values.yaml
126125
```
127126

128-
## Secret Management
127+
## Credentials Management
129128

130-
The chart requires three distinct vSphere access secrets:
129+
Cluster API Provider vSphere (CAPV) supports multiple methods to provide vCenter credentials and authorize clusters to use them:
131130

132-
1. **Cluster API Secret** (default name `vsphere-secret`)
133-
- Used by Cluster API to provision VMs
134-
- Contains vSphere credentials for infrastructure operations
131+
- **Secrets**: credentials are provided via `secret` used by `VSphereCluster`. This will create a unique relationship between the `VSphereCluster` and `secret` and the `secret` cannot be utilized by other clusters.
135132

136-
2. **Cloud Controller Manager Secret** (default name `vsphere-config-secret`)
137-
- Used by the vSphere Cloud Provider Interface
138-
- Contains configuration for vCenter
133+
- **VSphereClusterIdentity**: credentials are provided via `VSphereClusterIdentity`, a cluster scoped resource and enables multiple `VSphereClusters` to share the same set of credentials. The namespaces that are allowed to use the `VSphereClusterIdentity` can also be configured via a `LabelSelector`.
139134

140-
3. **CSI Controller Secret** (default name `csi-config-secret`)
141-
- Used by the Storage Controller Manager
142-
- Enables volume provisioning and attachment
135+
More details on the CAPV documentation: [Cluster API Provider vSphere](https://github.com/kubernetes-sigs/cluster-api-provider-vsphere)
143136

144-
You can leave the chart to create these secrets or reference existing ones:
137+
### Credentials through Secrets
138+
The chart creates three secrets by default, one for each component that requires vSphere credentials. These secrets are created in the same namespace as the `Cluster` resource and are labeled with the cluster name:
145139

146140
```yaml
147-
# Using existing secrets
148-
vsphere:
149-
secret:
150-
create: false
151-
name: vsphere-secret
141+
# Create the vsphere-secret for Cluster API
142+
cat <<EOF | kubectl apply -f -
143+
apiVersion: v1
144+
kind: Secret
145+
metadata:
146+
name: vsphere-secret
147+
namespace: my-cluster
148+
labels:
149+
cluster.x-k8s.io/cluster-name: "my-cluster"
150+
stringData:
151+
username: "administrator@vsphere.local"
152+
password: "YOUR_PASSWORD"
153+
EOF
154+
```
152155

153-
vSphereCloudControllerManager:
154-
secret:
155-
create: false
156-
name: vsphere-config-secret
157-
158-
vSphereStorageControllerManager:
159-
secret:
160-
create: false
161-
name: csi-config-secret
156+
```yaml
157+
# Create the vsphere-config-secret for Cloud Controller Manager
158+
cat <<EOF | kubectl apply -f -
159+
apiVersion: v1
160+
kind: Secret
161+
metadata:
162+
name: vsphere-config-secret
163+
namespace: my-cluster
164+
labels:
165+
cluster.x-k8s.io/cluster-name: "my-cluster"
166+
stringData:
167+
vsphere.conf: |
168+
global:
169+
port: 443
170+
insecure-flag: false
171+
password: "YOUR_PASSWORD"
172+
user: "administrator@vsphere.local"
173+
thumbprint: "YOUR_VCENTER_THUMBPRINT"
174+
vcenter:
175+
vcenter.example.com:
176+
datacenters:
177+
- "YOUR_DATACENTER"
178+
server: "vcenter.example.com"
179+
EOF
162180
```
163181

164-
### Create Cluster API Secret
182+
```yaml
183+
# Create the csi-config-secret for Storage Controller
184+
cat <<EOF | kubectl apply -f -
185+
apiVersion: v1
186+
kind: Secret
187+
metadata:
188+
name: csi-config-secret
189+
namespace: my-cluster
190+
labels:
191+
cluster.x-k8s.io/cluster-name: "my-cluster"
192+
stringData:
193+
csi-vsphere.conf: |
194+
[Global]
195+
cluster-id = "namespace/my-cluster"
196+
thumbprint = "YOUR_VCENTER_THUMBPRINT"
197+
insecure-flag = false
198+
[VirtualCenter "vcenter.example.com"]
199+
user = "administrator@vsphere.local"
200+
password = "YOUR_PASSWORD"
201+
datacenters = "YOUR_DATACENTER"
202+
EOF
203+
```
165204

166-
```bash
205+
### Credentials through VSphereClusterIdentity
206+
The chart can also be configured to use `VSphereClusterIdentity` for managing vSphere credentials. This allows multiple clusters to share the same credentials.
207+
208+
Deploy a secret with the credentials in the CAPV manager namespace (capv-system by default):
209+
210+
```yaml
167211
# Create the vsphere-secret for Cluster API
168212
cat <<EOF | kubectl apply -f -
169213
apiVersion: v1
170214
kind: Secret
171215
metadata:
172216
name: vsphere-secret
217+
namespace: capv-system
173218
labels:
174219
cluster.x-k8s.io/cluster-name: "my-cluster"
175220
stringData:
@@ -178,15 +223,32 @@ stringData:
178223
EOF
179224
```
180225

181-
### Create Cloud Controller Manager Secret
226+
Deploy a `VSphereClusterIdentity` that references the secret above. The `allowedNamespaces` selector can also be used to control which namespaces are allowed to use the identity:
182227

183-
```bash
228+
```yaml
229+
# Create the VSphereClusterIdentity
230+
cat <<EOF | kubectl apply -f -
231+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
232+
kind: VSphereClusterIdentity
233+
metadata:
234+
name: vsphere-cluster-identity
235+
spec:
236+
secretName: vsphere-secret
237+
allowedNamespaces:
238+
selector:
239+
matchLabels: {} # allow all namespaces
240+
```
241+
242+
> **Note**: The CSI secret and the Cloud Controller Manager secret must still be created separately.
243+
244+
```yaml
184245
# Create the vsphere-config-secret for Cloud Controller Manager
185246
cat <<EOF | kubectl apply -f -
186247
apiVersion: v1
187248
kind: Secret
188249
metadata:
189250
name: vsphere-config-secret
251+
namespace: my-cluster
190252
labels:
191253
cluster.x-k8s.io/cluster-name: "my-cluster"
192254
stringData:
@@ -205,15 +267,14 @@ stringData:
205267
EOF
206268
```
207269

208-
### Create CSI Controller Secret
209-
210-
```bash
270+
```yaml
211271
# Create the csi-config-secret for Storage Controller
212272
cat <<EOF | kubectl apply -f -
213273
apiVersion: v1
214274
kind: Secret
215275
metadata:
216276
name: csi-config-secret
277+
namespace: my-cluster
217278
labels:
218279
cluster.x-k8s.io/cluster-name: "my-cluster"
219280
stringData:
@@ -314,88 +375,7 @@ kubectl logs -l component=csi-controller-manager
314375

315376
## Configuration
316377

317-
Here the values you can override:
318-
319-
## Values
320-
321-
| Key | Type | Default | Description |
322-
|-----|------|---------|-------------|
323-
| cluster.controlPlane.addons.coreDNS | object | `{}` | KamajiControlPlane coreDNS configuration |
324-
| cluster.controlPlane.addons.konnectivity | object | `{}` | KamajiControlPlane konnectivity configuration |
325-
| cluster.controlPlane.addons.kubeProxy | object | `{}` | KamajiControlPlane kube-proxy configuration |
326-
| cluster.controlPlane.apiServer | object | `{"extraArgs":["--cloud-provider=external"]}` | extraArgs for the control plane components |
327-
| cluster.controlPlane.controllerManager.extraArgs[0] | string | `"--cloud-provider=external"` | |
328-
| cluster.controlPlane.dataStoreName | string | `"default"` | KamajiControlPlane dataStoreName |
329-
| cluster.controlPlane.kubelet.cgroupfs | string | `"systemd"` | kubelet cgroupfs configuration |
330-
| cluster.controlPlane.kubelet.preferredAddressTypes | list | `["InternalIP","ExternalIP","Hostname"]` | kubelet preferredAddressTypes order |
331-
| cluster.controlPlane.labels | object | `{"cni":"calico"}` | Labels to add to the control plane |
332-
| cluster.controlPlane.network.certSANs | list | `[]` | List of additional Subject Alternative Names to use for the API Server serving certificate |
333-
| cluster.controlPlane.network.serviceAddress | string | `""` | Address used to expose the Kubernetes API server. If not set, the service will be exposed on the first available address. |
334-
| cluster.controlPlane.network.serviceAnnotations | object | `{}` | Annotations to use for the control plane service |
335-
| cluster.controlPlane.network.serviceLabels | object | `{}` | Labels to use for the control plane service |
336-
| cluster.controlPlane.network.serviceType | string | `"LoadBalancer"` | Type of service used to expose the Kubernetes API server |
337-
| cluster.controlPlane.replicas | int | `2` | Number of control plane replicas |
338-
| cluster.controlPlane.version | string | `"v1.31.0"` | Kubernetes version |
339-
| cluster.metrics.enabled | bool | `false` | Enable metrics collection. ServiceMonitor custom resource definition must be installed on the Management cluster. |
340-
| cluster.metrics.serviceAccount | object | `{"name":"kube-prometheus-stack-prometheus","namespace":"monitoring-system"}` | ServiceAccount for scraping metrics |
341-
| cluster.metrics.serviceAccount.name | string | `"kube-prometheus-stack-prometheus"` | ServiceAccount name used for scraping metrics |
342-
| cluster.metrics.serviceAccount.namespace | string | `"monitoring-system"` | ServiceAccount namespace |
343-
| cluster.name | string | `""` | Cluster name. If unset, the release name will be used |
344-
| ipamProvider.enabled | bool | `true` | Enable the IPAMProvider usage |
345-
| ipamProvider.gateway | string | `"192.168.0.1"` | IPAMProvider gateway |
346-
| ipamProvider.prefix | string | `"24"` | IPAMProvider prefix |
347-
| ipamProvider.ranges | list | `["192.168.0.0/24"]` | IPAMProvider ranges |
348-
| nodePools[0].addressesFromPools | object | `{"enabled":true}` | Use an IPAMProvider pool to reserve IPs |
349-
| nodePools[0].addressesFromPools.enabled | bool | `true` | Enable the IPAMProvider usage |
350-
| nodePools[0].autoscaling.enabled | bool | `false` | Enable autoscaling |
351-
| nodePools[0].autoscaling.labels.autoscaling | string | `"enabled"` | Labels to use for autoscaling: make sure to use the same labels on the autoscaler configuration |
352-
| nodePools[0].autoscaling.maxSize | string | `"6"` | Maximum number of instances in the pool |
353-
| nodePools[0].autoscaling.minSize | string | `"2"` | Minimum number of instances in the pool |
354-
| nodePools[0].dataStore | string | `"datastore"` | VSphere datastore to use |
355-
| nodePools[0].dhcp4 | bool | `false` | Use dhcp for ipv4 configuration |
356-
| nodePools[0].diskGiB | int | `40` | Disk size of VM in GiB |
357-
| nodePools[0].folder | string | `"default-pool"` | VSphere folder to store VMs |
358-
| nodePools[0].memoryMiB | int | `4096` | Memory to allocate to worker VMs |
359-
| nodePools[0].name | string | `"default"` | |
360-
| nodePools[0].nameServers | list | `["8.8.8.8"]` | Nameservers for VMs DNS resolution if required |
361-
| nodePools[0].network | string | `"network"` | VSphere network for VMs and CSI |
362-
| nodePools[0].numCPUs | int | `2` | Number of vCPUs to allocate to worker instances |
363-
| nodePools[0].replicas | int | `3` | Number of worker VMs instances |
364-
| nodePools[0].resourcePool | string | `"*/Resources"` | VSphere resource pool to use |
365-
| nodePools[0].staticRoutes | list | `[]` | Static network routes for VMs if required |
366-
| nodePools[0].storagePolicyName | string | `""` | VSphere storage policy to use |
367-
| nodePools[0].template | string | `"ubuntu-2204-kube-v1.31.0"` | VSphere template to clone |
368-
| nodePools[0].users | list | `[{"name":"ubuntu","sshAuthorizedKeys":[],"sudo":"ALL=(ALL) NOPASSWD:ALL"}]` | Search domains suffixes if required searchDomains: [] # -- VM network domain if required domain: "" # -- IPv4 gateway if required gateway: "" # -- users to create on machines |
369-
| vSphere.dataCenter | string | `"datacenter"` | Datacenter to use |
370-
| vSphere.insecure | bool | `false` | If vCenter uses a self-signed cert |
371-
| vSphere.password | string | `"changeme"` | vSphere password |
372-
| vSphere.port | int | `443` | VSphere server port |
373-
| vSphere.secret | object | `{"create":false,"name":"vsphere-secret"}` | Create a secret with the VSphere credentials |
374-
| vSphere.secret.create | bool | `false` | Specifies whether Secret should be created from config values |
375-
| vSphere.secret.name | string | `"vsphere-secret"` | The name of an existing Secret for vSphere. |
376-
| vSphere.server | string | `"server.sample.org"` | VSphere server dns name or address |
377-
| vSphere.tlsThumbprint | string | `""` | VSphere https TLS thumbprint |
378-
| vSphere.username | string | `"admin@vcenter"` | vSphere username |
379-
| vSphereCloudControllerManager.enabled | bool | `true` | Installs vsphere-cloud-controller-manager on the management cluster |
380-
| vSphereCloudControllerManager.password | string | `"changeme"` | vSphere password |
381-
| vSphereCloudControllerManager.secret.create | bool | `false` | Specifies whether Secret should be created from config values |
382-
| vSphereCloudControllerManager.secret.name | string | `"vsphere-config-secret"` | The name of an existing Secret for vSphere. |
383-
| vSphereCloudControllerManager.username | string | `"admin@vcenter"` | vSphere username |
384-
| vSphereCloudControllerManager.version | string | `"v1.31.0"` | Version of the vsphere-cloud-controller-manager to install. The major and minor versions of releases should be equivalent to the compatible upstream Kubernetes release. |
385-
| vSphereStorageControllerManager.enabled | bool | `false` | Installs vsphere-storage-controller-manager on the management cluster. NB: CSI node drivers are always installed on the workload cluster. |
386-
| vSphereStorageControllerManager.logLevel | string | `"PRODUCTION"` | log level for the CSI components |
387-
| vSphereStorageControllerManager.namespace | string | `"kube-system"` | Target namespace for the vSphere CSI node drivers on the workload cluster |
388-
| vSphereStorageControllerManager.password | string | `"changeme"` | vSphere CSI password |
389-
| vSphereStorageControllerManager.secret.create | bool | `false` | Specifies whether Secret should be created from config values |
390-
| vSphereStorageControllerManager.secret.name | string | `"csi-config-secret"` | The name of an existing Secret for vSphere. |
391-
| vSphereStorageControllerManager.storageClass.allowVolumeExpansion | bool | `true` | Allow volume expansion |
392-
| vSphereStorageControllerManager.storageClass.default | bool | `true` | Configure as the default storage class |
393-
| vSphereStorageControllerManager.storageClass.enabled | bool | `false` | StorageClass enablement |
394-
| vSphereStorageControllerManager.storageClass.name | string | `"vsphere-csi"` | Name of the storage class |
395-
| vSphereStorageControllerManager.storageClass.parameters | object | `{}` | Optional storage class parameters |
396-
| vSphereStorageControllerManager.storageClass.reclaimPolicy | string | `"Delete"` | Reclaim policy |
397-
| vSphereStorageControllerManager.storageClass.volumeBindingMode | string | `"WaitForFirstConsumer"` | Volume binding mode |
398-
| vSphereStorageControllerManager.username | string | `"admin@vcenter"` | vSphere CSI username |
378+
See the values you can override [here](charts/capi-kamaji-vsphere/README.md).
399379

400380
## Maintainers
401381

0 commit comments

Comments
 (0)