Strimzi
Kafka Broker Pod Schedule Issue #11117
-
Preliminary DescriptionWe have set up a Kafka cluster based on a Kubernetes (k8s) cluster and Strimzi, and we have a specific constraint: the disks on the Node nodes in the k8s cluster are bound to the Nodes and are provided to the Kafka brokers for use as data disks. ExpectationWhen Strimzi schedules the brokers in the Kafka cluster to restart, we expect that the Node to which a broker pod is attached should remain unchanged before and after the restart. Current SituationWhen Strimzi schedules the brokers in the Kafka cluster to restart, the Node to which a broker pod is attached may change after the restart. A possible approach is to add different nodeAffinity to each broker pod's manifest, so each broker pod can bind to a specific node. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 3 replies
-
|
Did you take a look at the documentation? You should have the info you need here https://strimzi.io/docs/operators/latest/deploying#assembly-scheduling-str |
Beta Was this translation helpful? Give feedback.
-
|
Yes, I have checked it before post this issue My understanding is that Strimzi can only limit a batch of Broker Pods (organized by KafkaNodePool) to be scheduled onto a batch of Nodes with the same characteristics (such as having the same labels, etc.). It does not support restricting a single Broker Pod to be bound to a single Node. I mainly want to confirm if Strimzi actually has the feature to restrict a single Broker Pod to be bound to a single Node, and it is my understanding that is incorrect. |
Beta Was this translation helpful? Give feedback.
-
|
First of all - Strimzi does not do any scheduling. Scheduling of the Pods is done by Kubernetes. Strimzi only gives you a way how to set things such as affinity, tolerations, or taints for the Pods which affect how Kubernetes will schedule them. If you use local storage, then your storage provisioner has to mark that properly on the PVs. And then Kubernetes will use this information together with the information from the Pods to schedule the Pods. If your Pods using local storage gert scheduled on wrong nodes, it most likely means your local PVs are not configured properly. |
Beta Was this translation helpful? Give feedback.
-
|
Oh i see. Strimzi relies entirely on Kubernetes' scheduling mechanism and does not provide additional scheduling-related features. For the case mentioned above, i can use Kubernetes' native support for specifying nodeAffinity on persistent volumes (K8s PV Spec) to achieve the feature of "constraining the scheduling of a pod to a specific node(bind with the PV)". |
Beta Was this translation helpful? Give feedback.
First of all - Strimzi does not do any scheduling. Scheduling of the Pods is done by Kubernetes. Strimzi only gives you a way how to set things such as affinity, tolerations, or taints for the Pods which affect how Kubernetes will schedule them.
If you use local storage, then your storage provisioner has to mark that properly on the PVs. And then Kubernetes will use this information together with the information from the Pods to schedule the Pods. If your Pods using local storage gert scheduled on wrong nodes, it most likely means your local PVs are not configured properly.