Skip to content

Commit 81ecc4e

Browse files
qiujian16qiujian16
committed
New Update Strategy only when workload changes in ManifestWork
Signed-off-by: Jian Qiu <jqiu@redhat.com>
1 parent 964321d commit 81ecc4e

File tree

2 files changed

+197
-0
lines changed

2 files changed

+197
-0
lines changed

‎enhancements/sig-architecture/132-work-update-only-when-spec-change/README.md

+183
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
# New Update Strategy only when workload changes in ManifestWork
2+
3+
## Release Signoff Checklist
4+
5+
- [] Enhancement is `provisional`
6+
- [] Design details are appropriately documented from clear requirements
7+
- [] Test plan is defined
8+
- [] Graduation criteria for dev preview, tech preview, GA
9+
- [] User-facing documentation is created in [website](https://github.com/open-cluster-management-io/open-cluster-management-io.github.io/)
10+
11+
## Summary
12+
This proposal is to introduce new options in update stratgey of manifestwork to resolve issues
13+
on how to coordinate other actors and work agent when they apply the same resource on the spoke
14+
cluster.
15+
16+
## Motivation
17+
We have introduced several update strategy in `ManifestWork` to resolve resource conflict before:
18+
- `CreateOnly` let work-agent to create the resource only and ignore any further change on the
19+
resource
20+
- `ServerSideApply` utilize the server side apply mechanism in kube-apiserver, so if work-agent
21+
and another actor try to operator the same field in a resource, a conflict can be identified.
22+
23+
However, some issues are raised indicating the above strategy can still not meet all the requirements
24+
- [issue 631](https://github.com/open-cluster-management-io/ocm/issues/631) reports a case that user
25+
uses manifestwork to create a configmap, and then another actor on the spoke change the configmap. The
26+
user does not want the configmap to be changed back by the work-agent. However, when the configmap resource
27+
in the manifestwork is updated, the user wants the configmap to be updated accordingly.
28+
- [issue 670](https://github.com/open-cluster-management-io/ocm/issues/670) is a case when the user
29+
uses manifestwork to create argocd's `Application` resource. The `Application` resource has an `operation`
30+
field which is used to trigger the sync, and the field will be removed by the argocd when the sync is done.
31+
User in the manifestwork sets the `operation` field and when argocd removes the field, the user does not want
32+
the field to be updated back by the work-agent.
33+
- [issue 690](https://github.com/open-cluster-management-io/ocm/issues/690) is the case that user wants
34+
to create a deployment using manifestwork, but what HPA on the spoke the control the replicas of the deployment.
35+
Hence the user wants the work-agent to ignore the replicas field of the deployment in the manfiestwor if it is
36+
set.
37+
38+
39+
## Proposal
40+
41+
We would like to introduce new options in `Update` and `ServerSideApply` update strategy to resolve the above issue.
42+
So user can set the option in the manifestwork, so when another actor on the spoke cluster update the resource. The
43+
work-agent will ignore the change and not try to change the resource back. On the other hand, when the resource spec
44+
defined in the manifestwork is updated by the user, the work-agent will still update the resource accordingly. In
45+
summary, the option is to ignore the change triggered from spoke cluster only.
46+
47+
In addition, in `ServerSideApplly` strategy, we would also introduce a new `ignoreDifferences` options similar as what
48+
is defined in argoCD (https://argo-cd.readthedocs.io/en/stable/user-guide/diffing/). Such that user can choose to not
49+
let work-agent patch certain resource fields.
50+
51+
### Design Details
52+
53+
#### API change
54+
55+
The change will be added into the `updateStratey` field. For `Update` strategy, we will add an option
56+
57+
```go
58+
type UpdateConfig struct {
59+
// OnSpokeChange defines whether the resource should be overriden by the manifestwork it is changed
60+
// on the spoke by another actor.
61+
// +kubebuilder:default=Override
62+
// +kubebuilder:validation:Enum=Override;NoOverride
63+
// +kubebuilder:validation:Required
64+
// +required
65+
OnSpokeChange string `json:"onSpokeChange,omitempty"`
66+
}
67+
```
68+
69+
For existing `ServerSideApply` strategy, we will add:
70+
71+
```go
72+
type ServerSideApplyConfig struct {
73+
...
74+
// IgnoreFields defines a list of json paths in the resource that will not be updated on the spoke.
75+
IgnoreFields []string `json:"ignoreFields,omitempty"`
76+
77+
// OnSpokeChange defines whether the resource should be overriden by the manifestwork it is changed
78+
// on the spoke by another actor.
79+
// +kubebuilder:default=Override
80+
// +kubebuilder:validation:Enum=Override;NoOverride
81+
// +kubebuilder:validation:Required
82+
// +required
83+
OnSpokeChange string `json:"onSpokeChange,omitempty"`
84+
}
85+
```
86+
87+
#### agent implemntation
88+
89+
When work-agent identity that the `OnSpokeChange` is set `NoOverride` for a certain resource in the `ManifestWork`, the agent
90+
when apply the resource to the spoke cluster will add an annotation with the key `open-cluster-management.io/object-hash`.
91+
The value of the annotation is the computed hash of the resource spec in the `ManifestWork`. Later when another actor
92+
updates the resource, the work-agent will at first check if the object-hash mismatches with the current resource spec
93+
in the `ManifestWork`. If it is the same, the resource will not be updated so the change from spoke is ignored. When
94+
the resource in the manifestwork is updated, the annotation will not match which then trigger the work-agent to update.
95+
96+
To handle the `IgnoreFields` for `ServerSideApply`, we will remove the fields defined in the `IgnoreFields` and then
97+
generates the apply patch. The objec-hash will also be computed considering the `IgnoreFields`.
98+
99+
#### examples
100+
101+
To resolve issue 642, user can set the manifestwork with `OnSpokeChange` set to `NoOverride`
102+
103+
```yaml
104+
apiVersion: work.open-cluster-management.io/v1
105+
kind: ManifestWork
106+
metadata:
107+
namespace: <target managed cluster>
108+
name: hello-work-demo
109+
spec:
110+
workload: ...
111+
manifestConfigs:
112+
- resourceIdentifier:
113+
resource: secrets
114+
namespace: default
115+
name: some-secret
116+
updateStrategy:
117+
type: Update
118+
update:
119+
onSpokeChange: NoOverride
120+
```
121+
122+
To resolve issue 670, user also can do the same for argocd application.
123+
124+
```yaml
125+
apiVersion: work.open-cluster-management.io/v1
126+
kind: ManifestWork
127+
metadata:
128+
namespace: <target managed cluster>
129+
name: hello-work-demo
130+
spec:
131+
workload: ...
132+
manifestConfigs:
133+
- resourceIdentifier:
134+
group: argoproj.io/v1alpha1
135+
resource: application
136+
namespace: default
137+
name: application1
138+
updateStrategy:
139+
type: Update
140+
update:
141+
onSpokeChange: NoOverride
142+
```
143+
144+
To resolve issue 690, user can set like:
145+
146+
```yaml
147+
apiVersion: work.open-cluster-management.io/v1
148+
kind: ManifestWork
149+
metadata:
150+
namespace: <target managed cluster>
151+
name: hello-work-demo
152+
spec:
153+
workload: ...
154+
manifestConfigs:
155+
- resourceIdentifier:
156+
group: apps/v1
157+
resource: deployment
158+
namespace: default
159+
name: deploy1
160+
updateStrategy:
161+
type: ServerSideApply
162+
serverSideApply:
163+
ignoreFields:
164+
- ".spec.replicas"
165+
```
166+
167+
168+
### Test Plan
169+
170+
- test on `OnSpokeChange` with `Overrid` and `NonOverride` option
171+
- test on `IgnoreFields` with a single field, a full strcutrue and an item in the list.
172+
173+
### Graduation Criteria
174+
N/A
175+
176+
### Upgrade Strategy
177+
It will need upgrade on CRD of ManifestWork on hub cluster, and upgrade of work agent on managed cluster.
178+
179+
### Version Skew Strategy
180+
- The field is optional, and if it is not set, the manifestwork will be updated as is by the work agent with elder version
181+
- The elder version work agent will ignore the newly added field field.
182+
183+
## Alternatives

‎enhancements/sig-architecture/132-work-update-only-when-spec-change/metadata.yaml

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
title: return-resource-status-in-manifestwork
2+
authors:
3+
- "@qiujian16"
4+
reviewers:
5+
- "@deads2k"
6+
- "@elgnay"
7+
- "@zhujian7"
8+
approvers:
9+
- "@elgnay"
10+
creation-date: 2024-11-12
11+
last-updated: 2021-11-12
12+
status: provisional
13+
see-also:
14+
- "/enhancements/sig-architecture/47-manifestwork-updatestrategy"

0 commit comments

Comments
 (0)