Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Change status.conditions to the Kubernetes format & adjust the reconciliation logic #563

Closed
wants to merge 15 commits into from
Closed

[WIP] Change status.conditions to the Kubernetes format & adjust the reconciliation logic #563

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
8f49fdb
adjust function controller
tpiperatgod Jan 28, 2023
8dde122
adjust sink\source controller
tpiperatgod Jan 29, 2023
2cadb59
adjust mesh controller && fix function reconciliation logic
tpiperatgod Jan 29, 2023
36558b7
use status.componentHash to store component specification hash to det…
tpiperatgod Jan 29, 2023
cafc5d8
adjust sink controller
tpiperatgod Jan 29, 2023
83ea49e
adjust source controller
tpiperatgod Jan 29, 2023
0611258
fix mesh controller
tpiperatgod Jan 30, 2023
6fccbd1
update chart
tpiperatgod Jan 30, 2023
73445e6
fix ut
tpiperatgod Jan 30, 2023
12a3531
fix
tpiperatgod Jan 30, 2023
c491143
fix codacy check
tpiperatgod Jan 30, 2023
9d46275
adjust ci, change maxReplicas\replicas to minReplicas
tpiperatgod Jan 30, 2023
b5de2cb
add v1alpha2 api
tpiperatgod Feb 7, 2023
50e9b1d
add conversion
tpiperatgod Feb 21, 2023
9587f80
adjust controller
tpiperatgod Feb 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 4 additions & 7 deletions .ci/clusters/compute_v1alpha1_functionmesh.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
apiVersion: compute.functionmesh.io/v1alpha1
apiVersion: compute.functionmesh.io/v1alpha2
kind: FunctionMesh
metadata:
name: functionmesh-sample
Expand All @@ -7,8 +7,7 @@ spec:
- name: java-function
image: streamnative/pulsar-functions-java-sample:2.9.2.23
className: org.apache.pulsar.functions.api.examples.ExclamationFunction
replicas: 1
maxReplicas: 1
minReplicas: 1
input:
topics:
- persistent://public/default/functionmesh-input-topic
Expand All @@ -32,8 +31,7 @@ spec:
clusterName: test
- name: golang-function
image: streamnative/pulsar-functions-go-sample:2.9.2.23
replicas: 1
maxReplicas: 1
minReplicas: 1
input:
topics:
- persistent://public/default/functionmesh-java-topic
Expand All @@ -58,8 +56,7 @@ spec:
- name: python-function
image: streamnative/pulsar-functions-python-sample:2.9.2.23
className: exclamation_function.ExclamationFunction
replicas: 1
maxReplicas: 1
minReplicas: 1
input:
topics:
- persistent://public/default/functionmesh-golang-topic
Expand Down
2 changes: 1 addition & 1 deletion .ci/tests/integration-oauth2/cases/java-download-function/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ spec:
className: org.apache.pulsar.functions.api.examples.ExclamationFunction
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
minReplicas: 1
logTopic: persistent://public/default/logging-function-logs
input:
topics:
Expand Down
2 changes: 1 addition & 1 deletion .ci/tests/integration-oauth2/cases/java-function/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ spec:
className: org.apache.pulsar.functions.api.examples.ExclamationFunction
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
minReplicas: 1
logTopic: persistent://public/default/logging-function-logs
input:
topics:
Expand Down
3 changes: 1 addition & 2 deletions .ci/tests/integration-oauth2/cases/py-download-from-http-function/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,7 @@ spec:
className: exclamation
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
maxReplicas: 1
minReplicas: 1
logTopic: persistent://public/default/py-function-logs
input:
topics:
Expand Down
3 changes: 1 addition & 2 deletions .ci/tests/integration-oauth2/cases/py-download-function-legacy/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,7 @@ spec:
className: exclamation_function.ExclamationFunction
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
maxReplicas: 1
minReplicas: 1
logTopic: persistent://public/default/py-function-legacy-logs
input:
topics:
Expand Down
3 changes: 1 addition & 2 deletions .ci/tests/integration-oauth2/cases/py-download-function/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,7 @@ spec:
className: exclamation_function.ExclamationFunction
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
maxReplicas: 1
minReplicas: 1
logTopic: persistent://public/default/py-function-logs
input:
topics:
Expand Down
3 changes: 1 addition & 2 deletions .ci/tests/integration/cases/crypto-function/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ spec:
className: org.apache.pulsar.functions.api.examples.ExclamationFunction
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
maxReplicas: 5
minReplicas: 1
logTopic: persistent://public/default/logging-function-logs
input:
topics:
Expand Down
3 changes: 1 addition & 2 deletions .ci/tests/integration/cases/elasticsearch-sink/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,7 @@ metadata:
name: sink-sample
spec:
className: org.apache.pulsar.io.elasticsearch.ElasticSearchSink
replicas: 1
maxReplicas: 1
minReplicas: 1
input:
topics:
- persistent://public/default/input-sink-topic
Expand Down
9 changes: 3 additions & 6 deletions .ci/tests/integration/cases/functionmesh/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,7 @@ spec:
- name: java-function
image: streamnative/pulsar-functions-java-sample:2.9.2.23
className: org.apache.pulsar.functions.api.examples.ExclamationFunction
replicas: 1
maxReplicas: 1
minReplicas: 1
input:
topics:
- persistent://public/default/functionmesh-input-topic
Expand All @@ -32,8 +31,7 @@ spec:
clusterName: test
- name: golang-function
image: streamnative/pulsar-functions-go-sample:2.9.2.23
replicas: 1
maxReplicas: 1
minReplicas: 1
input:
topics:
- persistent://public/default/functionmesh-java-topic
Expand All @@ -58,8 +56,7 @@ spec:
- name: python-function
image: streamnative/pulsar-functions-python-sample:2.9.2.23
className: exclamation_function.ExclamationFunction
replicas: 1
maxReplicas: 1
minReplicas: 1
input:
topics:
- persistent://public/default/functionmesh-golang-topic
Expand Down
3 changes: 1 addition & 2 deletions .ci/tests/integration/cases/go-download-function/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,7 @@ metadata:
spec:
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
maxReplicas: 3
minReplicas: 1
logTopic: persistent://public/default/go-function-logs
input:
topics:
Expand Down
3 changes: 1 addition & 2 deletions .ci/tests/integration/cases/go-function/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,7 @@ spec:
image: streamnative/pulsar-functions-go-sample:2.9.2.23
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
maxReplicas: 1
minReplicas: 1
logTopic: persistent://public/default/go-function-logs
input:
topics:
Expand Down
3 changes: 1 addition & 2 deletions .ci/tests/integration/cases/health-check/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ spec:
className: org.apache.pulsar.functions.api.examples.ExclamationFunction
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
maxReplicas: 5
minReplicas: 1
liveness:
initialDelaySeconds: 10
periodSeconds: 10
Expand Down
3 changes: 1 addition & 2 deletions .ci/tests/integration/cases/java-download-function/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,7 @@ spec:
className: org.apache.pulsar.functions.api.examples.ExclamationFunction
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
maxReplicas: 5
minReplicas: 1
logTopic: persistent://public/default/logging-function-logs
input:
topics:
Expand Down
2 changes: 1 addition & 1 deletion .ci/tests/integration/cases/java-function-vpa/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ spec:
className: org.apache.pulsar.functions.api.examples.ExclamationFunction
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 2
minReplicas: 2
pod:
vpa:
updatePolicy:
Expand Down
3 changes: 1 addition & 2 deletions .ci/tests/integration/cases/java-function/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ spec:
className: org.apache.pulsar.functions.api.examples.ExclamationFunction
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
maxReplicas: 5
minReplicas: 1
logTopic: persistent://public/default/logging-function-logs
input:
topics:
Expand Down
3 changes: 1 addition & 2 deletions .ci/tests/integration/cases/java-log-config/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ spec:
className: org.apache.pulsar.functions.api.examples.ExclamationFunction
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
maxReplicas: 5
minReplicas: 1
logTopic: persistent://public/default/logging-function-logs
input:
topics:
Expand Down
3 changes: 1 addition & 2 deletions .ci/tests/integration/cases/java-log-level/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ spec:
className: org.apache.pulsar.functions.api.examples.ExclamationFunction
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
maxReplicas: 5
minReplicas: 1
logTopic: persistent://public/default/logging-function-logs
input:
topics:
Expand Down
2 changes: 1 addition & 1 deletion .ci/tests/integration/cases/logging-window-function/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ spec:
className: org.apache.pulsar.functions.api.examples.window.LoggingWindowFunction
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
minReplicas: 1
logTopic: persistent://public/default/window-function-logs
input:
topics:
Expand Down
3 changes: 1 addition & 2 deletions .ci/tests/integration/cases/mongodb-source/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,7 @@ metadata:
name: source-sample
spec:
className: org.apache.pulsar.io.debezium.mongodb.DebeziumMongoDbSource
replicas: 1
maxReplicas: 1
minReplicas: 1
output:
producerConf:
maxPendingMessages: 1000
Expand Down
3 changes: 1 addition & 2 deletions .ci/tests/integration/cases/py-download-function/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,7 @@ spec:
className: exclamation_function.ExclamationFunction
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
maxReplicas: 1
minReplicas: 1
logTopic: persistent://public/default/py-function-logs
input:
topics:
Expand Down
3 changes: 1 addition & 2 deletions .ci/tests/integration/cases/py-download-pip-function/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,7 @@ spec:
className: pulsarfunction.exclamation.ExclamationFunction
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
maxReplicas: 1
minReplicas: 1
logTopic: persistent://public/default/py-function-logs
input:
topics:
Expand Down
3 changes: 1 addition & 2 deletions .ci/tests/integration/cases/py-download-zip-function/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,7 @@ spec:
className: exclamation
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
maxReplicas: 1
minReplicas: 1
logTopic: persistent://public/default/py-function-logs
input:
topics:
Expand Down
3 changes: 1 addition & 2 deletions .ci/tests/integration/cases/python-function/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ spec:
className: exclamation_function.ExclamationFunction
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
maxReplicas: 1
minReplicas: 1
logTopic: persistent://public/default/py-function-logs
input:
topics:
Expand Down
3 changes: 1 addition & 2 deletions .ci/tests/integration/cases/python-log-config/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ spec:
className: exclamation_function.ExclamationFunction
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
maxReplicas: 1
minReplicas: 1
logTopic: persistent://public/default/py-function-logs
input:
topics:
Expand Down
3 changes: 1 addition & 2 deletions .ci/tests/integration/cases/python-log-level/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ spec:
className: exclamation_function.ExclamationFunction
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
maxReplicas: 1
minReplicas: 1
logTopic: persistent://public/default/py-function-logs
input:
topics:
Expand Down
2 changes: 1 addition & 1 deletion .ci/tests/integration/cases/python-stateful-function/manifests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ spec:
className: wordcount_function.WordCountFunction
forwardSourceMessageProperty: true
maxPendingAsyncRequests: 1000
replicas: 1
minReplicas: 1
logTopic: persistent://public/default/logging-stateful-function-logs
input:
topics:
Expand Down
9 changes: 3 additions & 6 deletions .ci/tests/integration/cases/reconciliation/manifests-one.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,7 @@ spec:
- name: sink-one
image: streamnative/pulsar-io-elastic-search:2.9.2.23
className: org.apache.pulsar.io.elasticsearch.ElasticSearchSink
replicas: 1
maxReplicas: 1
minReplicas: 1
input:
topics:
- persistent://public/default/elasticsearch-sink
Expand Down Expand Up @@ -37,8 +36,7 @@ spec:
- name: function-one
image: streamnative/pulsar-functions-java-sample:2.9.2.23
className: org.apache.pulsar.functions.api.examples.ExclamationFunction
replicas: 1
maxReplicas: 1
minReplicas: 1
input:
topics:
- persistent://public/default/functionmesh-input-topic
Expand All @@ -63,8 +61,7 @@ spec:
sources:
- name: source-one
className: org.apache.pulsar.io.debezium.mongodb.DebeziumMongoDbSource
replicas: 1
maxReplicas: 1
minReplicas: 1
output:
producerConf:
maxPendingMessages: 1000
Expand Down
37 changes: 34 additions & 3 deletions PROJECT
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
domain: functionmesh.io
layout:
- go.kubebuilder.io/v3
plugins:
go.sdk.operatorframework.io/v2-alpha: {}
projectName: function-mesh
repo: github.com/streamnative/function-mesh
resources:
Expand Down Expand Up @@ -40,7 +42,36 @@ resources:
kind: Sink
path: github.com/streamnative/function-mesh/api/compute/v1alpha1
version: v1alpha1
- api:
crdVersion: v1
namespaced: true
domain: functionmesh.io
group: compute
kind: Function
path: github.com/streamnative/function-mesh/api/compute/v1alpha2
version: v1alpha2
- api:
crdVersion: v1
namespaced: true
domain: functionmesh.io
group: compute
kind: Source
path: github.com/streamnative/function-mesh/api/compute/v1alpha2
version: v1alpha2
- api:
crdVersion: v1
namespaced: true
domain: functionmesh.io
group: compute
kind: Sink
path: github.com/streamnative/function-mesh/api/compute/v1alpha2
version: v1alpha2
- api:
crdVersion: v1
namespaced: true
domain: functionmesh.io
group: compute
kind: FunctionMesh
path: github.com/streamnative/function-mesh/api/compute/v1alpha2
version: v1alpha2
version: "3"

plugins:
go.sdk.operatorframework.io/v2-alpha: {}
Loading