File tree 5 files changed +180
-5
lines changed
docs/hub/connectors/outbound
version-0.11.11/hub/connectors/outbound
version-0.11.12/hub/connectors/outbound
version-0.12.0/hub/connectors/outbound
version-0.13.0/hub/connectors/outbound
5 files changed +180
-5
lines changed Original file line number Diff line number Diff line change @@ -165,10 +165,45 @@ In this case, additional transformation will be performed before records are sen
165
165
166
166
Read more about [JSON to JSON transformations].
167
167
168
+ # ## Offset Management
169
+
170
+ Fluvio Consumer Offset feature allows for a connector to store the offset in the Fluvio cluster and use it on restart.
171
+ To activate it, you need to provide the `consumer` name and set the `strategy : auto`.
172
+ See the example below :
173
+
174
+ ` ` ` yaml
175
+ apiVersion: 0.1.0
176
+ meta:
177
+ version: x.y.z
178
+ name: my-http-sink
179
+ type: http-sink
180
+ topic: http-sink-topic
181
+ consumer:
182
+ id: my-http-sink
183
+ offset:
184
+ strategy: auto
185
+ start: beginning
186
+ flush-period:
187
+ secs: 10
188
+ nanos: 0
189
+ http:
190
+ endpoint: "http://127.0.0.1/post"
191
+ ` ` `
192
+
193
+ After the connector processed any records, you can check the last stored offset value via :
194
+ ` ` ` bash
195
+ $ fluvio consumer list
196
+ CONSUMER TOPIC PARTITION OFFSET LAST SEEN
197
+ my-http-sink http-sink-topic 0 0 3s
198
+ ` ` `
199
+
200
+ Read more about [Fluvio Offset].
201
+
168
202
# # Contributing
169
203
170
204
Follow on the conventional `CONTRIBUTING.md` file to setup your environment and
171
205
contribute to this project.
172
206
173
207
[Transformations] : /docs/latest/fluvio/concepts/transformations
174
- [JSON to JSON transformations] : /docs/latest/hub/smartmodules/jolt
208
+ [JSON to JSON transformations] : /docs/latest/hub/smartmodules/jolt
209
+ [Fluvio Offset] : /docs/latest/fluvio/concepts/offsets
Original file line number Diff line number Diff line change @@ -165,10 +165,45 @@ In this case, additional transformation will be performed before records are sen
165
165
166
166
Read more about [JSON to JSON transformations].
167
167
168
+ # ## Offset Management
169
+
170
+ Fluvio Consumer Offset feature allows for a connector to store the offset in the Fluvio cluster and use it on restart.
171
+ To activate it, you need to provide the `consumer` name and set the `strategy : auto`.
172
+ See the example below :
173
+
174
+ ` ` ` yaml
175
+ apiVersion: 0.1.0
176
+ meta:
177
+ version: x.y.z
178
+ name: my-http-sink
179
+ type: http-sink
180
+ topic: http-sink-topic
181
+ consumer:
182
+ id: my-http-sink
183
+ offset:
184
+ strategy: auto
185
+ start: beginning
186
+ flush-period:
187
+ secs: 10
188
+ nanos: 0
189
+ http:
190
+ endpoint: "http://127.0.0.1/post"
191
+ ` ` `
192
+
193
+ After the connector processed any records, you can check the last stored offset value via :
194
+ ` ` ` bash
195
+ $ fluvio consumer list
196
+ CONSUMER TOPIC PARTITION OFFSET LAST SEEN
197
+ my-http-sink http-sink-topic 0 0 3s
198
+ ` ` `
199
+
200
+ Read more about [Fluvio Offset].
201
+
168
202
# # Contributing
169
203
170
204
Follow on the conventional `CONTRIBUTING.md` file to setup your environment and
171
205
contribute to this project.
172
206
173
207
[Transformations] : /docs/latest/fluvio/concepts/transformations
174
- [JSON to JSON transformations] : /docs/latest/hub/smartmodules/jolt
208
+ [JSON to JSON transformations] : /docs/latest/hub/smartmodules/jolt
209
+ [Fluvio Offset] : /docs/latest/fluvio/concepts/offsets
Original file line number Diff line number Diff line change @@ -165,10 +165,45 @@ In this case, additional transformation will be performed before records are sen
165
165
166
166
Read more about [JSON to JSON transformations].
167
167
168
+ # ## Offset Management
169
+
170
+ Fluvio Consumer Offset feature allows for a connector to store the offset in the Fluvio cluster and use it on restart.
171
+ To activate it, you need to provide the `consumer` name and set the `strategy : auto`.
172
+ See the example below :
173
+
174
+ ` ` ` yaml
175
+ apiVersion: 0.1.0
176
+ meta:
177
+ version: x.y.z
178
+ name: my-http-sink
179
+ type: http-sink
180
+ topic: http-sink-topic
181
+ consumer:
182
+ id: my-http-sink
183
+ offset:
184
+ strategy: auto
185
+ start: beginning
186
+ flush-period:
187
+ secs: 10
188
+ nanos: 0
189
+ http:
190
+ endpoint: "http://127.0.0.1/post"
191
+ ` ` `
192
+
193
+ After the connector processed any records, you can check the last stored offset value via :
194
+ ` ` ` bash
195
+ $ fluvio consumer list
196
+ CONSUMER TOPIC PARTITION OFFSET LAST SEEN
197
+ my-http-sink http-sink-topic 0 0 3s
198
+ ` ` `
199
+
200
+ Read more about [Fluvio Offset].
201
+
168
202
# # Contributing
169
203
170
204
Follow on the conventional `CONTRIBUTING.md` file to setup your environment and
171
205
contribute to this project.
172
206
173
207
[Transformations] : /docs/latest/fluvio/concepts/transformations
174
- [JSON to JSON transformations] : /docs/latest/hub/smartmodules/jolt
208
+ [JSON to JSON transformations] : /docs/latest/hub/smartmodules/jolt
209
+ [Fluvio Offset] : /docs/latest/fluvio/concepts/offsets
Original file line number Diff line number Diff line change @@ -165,10 +165,45 @@ In this case, additional transformation will be performed before records are sen
165
165
166
166
Read more about [JSON to JSON transformations].
167
167
168
+ # ## Offset Management
169
+
170
+ Fluvio Consumer Offset feature allows for a connector to store the offset in the Fluvio cluster and use it on restart.
171
+ To activate it, you need to provide the `consumer` name and set the `strategy : auto`.
172
+ See the example below :
173
+
174
+ ` ` ` yaml
175
+ apiVersion: 0.1.0
176
+ meta:
177
+ version: x.y.z
178
+ name: my-http-sink
179
+ type: http-sink
180
+ topic: http-sink-topic
181
+ consumer:
182
+ id: my-http-sink
183
+ offset:
184
+ strategy: auto
185
+ start: beginning
186
+ flush-period:
187
+ secs: 10
188
+ nanos: 0
189
+ http:
190
+ endpoint: "http://127.0.0.1/post"
191
+ ` ` `
192
+
193
+ After the connector processed any records, you can check the last stored offset value via :
194
+ ` ` ` bash
195
+ $ fluvio consumer list
196
+ CONSUMER TOPIC PARTITION OFFSET LAST SEEN
197
+ my-http-sink http-sink-topic 0 0 3s
198
+ ` ` `
199
+
200
+ Read more about [Fluvio Offset].
201
+
168
202
# # Contributing
169
203
170
204
Follow on the conventional `CONTRIBUTING.md` file to setup your environment and
171
205
contribute to this project.
172
206
173
207
[Transformations] : /docs/latest/fluvio/concepts/transformations
174
- [JSON to JSON transformations] : /docs/latest/hub/smartmodules/jolt
208
+ [JSON to JSON transformations] : /docs/latest/hub/smartmodules/jolt
209
+ [Fluvio Offset] : /docs/latest/fluvio/concepts/offsets
Original file line number Diff line number Diff line change @@ -165,10 +165,45 @@ In this case, additional transformation will be performed before records are sen
165
165
166
166
Read more about [JSON to JSON transformations].
167
167
168
+ # ## Offset Management
169
+
170
+ Fluvio Consumer Offset feature allows for a connector to store the offset in the Fluvio cluster and use it on restart.
171
+ To activate it, you need to provide the `consumer` name and set the `strategy : auto`.
172
+ See the example below :
173
+
174
+ ` ` ` yaml
175
+ apiVersion: 0.1.0
176
+ meta:
177
+ version: x.y.z
178
+ name: my-http-sink
179
+ type: http-sink
180
+ topic: http-sink-topic
181
+ consumer:
182
+ id: my-http-sink
183
+ offset:
184
+ strategy: auto
185
+ start: beginning
186
+ flush-period:
187
+ secs: 10
188
+ nanos: 0
189
+ http:
190
+ endpoint: "http://127.0.0.1/post"
191
+ ` ` `
192
+
193
+ After the connector processed any records, you can check the last stored offset value via :
194
+ ` ` ` bash
195
+ $ fluvio consumer list
196
+ CONSUMER TOPIC PARTITION OFFSET LAST SEEN
197
+ my-http-sink http-sink-topic 0 0 3s
198
+ ` ` `
199
+
200
+ Read more about [Fluvio Offset].
201
+
168
202
# # Contributing
169
203
170
204
Follow on the conventional `CONTRIBUTING.md` file to setup your environment and
171
205
contribute to this project.
172
206
173
207
[Transformations] : /docs/latest/fluvio/concepts/transformations
174
- [JSON to JSON transformations] : /docs/latest/hub/smartmodules/jolt
208
+ [JSON to JSON transformations] : /docs/latest/hub/smartmodules/jolt
209
+ [Fluvio Offset] : /docs/latest/fluvio/concepts/offsets
You can’t perform that action at this time.
0 commit comments