Release 3.5.1 of the Neo4j Kafka integration
We're very happy to present the new 3.5.1 release of the Neo4j Kafka integrations.
Our focus since the last release was on making our integration easier to use and fixing some issues.
We introduced several new features in the Sink:
The fixes #99: Provide a roundtrip-sink-config allows you to ingest data that that comes from another Neo4j instances as CDC events.
In a similar way fixes #154: provide a common pattern for ingestion allows you to define simple expressions in order to extract data from any nested event structure and transfrom that data into Nodes and/or Relationships.
For example to create a user and their purchases from the users
and orders
topics:
streams.sink.topic.pattern.node.users=User{!userId}
streams.sink.topic.pattern.relationship.orders=User{!userId} BOUGHT{purchase.price, purchase.currency} Product{!productId}
The fixes #102: Manual commit behavior for handling errors and retrievals allows you to use a manual committing consumer, moreover improves the streams.consume
procedure allowing to read data starting from a specific partition/offset.
Breaking changes
There is a little change about the Sink management, with the fixes 160: change the streams.sink.enabled to false the Streams plugin now has, for the property streams.sink.enabled
, the default value set to false
so you need to explict set it to true otherwise, if you only specify the topic mapping, you'll see a WARN
message into the neo4j.log
We also fixed several issues:
- fixes #186: Kafka even sink with manual commit fails with multiple topic subscriptions
- fixes #185: Consumer not working in Cluster environment
- Chunked docs
- fixes #102: Manual commit behavior for handling errors and retrievals
- fixes 160: change the streams.sink.enabled to false
- fixes #154: provide a common pattern for ingestion
- fixes #173: Update Documentation: Streams.Enabled and Producer.Enabled
- fixes #99: Provide a roundtrip-sink-config
- fixes #167
- fixes #164: If docker is not installed skip integration tests
- fixes #162: Conversion error in case of nested field of array records
- Fix YAML for NEO4J_ACCEPT_LICENSE_AGREEMENT and add auto-install of connector
- fixes #14: Add schema reference into Data Events
- fixes #147: Kafka Connect Sink Transient Error Mangement
- fixes #116: Connector Configuration (Sink)
- added Plugin Installation section
- Adjust config for kafka-connect-maven-plugin in Kafka connector