Releases · neo4j-contrib/neo4j-streams

02 Apr 15:42

4.0.1

0df75fa

4.0.1

New release

Assets 4

13 Mar 17:55

neo-technology-build-agent

4.0.0

aa20992

First Release for Neo4j v 4.0

Hi community!
We are very proud to share the first release of Streams Project for Neo4j 4.0 (here the docs).
This release supports the Multi database feature for sink and source in the following way.

Neo4j Streams: Source Module

You can set for each database instance a configuration suffix with the following pattern from.<DB_NAME> to the properties in your neo4j.conf file.

Following the list of new properties that allows to support multi-tenancy:

streams.source.topic.nodes.<TOPIC_NAME>.from.<DB_NAME>=<PATTERN>
streams.source.topic.relationships.<TOPIC_NAME>.from.<DB_NAME>=<PATTERN>
streams.source.enabled.from.<DB_NAME>=<true/false, default=true>
streams.procedures.enabled.<DB_NAME>=<true/false, default=true>

For more in-depth description, please got to the following doc page

Neo4j Streams: Sink Module

You can set for each database instance a configuration suffix with the following pattern to.<DB_NAME> to the properties in your neo4j.conf file.

Following the list of new properties that allows to support multi-tenancy:

streams.sink.topic.cypher.<TOPIC_NAME>.to.<DB_NAME>=<CYPHER_QUERY>
streams.sink.topic.cdc.sourceId.to.<DB_NAME>=<LIST_OF_TOPICS_SEPARATE_BY_SEMICOLON>
streams.sink.topic.cdc.schema.to.<DB_NAME>=<LIST_OF_TOPICS_SEPARATE_BY_SEMICOLON>
streams.sink.topic.pattern.node.<TOPIC_NAME>.to.<DB_NAME>=<NODE_EXTRACTION_PATTERN>
streams.sink.topic.pattern.relationship.<TOPIC_NAME>.to.<DB_NAME>=<RELATIONSHIP_EXTRACTION_PATTERN>
streams.sink.enabled.to.<DB_NAME>=<true/false, default=true>
streams.procedures.enabled.<DB_NAME>=<true/false, default=true>

For more in-depth description, please got to the following doc page

Multi Database Support

You can define into the json (or via the Confluent UI) a param named neo4j.database which is the targeted database name.

N.b. If no value is specified the connector will use the Neo4j's default db.

Assets 4

04 Mar 20:48

conker84

3.5.6

82678dd

Minor release 3.5.6

This minor release contains the following fixes:

fixes #284: Core server will not rejoin cluster if Kafka plugin is enabled.

Assets 4

23 Jan 16:00

neo-technology-build-agent

3.5.5

5975d40

3.5.5

Hello Neo4j Community!
We're very happy to present the new 3.5.5 release of the Neo4j Kafka integrations, that is the result of great community feedback!

Thanks a lot to everyone who contributed to this: @moxious, @jexp, @conker84 and @F-Guardian

Following a list of improvements:

with fixes #252: Source with non-existant topic hangs indefinitely (#257) we improved the behaviour in case the BROKER config auto.create.topics.enable is false and you're sending an event to a topic that is not created in the Kafka cluster. If you're publish events to a topic that is not created the Producer#send can block until the timeout defined by max.block.ms, so we add a system that checks every topic defined in the Kafka Cluster and log if the topic that you defined into your neo4j.conf file exist.
with fixes #247: Schema Ingestion Strategy: double node creation (#248) we fixed a bug where, under particular circumstances, can lead to a double node creation in case you're ingesting data with the Schema ingestion strategy
with Fixes an issue with wrong ids value in cdc when updating relationships we fixed a bug where that could lead to stream wrong ids for the target node while you're updating relationships

Other fixes:

fix typo in documentation section 4.2 (#253)
fixes #249: Correct typo in DLQ property (#262)
fixes #259: Fix Kafka plugin version (#260)
fixes #264: Neo4j Sink Connector: CDC ingestion with schema strategy

To get more info about it please look at the documentation.

Assets 4

20 Sep 09:56

neo-technology-build-agent

3.5.4

cb0d9b3

3.5.4

Hello Neo4j Community!
We're very happy to present the new 3.5.4 release of the Neo4j Kafka integrations, that is the result of great community feedback!

We improved the management of the DLQ. Please look at the documentation
We introduced the support for the transient errors into the Kafka Connect Plugin
With #211 we introduced the AVRO support to the Neo4j Sink Plugin, the streams.consume procedure manages it as well! Please look at the Supported Kafka deserializers
With #188 we introduced the support for a new file interexchange format, we called it CUD File Format and is a JSON file that represents Graph Entities (Nodes/Relationships) and how to mange them in term of Create/Update/Delete operations. i.e.:

{
  "op": "merge",
  "properties": {
    "foo": "value",
    "key": 1
  },
  "ids": {"key": 1, "otherKey":  "foo"},
  "labels": ["Foo","Bar"],
  "type": "node",
  "detach": true
}

will be transformed into the follwing Cypher Query:

UNWIND [..., {
  "op": "merge",
  "properties": {
    "foo": "value",
    "key": 1
  },
  "ids": {"key": 1, "otherKey":  "foo"},
  "labels": ["Foo","Bar"],
  "type": "node",
  "detach": true
}, ...] AS event
MERGE (n:Foo:Bar {key: event.ids.key, otherKey: event.ids.otherKey})
SET n += event.properties

For more details please look into the documentation

With #181 we validate a minimun amount of configuration params in orde to make the streams plugin work.

To get more info about it please look at the documentation.

Assets 4

02 Jul 08:21

neo-technology-build-agent

3.5.3

b597f09

3.5.3

We're very happy to present the new 3.5.3 release of the Neo4j Kafka integrations.

Our focus since the last release was on making our integration easier to use and fixing some issues.

We introduced several new features in the Sink:

With fixes #177: Create a Dead Letter Queue you can manage the Dead Lettere Queue in case of bad data, this feature is initially available only into the Streams Plugin, we plan to also release it for the Kafka Connect Sink Plugin in the near future.

With fixes #198: Tombstone record Management both the Streams Sink and the Kafka Connect Sink Plugin can get improved, in particular, the pattern strategy come out with the support to the Tombstone Record,
in order to leverage it your event should contain as key the record that you want to delete and null for the value.

To get more info about it please look at the documentation.

Assets 4

02 Jul 08:24

neo-technology-build-agent

3.4.4

edec8a9

3.4.4

We're very happy to present the new 3.4.4 release of the Neo4j Kafka integrations.

Our focus since the last release was on making our integration easier to use and fixing some issues.

We introduced several new features in the Sink:

To get more info about it please look at the documentation.

Assets 4

17 Jun 13:30

neo-technology-build-agent

3.5.2

02783ac

3.5.2

New release

Assets 4

17 Jun 15:08

neo-technology-build-agent

3.4.3

67d5815

3.4.3

New release

Assets 4

06 Jun 12:55

neo-technology-build-agent

3.5.1

96fa6ed

Release 3.5.1 of the Neo4j Kafka integration

We're very happy to present the new 3.5.1 release of the Neo4j Kafka integrations.

Our focus since the last release was on making our integration easier to use and fixing some issues.

We introduced several new features in the Sink:

The fixes #99: Provide a roundtrip-sink-config allows you to ingest data that that comes from another Neo4j instances as CDC events.

In a similar way fixes #154: provide a common pattern for ingestion allows you to define simple expressions in order to extract data from any nested event structure and transfrom that data into Nodes and/or Relationships.

For example to create a user and their purchases from the users and orders topics:

streams.sink.topic.pattern.node.users=User{!userId}
streams.sink.topic.pattern.relationship.orders=User{!userId} BOUGHT{purchase.price, purchase.currency} Product{!productId}

The fixes #102: Manual commit behavior for handling errors and retrievals allows you to use a manual committing consumer, moreover improves the streams.consume procedure allowing to read data starting from a specific partition/offset.

Breaking changes

There is a little change about the Sink management, with the fixes 160: change the streams.sink.enabled to false the Streams plugin now has, for the property streams.sink.enabled, the default value set to false so you need to explict set it to true otherwise, if you only specify the topic mapping, you'll see a WARN message into the neo4j.log

We also fixed several issues:

Assets 4

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Neo4j Streams: Source Module

Neo4j Streams: Sink Module

Multi Database Support

Releases: neo4j-contrib/neo4j-streams

4.0.1

First Release for Neo4j v 4.0

Neo4j Streams: Source Module

Neo4j Streams: Sink Module

Multi Database Support

Minor release 3.5.6

3.5.5

3.5.4

3.5.3

3.4.4

3.5.2

3.4.3

Release 3.5.1 of the Neo4j Kafka integration