A simple transfer data between NiFi and CQL (support Apache Cassandra, ScyllaDB, AstraDB, etc.). The implementation details:
- development NiFi v2 processor (with controller) and java application (support Java 17/21+)
- support Apache Cassandra v4/v5, ScyllaDB, AstraDB based on CQL (Cassandra Query Language)
- the Apache NiFi v2 does not support Apache Cassandra v4/v5 (NiFi v2 removed the Cassandra processor due to security vulnerabilities and unmaintained code. The processor supported only Cassandra v3 not newer.)
You can use this preferred way (NiFi v2 processor with controller):
- ✅ PutCQL, put data to CQL solution, where inputs are FlowFiles
- ✅ GetCQL, get data from CQL solution, where outputs are FlowFiles
- NOTE: You can download the latest version, see 'nifi-cql-nar-*.nar' file
or two other older alternative ways:
- ✅ ExecuteProcess with java application (see 'Ff2Cql-*.jar'), where inputs are CSV files
- ✅ ExecuteStreamCommand with java application (see 'Ff2Cql-*.jar'), where inputs are FlowFiles via stdin
You can see sample of relevant setting for CQL controller here:
- Cassandra setting
- access with IP addresses (name/password)
- Scylla setting
- access with IP address (without name/password)
- AstraDB setting
- access with Security Connection Bundle (SCB) and Token
- NOTE about SCB: see the file with '.zip' suffix
- How to get the SCB? Go to the https://astra.datastax.com/..., Menu Databases/Your database/Region/Download SCB
- NOTE about token: see the file with 'token.json' suffix
- How to get the token? Go to the https://astra.datastax.com/..., Menu Tokens/Generate Token/Token Details/Download Token Details
NOTE:
- These connection settings are used directly in unit tests.
