Registry / database / sequelize-kafka-connect

sequelize-kafka-connect

library4.0.0jsnpmunverified

A Node.js Kafka Connect connector for MySQL, PostgreSQL, SQLite, and MSSQL databases using Sequelize ORM. Version 4.0.0 allows streaming data from databases to Kafka (source connector) and from Kafka to databases (sink connector). It supports custom ETL transformations and offers both API and CLI usage. The library is currently in maintenance mode with limited updates and does not support Node.js >= 13 when using native mode with node-rdkafka. A key differentiator is its tight integration with Sequelize for schema management.

database messaging

npm install sequelize-kafka-connect

github ↗package ↗homepage ↗

INSTALL

IMPORT

SIG · SEQUELIZE-KAFKA-CO

sequelize-kafka-connect

databasejavascriptv4.0.0

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

runSourceConnector

✓ import { runSourceConnector } from 'sequelize-kafka-connect'

✗ const { runSourceConnector } = require('sequelize-kafka-connect')

ESM import is preferred; CommonJS require also works but may have interop issues depending on bundler

runSinkConnector

✓ import { runSinkConnector } from 'sequelize-kafka-connect'

✗ const runSinkConnector = require('sequelize-kafka-connect').runSinkConnector

Named exports are available; default export is not provided

ConverterFactory

✓ import { ConverterFactory } from 'sequelize-kafka-connect'

✗ import ConverterFactory from 'sequelize-kafka-connect'

ConverterFactory is a named export, not default; also available as a static class

Initializes a source connector that streams data from a PostgreSQL database to a Kafka topic. Replace kafka and database connection details with your environment.

import { runSourceConnector } from 'sequelize-kafka-connect'; const config = { kafka: { kafkaHost: process.env.KAFKA_HOST ?? 'localhost:9092', groupId: 'kc-sequelize-test', clientName: 'kc-sequelize-test-name', workerPerPartition: 1, options: { sessionTimeout: 8000, protocol: ['roundrobin'], fromOffset: 'earliest', fetchMaxBytes: 1024 * 100, fetchMinBytes: 1, fetchMaxWaitMs: 10, heartbeatInterval: 250, retryMinTimeout: 250, requireAcks: 1, }, }, topic: 'my_topic', partitions: 1, maxTasks: 1, pollInterval: 2000, produceKeyed: true, produceCompressionType: 0, connector: { options: { host: process.env.DB_HOST ?? 'localhost', port: 5432, dialect: 'postgres', database: 'mydb', username: process.env.DB_USER ?? 'user', password: process.env.DB_PASS ?? '', }, }, }; runSourceConnector(config, [], (err) => console.error(err)).then((cfg) => { console.log('Connector started, stop with cfg.stop()'); });

Debug

Known issues

breakingNative mode (noptions) requires node-rdkafka which only works with Node.js 9-12. Not compatible with Node.js >= 13.

fix

Use JavaScript mode (without noptions) or downgrade Node.js to 12.x.

affects: >=4.0.0

gotchaWhen using native mode, yarn is required to install node-rdkafka; npm will remove dependencies.

fix

Install yarn globally (npm i -g yarn) and use yarn add instead of npm install.

affects: >=1.0.0

gotchaThe library does not provide a default export; CommonJS require must destructure correctly.

fix

Use 'const { runSourceConnector } = require('sequelize-kafka-connect')' instead of 'const runSourceConnector = require('sequelize-kafka-connect')'.

affects: >=4.0.0

gotchaThe CLI commands nkc-sequelize-source and nkc-sequelize-sink are in beta and may have limited functionality.

fix

Prefer using the API programmatically until CLI is stable.

affects: >=4.0.0

deprecatedThe zkConStr (ZooKeeper connection string) option is deprecated in newer versions; use kafkaHost instead.

fix

Remove zkConStr and set kafkaHost to your Kafka broker address.

affects: >=3.0.0

Errors

Common errors & fixes

Error: Cannot find module 'node-rdkafka'

Native mode requires node-rdkafka to be manually installed.

fix

Run 'yarn add node-rdkafka@2.7.4' (requires yarn and Node.js 9-12).

TypeError: (0 , _sequelizeKafkaConnect.runSourceConnector) is not a function

Using incorrect import style (default import instead of named import).

fix

Change to 'import { runSourceConnector } from 'sequelize-kafka-connect''.

Error: connect ECONNREFUSED 127.0.0.1:9092

Kafka broker is not running or unreachable at the specified host/port.

fix

Ensure Kafka is running and accessible; check kafkaHost config.

SequelizeConnectionRefusedError: connect ECONNREFUSED

Database server is not running or connection details are incorrect.

fix

Verify database host, port, and credentials in connector.options.

Upgrade

Version history

4.0.0latest on npm

Audit

Dependencies

node-rdkafkaoptionalRequired for native Kafka support (noptions config); manual install needed due to build issues

Agent activity

21 hits · last 30 days

node

OpenAI (training)

Resources

githubhttps://github.com/nodefluent/sequelize-kafka-connect ↗

packagesequelize-kafka-connect ↗

homepagehttps://github.com/nodefluent/sequelize-kafka-connect#readme ↗