kafkajs

library2.2.4jsnpmunverified

KafkaJS is a modern Apache Kafka client library for Node.js, currently at stable version 2.2.4. It offers comprehensive features for producing and consuming messages, as well as administrative tasks, providing native support for Kafka 0.10+ and specifically 0.11 features like transactions and custom authentication mechanisms. The library ships with full TypeScript definitions, ensuring a robust development experience. It aims for a low-level, high-performance approach, with a relatively frequent release cadence for bug fixes and minor enhancements, as seen by multiple patch releases in quick succession after major version bumps. A key differentiator is its focus on Node.js specifics and robust error handling to prevent common issues like CPU spikes.

database http-networking

npm install kafkajs

github ↗package ↗homepage ↗

INSTALL

IMPORT

SIG · KAFKAJS

kafkajs

databasejavascriptv2.2.4

Install

—

Import

—

Disk

—

Pass rate

0/ 6

Env Coverage0 / 6

glibc

18–22

musl

18–22

Install & Compatibility

Where this runs

tested against v? · npm install

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

node 18–226 runs

build_error

glibc

node 18–226 runs

build_error

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

Kafka

✓ import { Kafka } from 'kafkajs'

✗ const { Kafka } = require('kafkajs')

The primary entry point for configuring the Kafka client. While CommonJS `require` works, ESM `import` is the standard for modern Node.js applications, especially with TypeScript.

Producer, Consumer, Admin

✓ import { Producer, Consumer, Admin } from 'kafkajs'

✗ import { Producer } from 'kafkajs/lib/producer'

These interfaces/classes are typically instantiated via the `Kafka` instance (e.g., `kafka.producer()`), but their types or constructors are directly exported from the main `kafkajs` package. Avoid internal path imports.

EachMessagePayload, KafkaConfig

✓ import type { EachMessagePayload, KafkaConfig } from 'kafkajs'

✗ import { EachMessagePayload } from 'kafkajs'

For TypeScript, it's best practice to use `import type` when importing only type definitions to ensure no runtime code is generated, improving bundle size and clarity.

Partitioners

✓ import { Partitioners } from 'kafkajs'

✗ const { DefaultPartitioner } = require('kafkajs/partitioners')

Provides access to built-in partitioners like `DefaultPartitioner` and `LegacyPartitioner`. It's recommended to import `Partitioners` and then access the specific partitioner via dot notation.

This quickstart demonstrates how to initialize a Kafka client, send messages using a producer, and consume messages using a consumer, including basic error handling and graceful shutdown.

import { Kafka, logLevel } from 'kafkajs'; const kafka = new Kafka({ clientId: 'my-app', brokers: [process.env.KAFKA_BROKER_1 ?? 'localhost:9092'], logLevel: logLevel.INFO, }); const producer = kafka.producer(); const consumer = kafka.consumer({ groupId: 'my-group' }); const run = async () => { // Producer await producer.connect(); await producer.send({ topic: 'test-topic', messages: [ { value: 'Hello KafkaJS user!' }, { key: 'my-key', value: 'Another message with a key!' }, ], }); console.log('Message sent by producer.'); // Consumer await consumer.connect(); await consumer.subscribe({ topic: 'test-topic', fromBeginning: true }); await consumer.run({ eachMessage: async ({ topic, partition, message }) => { console.log({ value: message.value?.toString(), key: message.key?.toString(), headers: message.headers, topic, partition, }); }, }); console.log('Consumer started, waiting for messages...'); }; run().catch(async (e) => { console.error(`[example/kafkajs] ${e.message}`, e); await producer.disconnect(); await consumer.disconnect(); process.exit(1); }); process.on('SIGTERM', async () => { console.log('SIGTERM received, disconnecting Kafka clients.'); await producer.disconnect(); await consumer.disconnect(); process.exit(0); });

Debug

Known issues

breakingUpgrading to v2.0.0 from v1.x introduces several breaking changes, including a new default partitioner, removal of Node.js 10/12 support, and changes to admin client methods and TypeScript enums. Not addressing these can lead to messages being routed incorrectly or runtime errors.

fix

Thoroughly read the official v2.0.0 migration guide (https://kafka.js.org/docs/migration-guide-v2.0.0) before upgrading. Pay special attention to partitioner configuration and Node.js version requirements.

affects: >=2.0.0

gotchaPrior to v2.1.0, KafkaJS consumers could exhibit 100% CPU utilization when all configured Kafka brokers were unavailable, leading to application unresponsiveness and crashes.

fix

Upgrade to KafkaJS v2.1.0 or newer to resolve the issue of excessive CPU usage during prolonged broker unavailability.

affects: <2.1.0

gotchaA regression in versions 2.2.0 through 2.2.2 caused issues with SASL/PLAIN authentication, preventing clients from connecting correctly when using this mechanism.

fix

Upgrade to KafkaJS v2.2.3 or newer to fix the SASL/PLAIN authentication regression.

affects: 2.2.0 - 2.2.2

gotchaConsumers might get stuck or become unresponsive after very brief throttling periods, potentially halting message processing, a bug resolved in v2.2.4.

fix

Ensure you are using KafkaJS v2.2.4 or newer to mitigate issues with consumers getting stuck after throttling events.

affects: <2.2.4

gotchaOlder versions could enter an infinite crash loop when no brokers were available during startup or reconnection, leading to application instability.

fix

Upgrade to KafkaJS v2.2.4 or newer to prevent infinite crash loops when no brokers are available, improving client resilience.

affects: <2.2.4

gotchaRe-using `groupId` across different applications or independent deployments of the same application can lead to unexpected consumer behavior, such as receiving messages for unsubscribed topics or frequent rebalances.

fix

Ensure each logical consumer group (e.g., for different applications or environments) uses a unique `groupId`. Describe your consumer group to verify expected members.

affects: all

Errors

Common errors & fixes

The group is rebalancing, so a rejoin is needed (ILLEGAL_GENERATION)

A consumer group coordination issue where a rebalance incorrectly led to an `ILLEGAL_GENERATION` error preventing consumers from rejoining the group.

fix

Upgrade KafkaJS to v2.2.4 or newer to fix the consumer group rejoining logic after `ILLEGAL_GENERATION` errors.

Application consuming 100% CPU without processing messages.

KafkaJS client entering a tight loop when all configured Kafka brokers are unreachable or unavailable.

fix

Upgrade KafkaJS to v2.1.0 or newer to resolve the 100% CPU utilization issue during broker unavailability.

Failed to connect: SASL authentication failed (when using PLAIN mechanism)

A regression in `v2.2.0` through `v2.2.2` broke SASL/PLAIN authentication support.

fix

Upgrade KafkaJS to v2.2.3 or newer to fix the regression in SASL/PLAIN authentication.

Error: Topic authorization failed (producer getting stuck)

A producer might get stuck or persistently fail to send messages after encountering a topic authorization error.

fix

Upgrade KafkaJS to v2.1.0 or newer to fix persistent errors when producing after a topic authorization error.

Upgrade

Version history

2.2.4latest on npm

Audit

Dependencies

No dependency data recorded yet.

Agent activity

9 hits · last 30 days

node

Resources

githubhttps://github.com/tulios/kafkajs ↗

packagehttps://www.npmjs.com/package/kafkajs ↗

homepagehttps://kafkajs.org ↗