Registry / database / exframe-etl-queue

exframe-etl-queue

JSON →
library3.1.6jsnpmunverified

exframe-etl-queue is a queue-based ETL step processor for Node.js (v18+), version 3.1.6. It manages enqueuing messages during extraction, automatically advancing them through ETL steps (Extract, Transform, Load) upon acknowledgment. Key differentiator vs generic queues is built-in ETL step lifecycle and error handling (messages go to error state on failure). Requires peer dependencies exframe-db 4.x and exframe-logger 3.x. Ships TypeScript types and uses CommonJS (require). Release cadence is not documented but based on semver.

database
npm install exframe-etl-queue
package ↗homepage ↗
INSTALL
IMPORT
SIG · EXFRAME-ETL-QUEUE
E
exframe-etl-queue
databasejavascriptv3.1.6
harness data pending
Install & Compatibility
Where this runs

No compatibility data collected yet for this library.

Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

EtlQueue
✓ const { EtlQueue } = require('exframe-etl-queue')
✗ const EtlQueue = require('exframe-etl-queue')
CommonJS destructured import; default import not available. Types available via @types package or bundled.
EtlSteps
✓ const { EtlSteps } = require('exframe-etl-queue')
✗ import { EtlSteps } from 'exframe-etl-queue'
Only CommonJS require supported; ESM import will fail.
EtlStep enum
✓ const { EtlSteps } = require('exframe-etl-queue'); const step = EtlSteps.Extract
✗ EtlSteps['Extract']
Using string keys works but may not provide type safety.

Demonstrates initializing EtlQueue with EtlSteps.Transform, enqueuing a message, and listening with a callback that updates targets.

const { EtlQueue, EtlSteps } = require('exframe-etl-queue'); const { db } = require('./lib/db'); const logger = console; async function main() { const queue = new EtlQueue(logger, EtlSteps.Transform, db); // Enqueue a message await queue.enqueue('myContext', 'orders', 'Q001', 'resumeToken', { operationType: 'insert', operationTimestamp: new Date(), fullDocument: { quoteNumber: 'Q001', amount: 100 }, updateDescription: null }); // Listen for messages await queue.listen('myContext', async (context, message) => { console.log(`Processing ${message.documentId}`); message.targets = [{ target: require('nanoid')() }]; }); } main().catch(console.error);
Debug
Known issues
breakingVersion 3 drops support for Node <18; peer dependencies require exframe-db 4.x and exframe-logger 3.x.
fix
Update Node to >=18, upgrade exframe-db to 4.x and exframe-logger to 3.x.
affects: >=3.0.0
deprecatedThe EtlQueue constructor expects logger as first argument; logging interface may change in future releases.
fix
Use a logger that supports 'info', 'error', 'warn' methods. No immediate change needed.
affects: >=3.0.0
gotchaExports are CommonJS only; using ES import syntax will cause errors.
fix
Use require() with destructuring, or use dynamic import() with .default? Not guaranteed.
affects: >=3.0.0
Errors
Common errors & fixes
Cannot find module 'exframe-db'
Peer dependency exframe-db@4.x not installed.
fix
npm install exframe-db@4
TypeError: EtlQueue is not a constructor
Importing default instead of named export.
fix
Use `const { EtlQueue } = require('exframe-etl-queue')`
The engine "node" is incompatible with this module.
Node version is below 18.
fix
Upgrade Node.js to >=18
Upgrade
Version history
3.1.6latest on npm
Audit
Dependencies
exframe-dbrequiredDatabase driver for persisting queue state and messages
exframe-loggerrequiredLogger for diagnostic output
Agent activity
5 hits · last 30 days
node
4
Resources
packageexframe-etl-queue ↗
homepagehttps://bitbucket.org/exzeo-usa/exframe#readme ↗
exframe-etl-queue — npm install exframe-etl-queue · libregistry