Registry / ai-ml / node-llama-cpp

node-llama-cpp

library3.18.1jsnpmunverified

Run large language models (LLMs) locally from Node.js using llama.cpp bindings. Version 3.18.1 provides pre-built binaries for macOS, Linux, and Windows (Metal, CUDA, Vulkan) with automatic fallback to source build via cmake (no node-gyp or Python required). Supports JSON schema enforcement, function calling, embedding, reranking, and chat sessions. Full TypeScript types included. Active development with frequent releases synced to upstream llama.cpp. Key differentiator: zero-config GPU acceleration and safe token injection prevention.

ai-ml

npm install node-llama-cpp

github ↗package ↗homepage ↗

INSTALL

IMPORT

SIG · NODE-LLAMA-CPP

node-llama-cpp

ai-mljavascriptv3.18.1

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

getLlama

✓ import { getLlama } from 'node-llama-cpp'

✗ const getLlama = require('node-llama-cpp').getLlama

Package is ESM-only; require() fails unless using dynamic import()

LlamaChatSession

✓ import { LlamaChatSession } from 'node-llama-cpp'

✗ import LlamaChatSession from 'node-llama-cpp/LlamaChatSession'

Named export, not default. Subpath imports are not supported.

getLlama

✓ import { getLlama } from 'node-llama-cpp'

✗ import getLlama from 'node-llama-cpp'

Default export does not exist; must use named import.

Loads a GGUF model and performs a simple chat prompt, showing file setup, model loading, and session creation.

import { fileURLToPath } from 'url'; import path from 'path'; import { getLlama, LlamaChatSession } from 'node-llama-cpp'; const __dirname = path.dirname(fileURLToPath(import.meta.url)); const llama = await getLlama(); const model = await llama.loadModel({ modelPath: path.join(__dirname, 'model.gguf') }); const context = await model.createContext(); const session = new LlamaChatSession({ contextSequence: context.getSequence() }); const response = await session.prompt('What is the capital of France?'); console.log(response);

Debug

Known issues

breakingPackage is ESM-only since v3.0 - CommonJS require() will not work.

fix

Use import syntax or dynamic import() inside CommonJS modules.

affects: >=3.0.0

breakingNode.js >=20.0.0 required - earlier versions cause runtime errors.

fix

Upgrade Node.js to a version >=20.

affects: all

deprecatedLlamaModel.load() is deprecated in favor of LlamaModel.loadModel().

fix

Use llama.loadModel() instead of llama.load().

affects: >=3.0.0

gotchaPre-built binaries may fail on uncommon architectures; build from source via cmake fallback.

fix

Ensure cmake and build tools are installed, or set NODE_LLAMA_CPP_SKIP_DOWNLOAD=true to force source build.

affects: all

gotchaModel path must be absolute; relative paths may not resolve correctly.

fix

Use path.resolve(__dirname, 'model.gguf') or path.join with __dirname.

affects: all

Errors

Common errors & fixes

Error [ERR_REQUIRE_ESM]: require() of ES Module not supported

Using require() to load an ESM-only package.

fix

Switch to import syntax or use dynamic import(): const { getLlama } = await import('node-llama-cpp');

TypeError: getLlama is not a function

Named import misused as default import.

fix

Use import { getLlama } from 'node-llama-cpp' instead of import getLlama from 'node-llama-cpp'.

Error: Model file not found at /some/path

Relative model path resolved incorrectly from process.cwd().

fix

Use an absolute path: path.resolve(__dirname, 'model.gguf').

Error: Dynamic library not found: libllama.dylib

Pre-built binary mismatch or missing cmake build.

fix

Reinstall package, force source build: npm rebuild node-llama-cpp --build-from-source

Upgrade

Version history

3.18.1latest on npm

Audit

Dependencies

typescriptoptionalPeer dependency: >=5.0.0 for TypeScript types and compilation support

Agent activity

7 hits · last 30 days

node

Resources

githubhttps://github.com/withcatai/node-llama-cpp ↗

packagenode-llama-cpp ↗

homepagehttps://node-llama-cpp.withcat.ai ↗