Registry /
llm-agents / langchain-nvidia-ai-endpoints
Install & Compatibility
Where this runs
tested against v1.4.3 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.95 runs
installs and imports cleanly · install 0.0s · import 2.884s · 77.7MB
glibcpy 3.10–3.95 runs
installs and imports cleanly · install 9.7s · import 2.704s · 88MB
82MB installed
● package 82MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
ChatNVIDIA
✓ from langchain_nvidia_ai_endpoints import ChatNVIDIA
NVIDIAEmbeddings
✓ from langchain_nvidia_ai_endpoints import NVIDIAEmbeddings
NVIDIARerank
✓ from langchain_nvidia_ai_endpoints import NVIDIARerank
ChatNVIDIADynamo
✓ from langchain_nvidia_ai_endpoints import ChatNVIDIADynamo
Used for Dynamo KV cache optimization support, introduced in v1.1.0.
NVIDIARAGRetriever
✓ from langchain_nvidia_ai_endpoints import NVIDIARAGRetriever
Connects to NVIDIA RAG Blueprint /v1/search endpoint, introduced in v1.2.1.
This quickstart demonstrates how to initialize `ChatNVIDIA` for text generation and `NVIDIAEmbeddings` for creating text embeddings. It requires `NVIDIA_API_KEY` to be set as an environment variable, obtainable from the NVIDIA API Catalog. Replace 'nvidia/nemotron-3-super-120b-a12b' and 'nvolveqa_40k' with desired model names available on NVIDIA AI Endpoints.
import os
from langchain_nvidia_ai_endpoints import ChatNVIDIA, NVIDIAEmbeddings
nvapi_key = os.environ.get('NVIDIA_API_KEY', '')
if not nvapi_key.startswith('nvapi-'):
print("Please set the NVIDIA_API_KEY environment variable. You can get one from the NVIDIA API Catalog.")
else:
# Initialize ChatNVIDIA for conversational AI
chat_model = ChatNVIDIA(model="nvidia/nemotron-3-super-120b-a12b", nvidia_api_key=nvapi_key)
chat_response = chat_model.invoke("Explain the concept of large language models.")
print("Chat Model Response:", chat_response.content)
# Initialize NVIDIAEmbeddings for semantic embeddings
embed_model = NVIDIAEmbeddings(model="nvolveqa_40k", nvidia_api_key=nvapi_key)
embedding_output = embed_model.embed_query("What are vector embeddings?")
print("Embedding Model Output Length:", len(embedding_output))
Debug
Known issues
breakingWith the release of LangChain 1.0, the broader LangChain ecosystem underwent significant changes, streamlining its core and moving some legacy functionality to `langchain-classic`. While `langchain-nvidia-ai-endpoints` adapted, ensure your overall `langchain` and `langchain-core` dependencies are compatible.fixReview LangChain's v1 migration guide. For `langchain-nvidia-ai-endpoints`, ensure `langchain-core` is updated to a compatible version (e.g., `0.1.15` or newer as per examples).
affects: >=1.0.0 (langchain ecosystem), >=1.0.0 (langchain-nvidia-ai-endpoints)
gotchaIncorrect or missing `NVIDIA_API_KEY` will lead to authentication failures. The key must start with `nvapi-`.fixObtain your `NVIDIA_API_KEY` from the NVIDIA API Catalog and ensure it is correctly set as an environment variable or passed directly during class instantiation. Verify it starts with `nvapi-`.
affects: All versions
gotchaUsing older versions of `langchain-nvidia-ai-endpoints` with newer NVIDIA AI Foundation Models may result in uninformative errors or lack of support for latest model features. For example, specific models might require a minimum package version.fixAlways keep `langchain-nvidia-ai-endpoints` updated to the latest version to ensure compatibility with the newest NVIDIA models and features. Refer to GitHub releases for model-specific support.
affects: <1.2.1
gotchaWhen self-hosting NVIDIA NIM microservices, incorrect `base_url` or port configuration for models like `NVIDIARAGRetriever` can lead to `NVIDIARAGConnectionError`.fixEnsure the `base_url` parameter points to the correct endpoint (e.g., `http://localhost:8081` for RAG server) and that the NIM container is running and accessible.
affects: All versions
Errors
Common errors & fixes
Exception: [403] Forbidden Invalid UAM response
This error, or similar '401 Unauthorized' or 'ValueError: Invalid API key' messages, typically indicates an issue with the provided NVIDIA API key, such as it being incorrect, expired, or not having the necessary permissions for the requested model.
fixEnsure the `NVIDIA_API_KEY` environment variable is correctly set with a valid API key, or pass the `nvidia_api_key` parameter explicitly when initializing `ChatNVIDIA` or `NVIDIAEmbeddings`. Regenerate your API key from the NVIDIA API Catalog if it is expired or you suspect it's invalid.
AttributeError: module 'langchain_nvidia_ai_endpoints' has no attribute '__version__'
The `langchain_nvidia_ai_endpoints` package does not expose a `__version__` attribute for programmatic access to its version information.
fixTo check the installed version, use `pip show langchain-nvidia-ai-endpoints` in your terminal.
ValueError: Unknown model name <model_name> specified.
This error occurs when the model name provided to `ChatNVIDIA` or `NVIDIAEmbeddings` is incorrect, outdated, or not accessible with your API key. This can also manifest as `Exception: [404] Not Found The model <model_name> does not exist.`
fixUse `ChatNVIDIA.get_available_models()` or `NVIDIAEmbeddings.get_available_models()` to retrieve a list of currently supported and available model names for your API key, and then use one of those names.
AttributeError: 'NVIDIAEmbeddings' object has no attribute 'truncate'
This error typically occurs when attempting to call a method named `truncate` on an `NVIDIAEmbeddings` object, but the `truncate` functionality is provided as a parameter during the constructor's initialization, not as a separate method. This can also be an issue if using an older version of the library where the `truncate` parameter was not available in the constructor.
fixPass the `truncate` option as a parameter when initializing `NVIDIAEmbeddings` (e.g., `NVIDIAEmbeddings(truncate='END')`) instead of trying to call it as a method. Ensure you are using a recent version of the `langchain-nvidia-ai-endpoints` library where this parameter is supported.
Upgrade
Version history
1.4.3latest on PyPI · released Jul 2, 2026
Audit
Dependencies
langchain-corerequiredCore LangChain abstractions; specific versions may be required for full compatibility with newer features.