Dask-CUDA is a Python library providing utilities to facilitate interactions between Dask and NVIDIA CUDA-enabled GPUs. It extends `dask.distributed`'s `LocalCluster` and `Worker` to manage and deploy Dask workers efficiently on GPU systems. Key features include automatic instantiation of per-GPU workers, setting CPU affinity for optimal performance, and robust GPU memory management, including spilling to host memory. It is a core component of the RAPIDS suite for GPU-accelerated data science. The library maintains an active development status with regular releases, currently at version 26.4.0.
pip install dask-cudaVerified import paths — ran on the pinned version, not inferred.
This quickstart demonstrates how to set up a `LocalCUDACluster` and connect a `dask.distributed.Client`. It highlights common configurations like specifying visible GPUs, configuring RAPIDS Memory Manager (RMM) for memory pooling, and enabling cuDF spilling to prevent out-of-memory errors on large datasets. The use of an `if __name__ == "__main__":` block is crucial for standalone scripts.
Remove `numba` from your environment if it was installed as a direct `dask-cuda` dependency. Ensure `numba-cuda` is installed if you rely on CUDA JIT features.
Install `distributed-ucxx` (e.g., `pip install distributed-ucxx` or `conda install distributed-ucxx -c conda-forge`) and update Dask configuration keys from `distributed.comm.ucx.*` to `distributed-ucxx.*` if previously configured manually.
Upgrade your NVIDIA CUDA Toolkit to a supported version (e.g., CUDA 12.x or 13.x) or use an older `dask-cuda` version that supports CUDA 11.
Adopt the recommended Dask DataFrame API for GPU acceleration by setting `dask.config.set({"dataframe.backend": "cudf"})` after importing `dask` and ensuring `cudf` is installed.Always wrap your `LocalCUDACluster` and `Client` creation in an `if __name__ == "__main__":` block when running as a script.