Coiled is a Python client library that simplifies scaling Python code and Dask clusters on the cloud (AWS, GCP, Azure). It handles cloud resource management, networking, and software environments, allowing data engineers and scientists to focus on their code. The library is actively maintained, with frequent updates to support new features and cloud capabilities. Its current version is 1.134.0, and it generally follows a continuous release cadence with frequent minor updates.
pip install "coiled[dask]"Verified import paths — ran on the pinned version, not inferred.
This quickstart demonstrates how to create and use a Dask cluster and a serverless Python function with Coiled. Before running, ensure you have authenticated with `coiled login` and connected your cloud account using `coiled setup <aws|gcp|azure>` via the Coiled CLI. You can specify a software environment for consistency or Coiled will attempt to synchronize your local environment.
Use `coiled.Cluster(software='my-software-environment')` or `coiled.Cluster(container='daskdev/dask:latest')` to specify a managed environment or Docker image. For serverless functions, similar options apply to the `@coiled.function` decorator.
Run `pip install coiled` then `coiled login` and `coiled setup <your-cloud-provider>` in your terminal to authenticate and configure your cloud account.
Always include `client.close()` and `cluster.close()` in your scripts when you are done with the Dask cluster. For `coiled.function` with `keepalive` set, be mindful of the duration to prevent VMs from staying active longer than needed.
Always specify the `region` parameter when creating a `coiled.Cluster` or using `@coiled.function` to match the location of your data (e.g., `coiled.Cluster(region='us-east-2')`).
Install the coiled library using pip: `pip install coiled` or conda: `conda install -c coiled coiled`.
Ensure proper SSL configuration is provided to your Dask client or task runner, often by explicitly setting a `security` parameter in the `coiled.Cluster` or `DaskTaskRunner` configuration, or by passing an `ssl_context` argument where expected.
Ensure your web browser allows all cookies, temporarily disable browser extensions like 'Brave Shields' if in use, or re-authenticate the Coiled client using the `coiled login` command.
Use the correct and supported method for setting environment variables for Dask workers within Coiled, which might involve using the `environment_vars` parameter directly in `coiled.Cluster` or passing a dictionary of Dask configurations via the `config` argument, e.g., `coiled.Cluster(config={'distributed.scheduler.worker-saturation': '1.0'})`.