Install & Compatibility
Where this runs
tested against v3.5.6 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.95 runs
installs and imports cleanly · install 0.0s · import 0.190s · 18.3MB
glibcpy 3.10–3.95 runs
installs and imports cleanly · install 1.6s · import 0.170s · 19MB
16MB installed
● package 16MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
get_reusable_executor
✓ from loky import get_reusable_executor
This is the recommended entry point for most common use cases, providing a managed, reusable process pool.
ProcessPoolExecutor
✓ from loky import ProcessPoolExecutor
This provides a direct, robust replacement for `concurrent.futures.ProcessPoolExecutor` with enhanced error handling.
set_start_method
✓ from loky import set_start_method
✗ import multiprocessing; multiprocessing.set_start_method('spawn')
Loky has its own `set_start_method` function which *must* be used instead of `multiprocessing.set_start_method` to ensure proper behavior and compatibility with Loky's internal process management.
This quickstart demonstrates the two primary ways to use Loky: `get_reusable_executor()` for a managed and persistent pool, and `ProcessPoolExecutor()` for a direct `concurrent.futures`-like experience. The `if __name__ == "__main__":` block is included for robust multiprocessing execution across different operating systems.
import os
from loky import get_reusable_executor
def worker_function(x):
# Simulate some work
pid = os.getpid()
return f"Processed {x} by PID {pid}"
if __name__ == "__main__":
# Using get_reusable_executor for managed process pool
with get_reusable_executor(max_workers=2) as executor:
results = list(executor.map(worker_function, range(5)))
print(results)
# Direct ProcessPoolExecutor usage (similar to concurrent.futures)
from loky import ProcessPoolExecutor
with ProcessPoolExecutor(max_workers=2) as executor:
results_direct = list(executor.map(worker_function, range(5, 10)))
print(results_direct)
Debug
Known issues
breakingLoky's `set_start_method` is incompatible with `multiprocessing.set_start_method`. Attempting to use the standard library's function will not configure Loky's process startup correctly and can lead to unexpected behavior or errors.fixAlways import and use `loky.set_start_method()` when configuring the process start method for Loky executors.
affects: All versions
gotchaWhen using `loky.get_reusable_executor()` on Windows, worker processes are kept alive for reuse, which can prevent cleanup operations (e.g., `os.chdir()` or deleting temporary directories) if they were used within the worker context. Even `loky.ProcessPoolExecutor()` may leave one process active after explicit shutdown.fixFor scenarios requiring strict process termination and resource cleanup, consider if `get_reusable_executor()`'s persistence model is suitable. If not, explicitly manage process lifecycle and ensure all resources are released or cleaned up outside the main process before attempting directory changes or deletions.
affects: All versions, particularly on Windows
gotchaWhile `loky` transparently integrates `cloudpickle` to serialize non-picklable objects, this serialization can introduce performance overhead compared to Python's standard `pickle` module, especially for very large objects or high-frequency task submission.fixFor performance-critical applications, profile the serialization overhead. If it's a bottleneck, optimize objects to be standard-picklable or explore custom reducers for specific data types to minimize `cloudpickle`'s impact. Consider setting `LOKY_PICKLER=pickle` if `cloudpickle`'s broader serialization capabilities are not needed.
affects: All versions
gotchaLoky on POSIX systems (e.g., Linux, macOS) uses `fork+exec` for all processes to ensure consistent and robust spawn behavior, which is safer when interacting with third-party libraries (e.g., OpenMP, macOS Accelerate) compared to `multiprocessing.Pool`'s default `fork` (or pre-Python 3.8 macOS `fork`). This difference might subtly alter behavior if your code relies on `fork` without `exec` semantics.fixBe aware of the `fork+exec` behavior. If you encounter issues with shared memory or inherited resources, ensure all necessary state is explicitly passed to worker processes rather than relying on implicit inheritance. If working with very old Python versions or specific `multiprocessing.Pool` expectations, consider how this difference might affect your application.
affects: All versions on POSIX systems
Errors
Common errors & fixes
RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase.
This error occurs when attempting to create new processes in a Python script without protecting the entry point, especially on Windows and macOS where the default process start method is 'spawn' and the main module is re-imported by child processes.
fixWrap the code that creates and uses the `loky` executor (or any multiprocessing code) within an `if __name__ == '__main__':` block.
TypeError: cannot pickle '...' object
Objects and functions passed between processes via `loky`'s `ProcessPoolExecutor` must be 'picklable' (serializable). This error occurs when an unpicklable object, such as a local function, a lambda, or a complex object with unpicklable attributes (like a `_thread.lock` or `weakref`), is implicitly or explicitly sent to a worker process.
fixEnsure that all arguments to functions executed by the `loky` executor, as well as the function itself and any objects it closes over, are picklable. For complex objects, you might need to implement `__reduce__` or use a more powerful serialization library like `dill` by configuring `loky` to use it via `loky.set_loky_pickler('dill')`. Loky-backed parallel loops cannot be called in a multiprocessing, setting n_jobs=1 warning
This warning typically arises when `joblib` (which uses `loky` as a backend) is invoked within an already existing multiprocessing context, indicating that nested parallelism might not be correctly handled or that the inner `loky` pool is effectively disabled or limited to a single job.
fixAvoid nesting parallel calls. If using `joblib`, ensure that `Parallel` is not called from within a function that is itself being executed by another multiprocessing pool. Consider refactoring your code to flatten the parallelism or use thread-based parallelism for inner loops if appropriate.
BrokenProcessPool: A task has failed to un-serialize. Please ensure that the arguments of the function are all picklable.
This error occurs when a worker process fails to deserialize a task or its arguments, usually due to an unpicklable object being passed as part of the task, or a corruption in the serialization stream, leading to the worker process crashing or becoming unresponsive.
fixSimilar to `TypeError: cannot pickle`, meticulously review the function arguments and any objects referenced by the function that are being sent to `loky`'s worker processes to ensure they are all picklable. If `cloudpickle` (loky's default advanced pickler) is still failing, consider simplifying the objects or explicitly handling their serialization.
Upgrade
Version history
3.5.6latest on PyPI · released Aug 27, 2025
Audit
Dependencies
pythonrequiredRequires Python 3.9 or higher for the latest versions.
cloudpickleoptionalOptional dependency that enables serialization of a wider range of objects (e.g., lambda functions, interactively defined functions), especially those in the __main__ module, avoiding common pickling errors.
psutiloptionalOptional dependency used for early detection of memory leaks in worker processes.