Registry / gcp / pathwaysutils

pathwaysutils

library0.1.9pypypi✓ verified 85d ago

Pathwaysutils is a Python package providing essential utilities and tools to streamline the deployment and execution of JAX workloads on the Pathways on Cloud architecture. It simplifies large-scale machine learning computations across multiple TPU slices by handling cloud-specific adaptations. Pathways is used internally at Google for models like Gemini, and this package brings similar benefits to Google Cloud customers. The current version is 0.1.7, and it is actively developed with releases tightly coupled to JAX versions.

gcp ai-ml

pip install pathwaysutils

github ↗package ↗changelog ↗

INSTALL

IMPORT

SIG · PATHWAYSUTILS

pathwaysutils

gcppythonv0.1.9

Install

18.0s avg

Import

4964ms

Disk

738MB

Pass rate

4/ 10

Env Coverage4 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v0.1.4 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

glibc

py 3.10

✕ build_error

✓ 19.28s

py 3.11

✕ build_error

✓ 18.38s

py 3.12

✕ build_error

✓ 16.88s

py 3.13

✕ build_error

✓ 17.38s

py 3.9

✕ build_error

738MB installed

● package 738MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

pathwaysutils

✓ import pathwaysutils

The quickstart demonstrates the essential initialization step for using pathwaysutils. It configures JAX to leverage the Pathways on Cloud infrastructure, allowing JAX applications to orchestrate workloads across distributed TPU slices. Note that Pathways on Cloud is in Preview and requires prior access.

import pathwaysutils import jax # Pathways on Cloud is currently in Preview and requires access. # Ensure your environment variables are correctly configured for Pathways on Cloud. # Example: os.environ['JAX_PLATFORMS'] = 'proxy' # Initialize pathwaysutils to configure the JAX backend for Pathways on Cloud. # This also registers a custom ArrayHandler for checkpointing and disables JAX's compilation cache. pathwaysutils.initialize() # Verify JAX devices are accessible through the Pathways backend print(f"JAX devices available via Pathways: {jax.devices()}") # Further JAX workload execution would follow here.

Debug

Known issues

breakingPathways on Cloud is currently in Preview. Access requires contacting your Google Cloud account representative. Without access, the utilities will not function as intended.

fix

Contact your Google Cloud account representative to request access to Pathways on Cloud.

affects: All versions (as of 0.1.7)

gotchaPathways releases are tightly coupled with JAX versions. Mismatched versions can lead to compatibility and stability issues.

fix

Always verify that your Pathways artifacts and JAX version are aligned. Check release notes or documentation for compatible JAX versions.

affects: All versions

gotchaCalling `pathwaysutils.initialize()` disables the standard JAX compilation cache. This is intentional for Pathways workloads but might affect expectations if you rely on the JAX cache elsewhere.

fix

Be aware that the JAX compilation cache is managed by Pathways. Ensure your Cloud Storage bucket used for Pathways has the necessary permissions (storage.buckets.get, storage.buckets.update, storage.objects.list, storage.objects.create, storage.objects.get) for its caching mechanisms.

affects: All versions

gotchaA Pathways cluster can only maintain a session with one client at a time. Attempts by multiple clients to connect simultaneously will result in connection errors for subsequent clients.

fix

Ensure only one client is connected to a Pathways cluster at any given time. After an unexpected disconnection, you may need to restart the Pathways cluster.

affects: All versions

gotchaErrors after importing `pathwaysutils` might be caused by outdated Flask or Werkzeug versions. Upgrading these packages can sometimes resolve the issue, but may introduce conflicts with other dependencies.

fix

Check for outdated Flask or Werkzeug versions. If found, consider upgrading them cautiously, being mindful of other package dependencies. You might need to manage your Python environment carefully or use a dedicated virtual environment.

affects: Potentially all versions

Errors

Common errors & fixes

RuntimeError: Unable to initialize backend 'proxy': UNAVAILABLE: Unable to establish connection to ifrt_proxy server, please check provided address 'example-workload-proxy-0-0.example-workload.default.svc.example-cluster-domain.:38676'

This error occurs when the Pathways IFRT proxy client cannot connect to the IFRT proxy server, often due to network misconfiguration, firewall issues, or an unprovisioned Pathways cluster.

fix

Ensure your VPC network and firewall are correctly configured to allow the connection, verify the Pathways cluster is provisioned, and check for any startup error messages on the proxy server.

LocalProxy.init() got an unexpected keyword argument 'unbound_message'. If you see this error after importing pathwaysutils, check if there are outdated versions of Flask or Werkzeug installed in your environment.

This error typically indicates a dependency conflict where `pathwaysutils` is imported, but an older, incompatible version of Flask or Werkzeug is present in the environment.

fix

Upgrade Flask and/or Werkzeug to compatible versions. You can list installed versions with `pip3 list` and then upgrade them, being mindful of other project dependencies.

ModuleNotFoundError: No module named 'pathwaysutils'

This error occurs when the Python interpreter cannot find the `pathwaysutils` package, typically because it has not been installed, or the Python environment/path is not correctly configured.

fix

Install the package using pip: `pip install pathwaysutils`. If already installed, ensure you are running your script in the correct Python environment where the package is installed.

Internal error from proxy server during Array::IsDeleted(): UNAVAILABLE:Connection to IFRT proxy server was terminated: FAILED_PRECONDITION: GrpcClientSession: writes no longer allowed.

This error indicates that the IFRT Proxy server has unexpectedly disconnected from the client during a JAX workload execution, often requiring a client-side restart.

fix

Restart your client environment. For notebooks, restart the notebook kernel and rerun your code. If the issue persists, the Pathways cluster itself might need restarting to enable new connections.

Upgrade

Version history

0.1.9latest on PyPI · released Jun 12, 2026

Audit

Dependencies

absl-pyrequiredRequired for core functionalities.

fastapirequiredLikely used for API interactions within the Pathways ecosystem.

uvicornrequiredASGI server, often used with FastAPI.

orbax-checkpointrequiredEssential for efficient checkpoint operations.

jaxrequiredCore machine learning framework that pathwaysutils extends.

packagingrequiredFor version parsing and compatibility checks.

requestsrequiredFor making HTTP requests.