Registry / gcp / gsutil

gsutil

JSON →
library5.36pypypiunverified

gsutil is a Python application that serves as a command-line tool for interacting with Google Cloud Storage. It allows users to perform a wide range of bucket and object management tasks, such as creating and deleting buckets, uploading, downloading, and deleting objects, and managing ACLs. While `gsutil` is functional and widely documented, Google now recommends using `gcloud storage` commands from the Google Cloud CLI for Cloud Storage interactions due to better performance and support for newer features. The current version is 5.36, and it sees regular maintenance releases.

pip install gsutil
INSTALL
IMPORT
SIG · GSUTIL
G
gsutil
gcppythonv5.36
harness data pending
Install & Compatibility
Where this runs

No compatibility data collected yet for this library.

Code
Verified usage

This quickstart demonstrates how to interact with `gsutil` from Python by calling it as a subprocess. This is the common pattern for programmatic use, as `gsutil` is primarily a command-line interface. It includes creating a bucket, uploading a file, listing contents, downloading a file, and cleaning up resources.

import subprocess import os # Ensure gcloud is authenticated (e.g., via `gcloud auth login` in terminal) # For programmatic use, consider setting GOOGLE_APPLICATION_CREDENTIALS # os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = '/path/to/keyfile.json' bucket_name = os.environ.get('GSUTIL_TEST_BUCKET', 'my-test-bucket-12345') file_name = 'test_file.txt' file_content = 'Hello, Cloud Storage!' # Create a local file to upload with open(file_name, 'w') as f: f.write(file_content) try: # Create a bucket print(f"Creating bucket gs://{bucket_name}...") subprocess.run(['gsutil', 'mb', f'gs://{bucket_name}'], check=True, capture_output=True) print(f"Bucket gs://{bucket_name} created.") # Upload a file print(f"Uploading {file_name} to gs://{bucket_name}/{file_name}...") subprocess.run(['gsutil', 'cp', file_name, f'gs://{bucket_name}/{file_name}'], check=True, capture_output=True) print("File uploaded.") # List bucket contents print(f"Listing contents of gs://{bucket_name}:") result = subprocess.run(['gsutil', 'ls', f'gs://{bucket_name}'], check=True, capture_output=True, text=True) print(result.stdout) # Download the file downloaded_file_name = 'downloaded_test_file.txt' print(f"Downloading gs://{bucket_name}/{file_name} to {downloaded_file_name}...") subprocess.run(['gsutil', 'cp', f'gs://{bucket_name}/{file_name}', downloaded_file_name], check=True, capture_output=True) print("File downloaded.") with open(downloaded_file_name, 'r') as f: content = f.read() print(f"Downloaded content: {content}") except subprocess.CalledProcessError as e: print(f"gsutil command failed: {e}") print(f"STDOUT: {e.stdout.decode()}") print(f"STDERR: {e.stderr.decode()}") finally: # Clean up (delete file and bucket) print(f"Cleaning up: deleting objects from gs://{bucket_name}...") subprocess.run(['gsutil', '-m', 'rm', '-r', f'gs://{bucket_name}/**'], check=False, capture_output=True) print(f"Deleting bucket gs://{bucket_name}...") subprocess.run(['gsutil', 'rb', f'gs://{bucket_name}'], check=False, capture_output=True) print("Cleanup complete.") if os.path.exists(file_name): os.remove(file_name) if os.path.exists(downloaded_file_name): os.remove(downloaded_file_name)
gsutil --version
Debug
Known issues
breaking`gsutil` is a legacy Cloud Storage CLI and minimally maintained. Google strongly recommends migrating to `gcloud storage` commands for all Cloud Storage interactions.
fix
Transition scripts and workflows to use `gcloud storage` commands. Refer to the 'Transition from gsutil to gcloud storage' documentation.
affects: All versions, as of March 2026. Explicitly noted from version 5.36 onwards.
gotchaThere is no official Python client library for `gsutil` itself. Programmatic interaction from Python is typically achieved by invoking `gsutil` commands via `subprocess` calls. For a native Python API, use `google-cloud-storage`.
fix
For direct integration into Python applications, use the `google-cloud-storage` client library (e.g., `pip install google-cloud-storage`). If `gsutil` CLI functionality is strictly required, use Python's `subprocess` module to execute `gsutil` commands.
affects: All versions.
breaking`gsutil` dropped support for older Python versions. Versions 5.35 and later require Python 3.9 to 3.13.
fix
Ensure your environment uses a compatible Python version (3.9-3.13 for recent `gsutil` versions). Upgrade Python if necessary.
affects: 5.32 and later.
gotchaScripts that parse `gsutil` command output may break when transitioning to `gcloud storage` due to differences in output formatting, error messages, and data listings.
fix
Thoroughly review and update any automated scripts that rely on parsing `gsutil` output if migrating to `gcloud storage`.
affects: All versions, when considering migration.
gotchaWhen transferring a large number of files or large single files, `gsutil`'s performance might require manual optimization, whereas `gcloud storage` is engineered for speed by default.
fix
For optimal performance with large transfers, especially for new projects, consider using `gcloud storage`. If sticking with `gsutil`, explore options like the `-m` (multi-threaded/multi-processing) flag for `cp` and `rsync` operations, or adjusting `parallel_process_count` and `parallel_thread_count` settings in the `.boto` configuration.
affects: All versions.
Errors
Common errors & fixes
CommandException: Downloading this composite object requires integrity checking with CRC32c, but your crcmod installation isn't using the module's C extension, so the hash computation will likely throttle download performance.
`gsutil` encounters this when downloading composite objects, as it needs the `crcmod` C extension for efficient CRC32c integrity checking, which is often missing or not properly compiled.
fix
Install the `crcmod` C extension by running `pip install -U crcmod`. Ensure that development tools (like `gcc` and Python headers) are installed on your system if you encounter compilation issues.
gsutil: command not found
The `gsutil` executable is not found in your system's PATH environment variable, meaning the shell cannot locate the command, or the Google Cloud SDK (which includes `gsutil`) is not installed.
fix
Install the Google Cloud SDK and ensure that its `bin` directory (e.g., `/path/to/google-cloud-sdk/bin`) is added to your system's PATH. Alternatively, execute `gsutil` by specifying its full path.
AccessDeniedException: 403 <user/service_account> does not have storage.objects.list access to the Google Cloud Storage bucket.
The authenticated user account or service account lacks the necessary IAM permissions to perform the requested operation (e.g., listing objects, reading, or writing) on the specified Google Cloud Storage bucket or object.
fix
Grant the appropriate IAM role (e.g., `Storage Object Viewer`, `Storage Object Creator`, `Storage Admin`, or a custom role with specific permissions) to the principal on the bucket or object via the Google Cloud Console or `gcloud iam` commands.
You are attempting to access protected data with no configured credentials. Please visit https://cloud.google.com/console#/project and sign up for an account, and then run the "gcloud auth login" command to configure gsutil to use these credentials.
`gsutil` requires valid credentials to access protected Cloud Storage resources, but none are configured or the existing authentication token has expired.
fix
Run `gcloud auth login` in your terminal to authenticate with your Google Cloud account through a web browser. If using a service account, activate it with `gcloud auth activate-service-account --key-file=/path/to/key.json`.
AccessDeniedException: 403 Forbidden
The authenticated user or service account lacks the necessary IAM permissions to perform the requested operation (e.g., upload, download, delete) on the specified Google Cloud Storage bucket or object.
fix
Grant the required IAM roles (e.g., `roles/storage.objectViewer`, `roles/storage.objectCreator`, `roles/storage.objectAdmin`, `roles/storage.bucketAdmin`) to the principal associated with your `gcloud` credentials in the Google Cloud Console for the relevant project and resource.
Upgrade
Version history
5.36latest on PyPI · released Mar 3, 2026
Audit
Dependencies
google-cloud-storageoptionalOfficial Python client library for programmatic interaction with Google Cloud Storage, recommended over direct gsutil calls from Python.
crcmodrequiredRequired for integrity checks.
google-apitoolsrequiredGoogle API client library dependencies.
google-authrequiredGoogle authentication library dependencies.
Agent activity
27 hits · last 30 days
node
20
OpenAI (training)
1
Resources
gsutil — pip install gsutil · libregistry