Registry / gcp / gcp-storage-emulator

gcp-storage-emulator

JSON →
library2024.8.3pypypiunverified

The `gcp-storage-emulator` is a Python library that provides a stub emulator for the Google Cloud Storage API, enabling local development and testing without requiring a connection to the actual Google Cloud Storage service. It's actively maintained, with frequent releases, and is currently at version 2024.8.3. This emulator is particularly useful for accelerating development cycles and running integration tests locally.

pip install gcp-storage-emulator
INSTALL
IMPORT
SIG · GCP-STORAGE-EMULAT
G
gcp-storage-emulator
gcppythonv2024.8.3
Install
2.1s avg
Import
582ms
Disk
22MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v2024.8.3 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.910 runs
installs and imports cleanly · install 0.0s · import 0.378s · 20.1MB
glibc
py 3.103.910 runs
installs and imports cleanly · install 2.1s · import 0.320s · 21MB
22MB installed
● package 22MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

create_server
from gcp_storage_emulator import create_server
from gcp_storage_emulator import create_server

This quickstart demonstrates how to programmatically start the `gcp-storage-emulator`, configure the `google-cloud-storage` client to interact with it, and perform basic operations like creating a bucket, uploading a blob, and downloading its content. The emulator is configured for in-memory storage for ephemeral testing.

import os from google.cloud import storage, exceptions from gcp_storage_emulator.server import create_server HOST = "localhost" PORT = 9023 BUCKET_NAME = "my-test-bucket" FILE_NAME = "test-blob.txt" CONTENT = b"Hello, GCS Emulator!" # 1. Start the emulator server # The default_bucket parameter creates the bucket automatically upon server start. server = create_server(HOST, PORT, in_memory=True, default_bucket=BUCKET_NAME) server.start() try: # 2. Configure the Google Cloud Storage client to use the emulator # Use os.environ.setdefault to allow external configuration. os.environ.setdefault("STORAGE_EMULATOR_HOST", f"http://{HOST}:{PORT}") # 3. Create a client. Project ID is arbitrary for the emulator. client = storage.Client(project="test-project") # 4. Get the bucket (created by default_bucket in server or create manually) bucket = client.bucket(BUCKET_NAME) # Ensure the bucket exists (useful if not using default_bucket param) try: bucket.create() print(f"Bucket '{BUCKET_NAME}' created.") except exceptions.Conflict: # Bucket already exists print(f"Bucket '{BUCKET_NAME}' already exists.") pass # 5. Upload a blob blob = bucket.blob(FILE_NAME) blob.upload_from_string(CONTENT) print(f"Uploaded '{FILE_NAME}' with content: {CONTENT.decode()}.") # 6. Download the blob downloaded_content = blob.download_as_bytes() print(f"Downloaded '{FILE_NAME}' with content: {downloaded_content.decode()}.") # 7. List blobs in the bucket print(f"Blobs in '{BUCKET_NAME}':") for listed_blob in bucket.list_blobs(): print(f" - {listed_blob.name}") finally: # 8. Stop the emulator server server.stop() print("GCP Storage Emulator stopped.") # Clean up the environment variable if needed, though usually not critical after stop if "STORAGE_EMULATOR_HOST" in os.environ: del os.environ["STORAGE_EMULATOR_HOST"]
gcp-storage-emulator --version
Debug
Known issues
breakingThe `gcp-storage-emulator` only supports a limited subset of the full Google Cloud Storage API. Advanced features, specific HTTP headers, or less common API methods might not be fully emulated, leading to unexpected behavior or errors.
fix
Consult the project's GitHub README and open issues for supported features. For critical missing features, consider contributing or using a more comprehensive solution like MinIO configured for GCS interoperability if S3 API compatibility is acceptable.
affects: <=2024.8.3
gotchaWhen running the emulator in Docker, especially if the client is outside the Docker network, resumable uploads or blob downloads might fail with connection errors (e.g., `host='0.0.0.0'`) or 404s due to incorrect hostname resolution in callback URLs.
fix
Ensure the `STORAGE_EMULATOR_HOST` environment variable points to a resolvable IP or hostname (e.g., `http://localhost:9023` or `http://host.docker.internal:9023` if using Docker Compose) that the client can reach. Check GitHub issues for specific workarounds related to `0.0.0.0` hostnames in callback URLs.
affects: <=2024.8.3
gotchaBy default, the emulator persists data to a local `.cloudstorage` directory relative to the current working directory. This can lead to unexpected state between test runs or local development sessions.
fix
For ephemeral testing, use the `--in-memory` CLI flag or `in_memory=True` when calling `create_server()`. For controlled persistence, specify `STORAGE_BASE` and `STORAGE_DIR` environment variables or use the `gcp-storage-emulator wipe` command.
affects: <=2024.8.3
gotchaThe `google-cloud-storage` client library automatically detects the emulator through the `STORAGE_EMULATOR_HOST` environment variable. If this variable is unset or points to an incorrect address, the client will attempt to connect to the production Google Cloud Storage API, potentially causing authentication errors or unexpected cloud costs.
fix
Always set `STORAGE_EMULATOR_HOST` to the emulator's address (e.g., `http://localhost:9023`) before initializing `google.cloud.storage.Client`. Use `os.environ.setdefault()` for flexible configuration.
affects: <=2024.8.3
Upgrade
Version history
2024.8.3latest on PyPI · released Aug 3, 2024
Audit
Dependencies
google-cloud-storagerequiredRequired for the client library to interact with the emulator. The emulator is designed to work with Google's official client libraries.
Agent activity
31 hits · last 30 days
node
26
OpenAI (training)
1
Resources