Registry /
gcp / google-cloud-bigquery-datatransfer
Install & Compatibility
Where this runs
tested against v3.23.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.95 runs
installs and imports cleanly · install 0.0s · import 1.784s · 69.5MB
glibcpy 3.10–3.95 runs
installs and imports cleanly · install 5.8s · import 1.278s · 68MB
68MB installed
● package 68MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
DataTransferServiceClient
✓ from google.cloud.bigquery_datatransfer import DataTransferServiceClient
✗ from google.cloud import bigquery_datatransfer_v1
This quickstart initializes the BigQuery Data Transfer client and lists available data sources within a specified project and location. Replace 'your-project-id' with your actual Google Cloud Project ID, or ensure the GOOGLE_CLOUD_PROJECT environment variable is set. Authentication is handled implicitly via Application Default Credentials (e.g., `gcloud auth application-default login` for local development or service account credentials on Google Cloud infrastructure). Ensure the BigQuery Data Transfer API is enabled in your Google Cloud project.
import os
from google.cloud import bigquery_datatransfer_v1
# Set your Google Cloud Project ID and Location
# For local development, set the GOOGLE_APPLICATION_CREDENTIALS environment variable.
# Example: export GOOGLE_APPLICATION_CREDENTIALS="/path/to/your/key.json"
project_id = os.environ.get("GOOGLE_CLOUD_PROJECT", "your-project-id")
location = "us"
client = bigquery_datatransfer_v1.DataTransferServiceClient()
parent = client.location_path(project_id, location)
try:
print(f"Listing data sources in project '{project_id}' and location '{location}':")
for data_source in client.list_data_sources(parent=parent):
print(f" Name: {data_source.display_name} (ID: {data_source.data_source_id})")
except Exception as e:
print(f"Error listing data sources: {e}
Ensure the BigQuery Data Transfer API is enabled for project '{project_id}' and your credentials have sufficient permissions.")
Debug
Known issues
breakingFuture versions of Google Cloud client libraries, including `google-cloud-bigquery-datatransfer`, will drop support for Python 3.7 and 3.8. Python 3.7 reached end-of-life (EOL) in June 2023, and 3.8 in October 2024. New major versions of the library will be incompatible with these EOL Python runtimes.fixUpgrade your Python environment to Python 3.9 or newer to ensure continued compatibility and support. Check the official `google-cloud-python` documentation for the latest supported Python versions.
affects: All versions after Python 3.8 EOL (October 2024), specific breaking changes will be in new major versions.
gotchaBigQuery Data Transfer Service's data source connectors (e.g., Google Ads, Display & Video 360) periodically undergo API version upgrades. These upgrades can introduce schema changes, such as new, modified, or deprecated columns, which may affect existing transfer configurations and downstream data processing pipelines.fixRegularly monitor the BigQuery Data Transfer Service data source change log for announcements related to your specific data sources. Be prepared to update your BigQuery schemas, ETL processes, and queries as needed to accommodate schema changes.
affects: All versions, as this is a service-level change for data connectors.
gotchaAuthentication requires appropriate IAM roles. For service accounts creating transfers, 'BigQuery Admin' and 'BigQuery Data Transfer Service Agent' roles are often necessary, in addition to permissions to access the source data. Incorrect permissions are a common cause of transfer failures.fixEnsure the service account or user initiating the transfer has the 'BigQuery Admin' (roles/bigquery.admin) role, the 'BigQuery Data Transfer Service Agent' role (roles/bigquery.datatransfer.serviceAgent), and read/write access to the source/destination datasets/tables. For external data sources, manual authorization via OAuth may also be required initially.
affects: All versions.
gotchaEffective February 1, 2026, BigQuery requests that read data from multi-region Cloud Storage buckets will incur Cloud Storage multi-region data transfer fees. This can significantly impact billing for transfers involving data stored across different regions.fixReview your data transfer configurations. To minimize costs, consider co-locating Cloud Storage buckets with your BigQuery datasets in the same region where possible.
affects: All versions, effective Feb 1, 2026.
deprecatedBigQuery will limit the use of Legacy SQL starting June 1, 2026. If a project has not used Legacy SQL between November 1, 2025, and June 1, 2026, it will no longer be able to use it. Existing workloads might continue, but new ones may fail.fixMigrate any existing transfer configurations or queries that rely on Legacy SQL to Standard SQL. Review BigQuery documentation on migrating from Legacy SQL.
affects: All versions, impacting usage after June 1, 2026.
breakingThe executed Python script contains a fundamental syntax error (e.g., an unclosed string literal, malformed statement), preventing the script from being parsed and executed correctly.fixReview the Python script at the indicated line number (line 20 in this case) and correct the syntax. Ensure all strings are properly closed, parentheses/brackets are matched, and statements follow Python's grammar rules. This is a user-code error.
affects: All Python versions
breakingThe script contains a SyntaxError, specifically an 'unterminated f-string literal'. This indicates a coding error in the Python script itself, where an f-string (formatted string literal) was opened but not properly closed with a matching quote.fixReview the Python script at the indicated line number (line 20 in this case) and ensure all f-strings and other string literals are correctly opened and closed with matching quotes. For example, 'print(f"Error: {e}")' instead of 'print(f"Error: {e}')'. affects: All versions (this is a coding error, not a library or environment issue).
Upgrade
Version history
3.23.0latest on PyPI · released Jun 3, 2026
Audit
Dependencies
google-api-corerequiredCore Google API client functionality.
google-authrequiredHandles Google Cloud authentication.
grpciorequiredgRPC client for API communication.
proto-plusrequiredProvides Pythonic wrappers for Protocol Buffer messages.
protobufrequiredProtocol Buffers for data serialization.