g3t-etl

library0.0.1rc26pypypi✓ verified 85d ago

g3t-etl (Gen3 Transfer ETL) is a utility library for working with Gen3 data commons. It provides tools for data transfer, validation, and extraction. Current version 0.0.1rc26, requires Python >=3.9, <4. Under active development with frequent releases.

data

pip install g3t-etl

github ↗package ↗

INSTALL

IMPORT

SIG · G3T-ETL

g3t-etl

datapythonv0.0.1rc26

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

g3t_etl

✓ import g3t_etl

Direct import recommended for most use cases.

TransferClient

✓ from g3t_etl.transfer import TransferClient

✗ from g3t_etl import TransferClient

TransferClient is in the transfer submodule.

ValidationError

✓ from g3t_etl.validation import ValidationError

✗ from g3t_etl.errors import ValidationError

ValidationError is in validation module, not errors.

Initialize a TransferClient with Gen3 endpoint and API key. List available projects as a basic operation.

import os from g3t_etl.transfer import TransferClient # Initialize client (uses GEN3_API_KEY or GEN3_CREDENTIALS env vars) client = TransferClient(endpoint='https://example.com', api_key=os.environ.get('GEN3_API_KEY', '')) # List available projects projects = client.list_projects() print(projects)

g3t-etl --version

Debug

Known issues

breakingIn version 0.0.1rc25, the TransferClient constructor changed from requiring 'gen3_endpoint' and 'gen3_api_key' to 'endpoint' and 'api_key'. Update your code accordingly.

fix

Use TransferClient(endpoint='...', api_key='...') instead of the old keyword arguments.

affects: >=0.0.1rc25

deprecatedThe 'validate_manifest' function in g3t_etl.validation is deprecated in favor of 'validate_manifest_with_schema' which provides more detailed errors.

fix

Replace g3t_etl.validation.validate_manifest() with g3t_etl.validation.validate_manifest_with_schema().

affects: <=0.0.1rc26

gotchaThe library expects environment variables GEN3_API_KEY or GEN3_CREDENTIALS to be set for authentication. Missing these will cause silent authentication failures.

fix

Set GEN3_API_KEY (or GEN3_CREDENTIALS) environment variable before using any client.

affects: >=0.0.1

Errors

Common errors & fixes

ModuleNotFoundError: No module named 'g3t_etl'

Library not installed or installed under a different name.

fix

Run: pip install g3t-etl

AttributeError: 'TransferClient' object has no attribute 'list_projects'

Using outdated version where method was named differently or not available.

fix

Update to the latest version: pip install --upgrade g3t-etl

g3t_etl.transfer.TransferError: 401 Unauthorized

Invalid or missing API key.

fix

Check that GEN3_API_KEY environment variable is set correctly and not expired.

Upgrade

Version history

0.0.1rc26latest on PyPI · released Jun 21, 2024

Audit

Dependencies

gen3requiredCore Gen3 SDK for API interactions

pandasrequiredData manipulation and tabular I/O

requestsrequiredHTTP client

Agent activity

14 hits · last 30 days

node

Resources

githubhttps://github.com/ACED-IDP/gen3_util ↗

packagehttps://pypi.org/project/g3t-etl/ ↗