Registry / gcp / cloudml-hypertune

cloudml-hypertune

JSON →
library0.1.0.dev6pypypi✓ verified 86d ago

Cloudml-hypertune is a lightweight Python library providing helper functions to report hyperparameter tuning metrics to Google Cloud's Vertex AI (formerly Cloud ML Engine). It enables the hyperparameter tuning service to track and optimize model training trials by collecting objective metrics. Despite its `0.1.0.dev6` version being quite old (last released December 2019), it remains the standard way to report custom metrics for hyperparameter tuning on Google Cloud.

pip install cloudml-hypertune
INSTALL
IMPORT
SIG · CLOUDML-HYPERTUNE
C
cloudml-hypertune
gcppythonv0.1.0.dev6
Install
2.4s avg
Import
10ms
Disk
17MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v0.1.0.dev6 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.915 runs
installs and imports cleanly · install 0.0s · import 0.007s · 19.2MB
glibc
py 3.103.915 runs
installs and imports cleanly · install 2.4s · import 0.006s · 20MB
17MB installed
● package 17MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

HyperTune
import hypertune hpt = hypertune.HyperTune()
from cloudml_hypertune import HyperTune
The primary class 'HyperTune' is typically accessed via the top-level 'hypertune' module, not directly imported from 'cloudml_hypertune'.

This quickstart demonstrates how to use `cloudml-hypertune` within a training script to report a metric. The script accepts hyperparameters as command-line arguments, which is a critical requirement for Google Cloud's hyperparameter tuning service to inject trial-specific values. The `HyperTune` instance then reports the objective metric and the current training step. You would run this script within a container on Vertex AI/Cloud ML Engine.

import hypertune import argparse import os def train_model(learning_rate, num_epochs, metric_tag): # Simulate model training with hyperparameters # In a real scenario, this would be your ML training loop print(f"Training with learning_rate={learning_rate}, num_epochs={num_epochs}") # Simulate a metric, e.g., validation accuracy # In a real scenario, you'd get this from your model's evaluation metric_value = 0.5 + (learning_rate * 0.1) + (num_epochs * 0.01) # Report the metric to Cloud AI Platform / Vertex AI hpt = hypertune.HyperTune() hpt.report_hyperparameter_tuning_metric( hyperparameter_metric_tag=metric_tag, # Must match config.yaml objective metricTag metric_value=metric_value, global_step=num_epochs # Or current training step ) print(f"Reported metric '{metric_tag}': {metric_value} at step {num_epochs}") if __name__ == '__main__': parser = argparse.ArgumentParser() # Define hyperparameters as command-line arguments parser.add_argument( '--learning_rate', type=float, default=0.01, help='Learning rate for training.' ) parser.add_argument( '--num_epochs', type=int, default=10, help='Number of epochs for training.' ) parser.add_argument( '--metric_tag', type=str, default='accuracy', help='Tag for the metric reported to HyperTune.' ) args = parser.parse_args() train_model(args.learning_rate, args.num_epochs, args.metric_tag)
Debug
Known issues
gotchaThe `cloudml-hypertune` library is specifically designed to work within Google Cloud's hyperparameter tuning services (Vertex AI/AI Platform). It is not a standalone hyperparameter tuning framework and will not function as such outside of this cloud environment for driving optimization.
fix
Ensure your training job is submitted to Vertex AI (or deprecated Cloud ML Engine) with a hyperparameter tuning configuration.
affects: All
breakingHyperparameters you wish to tune *must* be exposed as command-line arguments in your training script. The Vertex AI tuning service passes trial-specific hyperparameter values via these arguments, not directly to the `hypertune` library.
fix
Modify your training script to parse hyperparameters as `argparse` arguments, ensuring they are named consistently with your tuning job configuration.
affects: All
gotchaThe `hyperparameter_metric_tag` passed to `hpt.report_hyperparameter_tuning_metric` must exactly match the `metric_id` (or `hyperparameterMetricTag` in older configs) specified in your Vertex AI hyperparameter tuning job configuration. A mismatch will result in trials failing to report metrics or the tuning job not recognizing the objective.
fix
Carefully verify that the metric tag string in your code is identical to the one in your Vertex AI hyperparameter tuning job configuration.
affects: All
gotchaVersion `0.1.0.dev6` was released in December 2019. While still functional and referenced in Google Cloud documentation for Vertex AI, its lack of recent updates may lead to assumptions of abandonment or compatibility issues with very new Python features, though it's a very stable, minimal utility.
fix
No direct fix needed, but be aware of its static nature. If encountering unexpected behavior with bleeding-edge Python or ML frameworks, consider if the issue is external to `cloudml-hypertune`.
affects: All
gotchaTraining runs that result in `NaN` values in loss functions or other unhandled exceptions will cause hyperparameter tuning trials to fail. This not only wastes resources but also prevents the tuning algorithm from learning from that trial.
fix
Implement robust error handling and numeric stability checks (e.g., clipping gradients, checking for `NaN`s) in your training code. Ensure `report_hyperparameter_tuning_metric` is called even if training terminates early due to an error, or within a `finally` block if a fallback metric can be determined.
affects: All
Errors
Common errors & fixes
Hyperparameter tuning failed
The hyperparameter tuning service could not retrieve the objective metric from the training job. This often happens due to a mismatch between the `hyperparameter_metric_tag` in your code and the metric configuration in Vertex AI.
fix
Verify that the `hyperparameter_metric_tag` used in `hpt.report_hyperparameter_tuning_metric()` exactly matches the `metric_id` specified in your Vertex AI hyperparameter tuning job configuration. Also ensure the `report_hyperparameter_tuning_metric` call is actually reached and executed during training.
Trials show status 'Failed' but logs don't show Python errors
The training code might be completing without explicitly reporting a metric, or the metric is being reported incorrectly. Older TensorFlow Estimator-based training might not be correctly outputting to event files in a format recognized by the tuning service if `cloudml-hypertune` isn't used.
fix
Ensure `hpt.report_hyperparameter_tuning_metric` is called with a valid `metric_value` and `hyperparameter_metric_tag` at the end of each evaluation step or epoch. For non-TensorFlow models or custom evaluation loops, `cloudml-hypertune` is the definitive way to report.
Google Cloud ML Engine does not return objective values when hyperparameter tuning
The `cloudml-hypertune` library's `report_hyperparameter_tuning_metric` function was not invoked or did not successfully transmit the metric. This could be due to an uncaught exception in the training code preventing the call, or an incorrect metric tag.
fix
Debug your training script to ensure `hpt.report_hyperparameter_tuning_metric` is called. Add logging around this call to confirm its execution and the values being passed. Double-check the `hyperparameter_metric_tag` against your Vertex AI job configuration.
Upgrade
Version history
0.1.0.dev6latest on PyPI · released Dec 18, 2019
Audit
Dependencies

No dependency data recorded yet.

Agent activity
65 hits · last 30 days
node
58
OpenAI (training)
1
Resources
cloudml-hypertune — pip install cloudml-hypertune · libregistry