The `azureml-dataprep` library is part of the Azure ML Python SDK v1, providing capabilities to load, transform, and write data for machine learning workflows within the v1 ecosystem. As of version 5.4.3, it is primarily used for creating `Dataset` objects that integrate with Azure ML workspaces (v1). While receiving maintenance updates, it is largely superseded by the Azure ML SDK v2 (`azure-ai-ml`) for new development, which offers different data handling paradigms.
pip install azureml-dataprepVerified import paths — ran on the pinned version, not inferred.
Demonstrates how to read local data into a `Dataflow` object, apply a basic transformation, and convert it to a Pandas DataFrame. This showcases core `azureml-dataprep` functionalities for local data preparation without requiring an active Azure ML workspace connection.
For new projects, consider using `azure-ai-ml` and its integrated data handling capabilities, which often leverage standard Python data libraries like Pandas and PyArrow with direct cloud storage access, or MLTable assets.
Be mindful of the lazy evaluation paradigm. Use `head()` or `to_pandas_dataframe()` periodically during development to inspect intermediate results and ensure transformations are applied as expected.
Use isolated virtual environments for projects that rely on `azureml-dataprep` to prevent dependency conflicts. If possible, avoid mixing v1 and v2 SDK components in the same environment.
Ensure your Python environment is running Python 3.8 or a later compatible version (e.g., 3.9, 3.10) before installing `azureml-dataprep`.
Ensure `azureml-dataprep` is explicitly installed in the *remote compute environment* (e.g., via a custom `Environment` definition with `pip install -U azureml-dataprep`), use a supported Python version (3.7-3.12), or reduce the number of features/split large datasets.
Downgrade your Python environment to a version supported by `azureml-dataprep` (e.g., Python 3.11 or lower for SDK v1) or explicitly install `setuptools` using `pip install setuptools` if a specific Python 3.12+ version is required, though full compatibility isn't guaranteed.
Ensure all `azureml` SDK packages (`azureml-core`, `azureml-dataprep`, `azureml-dataset-runtime`, etc.) are updated to compatible versions within the same environment, often achieved by `pip install -U azureml-sdk` or by specifying a known working set of versions.
Add `azureml-dataprep[pandas,fuse]` to the `pip_packages` list within your Azure ML `Environment` definition for the remote compute target, for example: `conda_dep = CondaDependencies().add_pip_package("azureml-dataprep[pandas,fuse]")`.