Install & Compatibility
Where this runs
tested against v0.32.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.95 runs
installs and imports cleanly · install 0.0s · import 0.018s · 440.5MB
glibcpy 3.10–3.95 runs
installs and imports cleanly · install 12.8s · import 0.012s · 398MB
409MB installed
● package 409MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
koalas
✓ import databricks
✗ import databricks.koalas as ks
This quickstart demonstrates how to create a Koalas DataFrame from a pandas DataFrame, perform a basic operation (calculate mean), and convert it back to a pandas DataFrame. Ensure you have PySpark configured in your environment for this to run against a Spark session.
import databricks.koalas as ks
import pandas as pd
# Create a pandas DataFrame
pdf = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]})
# Convert to a Koalas DataFrame
kdf = ks.DataFrame(pdf)
print("Koalas DataFrame head:")
print(kdf.head())
print("Mean of column 'A':", kdf['A'].mean())
# Convert back to a pandas DataFrame
pdf_result = kdf.to_pandas()
print("\nConverted back to pandas DataFrame:")
print(pdf_result)
Debug
Known issues
breakingKoalas as a standalone library is deprecated. All its functionality has been officially integrated into PySpark as 'pandas API on Spark' starting with Apache Spark 3.2. Users are strongly advised to migrate to PySpark directly.fixFor Apache Spark 3.2 and later, use `import pyspark.pandas as ps` instead of `import databricks.koalas as ks`. Migrate existing Koalas code to use `pyspark.pandas`.
affects: All versions, especially 1.8.0 and above. Affects users of Apache Spark 3.2+.
breakingThe default plotting backend for Koalas switched from Matplotlib to Plotly in version 1.7.0. This can change the visual output and require different plotting options.fixReview existing plotting code. If you prefer Matplotlib, you might need to explicitly set the backend, e.g., `ks.set_option('plotting.backend', 'matplotlib')` or adapt to Plotly's capabilities. affects: 1.7.0 and later
gotchaKoalas historically had different behavior than pandas regarding unnamed Series. Prior to v1.2.0, Koalas would automatically name a Series '0' if no name was specified, unlike pandas which allows a truly unnamed Series. This was fixed in v1.2.0 to align with pandas.fixEnsure your code explicitly handles Series names if consistency is critical, or upgrade to Koalas 1.2.0+ for pandas-like unnamed Series behavior.
affects: Prior to 1.2.0
gotchaCompatibility with specific pandas versions can introduce subtle bugs. For example, Koalas 1.8.2 addressed an issue with `_builtin_table` import in `groupby.apply` that affected pandas versions 1.3.0 and above.fixAlways check release notes for specific pandas version compatibility. Keep Koalas updated to the latest available version if you are using recent pandas versions.
affects: May vary depending on pandas version. Specifically, pandas >=1.3.0 with Koalas <1.8.2.
gotchaEarly versions of Koalas (pre-1.5.0) had limited or inconsistent support for complex Index operations (e.g., chained arithmetic operations), sometimes raising `AssertionError`.fixUpgrade to Koalas 1.5.0 or later for improved Index operation support, or refactor complex index manipulations to simpler steps in older versions.
affects: Prior to 1.5.0
Upgrade
Version history
1.8.2latest on PyPI · released Oct 19, 2021
Audit
Dependencies
pysparkrequiredRequired for Koalas to function, as it runs on Apache Spark.
pandasrequiredProvides the API Koalas implements; ensures compatibility with pandas data structures and operations.