Install & Compatibility
Where this runs
tested against v1.2.6 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.920 runs
installs and imports cleanly · install 0.0s · import 0.000s · 17.8MB
glibcpy 3.10–3.920 runs
installs and imports cleanly · install 1.6s · import 0.000s · 18MB
16MB installed
● package 16MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
h3_pyspark
✓ import h3_pyspark
✗ import h3
This quickstart demonstrates how to initialize a SparkSession, create a DataFrame with geospatial coordinates, and use `h3_pyspark.geo_to_h3` to convert latitude and longitude to an H3 index. It also includes an example of `h3_pyspark.index_shape` for indexing GeoJSON polygons. Ensure `pyspark` is configured correctly for your environment.
from pyspark.sql import SparkSession, functions as F
import h3_pyspark
import os
# Initialize Spark Session (adjust master for your environment, e.g., 'local[*]'):
spark = SparkSession.builder.master(os.environ.get('SPARK_MASTER', 'local[*]')).appName("H3PySparkQuickstart").getOrCreate()
# Create a DataFrame with latitude, longitude, and desired H3 resolution
data = [{"lat": 37.769377, "lng": -122.388903, 'resolution': 9}]
df = spark.createDataFrame(data)
# Convert geographic coordinates to H3 index
df_with_h3 = df.withColumn('h3_index', h3_pyspark.geo_to_h3(F.col('lat'), F.col('lng'), F.col('resolution')))
df_with_h3.show()
# Example of an extension function: index_shape for GeoJSON polygons
geojson_polygon = "{\"type\":\"Polygon\",\"coordinates\":[[[-122.4,37.8],[-122.3,37.8],[-122.3,37.7],[-122.4,37.7],[-122.4,37.8]]]}"
polygon_df = spark.createDataFrame([{'id': 1, 'geometry': geojson_polygon, 'resolution': 9}])
polygon_h3_df = polygon_df.withColumn(
'h3_cells',
h3_pyspark.index_shape(F.col('geometry'), F.col('resolution'))
)
polygon_h3_df.show(truncate=False)
spark.stop()
Debug
Known issues
breakingThe underlying `h3-py` library (which `h3-pyspark` wraps) introduced significant breaking changes in its 4.x versions, primarily around function naming conventions (e.g., `kRing` became `gridDisk`) and error handling.fixRefer to the `h3-py` migration guide for changes between H3 v3.x and v4.x. Adapt your code to the new function names and error handling. Verify the version of `h3-py` installed alongside `h3-pyspark` to ensure compatibility.
affects: Users upgrading `h3-py` dependency to 4.x alongside h3-pyspark 1.x, or migrating code from `h3-py` 3.x to `h3-pyspark` 1.x with implicit `h3-py` 4.x.
gotchaPrior to version 1.2.4, `h3-pyspark` functions might not robustly handle null values in input columns to UDFs, potentially leading to errors or unexpected behavior.fixUpgrade to `h3-pyspark` version 1.2.4 or newer to benefit from improved null value handling. Ensure your input data is clean, or explicitly handle nulls (e.g., `na.drop()`, `fillna()`) before passing to H3 functions. [cite: `1.2.4` release notes]
affects: < 1.2.4
gotchaThe `index_shape` function in versions prior to 1.2.3 had a known bug where it might miss H3 cells for long line segments, leading to incomplete or inaccurate spatial indexing for complex geometries.fixUpgrade to `h3-pyspark` version 1.2.3 or newer, which includes a fix for this bug and improved error handling for malformed geometries. [cite: `1.2.3` release notes]
affects: < 1.2.3
gotchah3-pyspark assumes that geospatial geometries are represented as GeoJSON strings within a Spark DataFrame column, rather than other formats like WKT.fixEnsure that your geometry data is formatted as GeoJSON strings before passing it to functions like `h3_pyspark.index_shape`. Convert from other formats if necessary.
affects: All versions
Upgrade
Version history
1.2.6latest on PyPI · released Mar 10, 2022
Audit
Dependencies
pysparkrequiredProvides the Spark DataFrame API and execution environment for the H3 operations. This library is a binding to PySpark.
h3requiredThe core Python binding for the H3 geospatial indexing system, which h3-pyspark wraps and extends.