sparkxgb

library0.4pypypiunverified

sparkxgb is a Python wrapper for XGBoost on Apache Spark, providing integration utilities for distributed training and prediction on Spark DataFrames. Version 0.4 is stable but infrequently updated; rely on official XGBoost Spark integration for newer APIs.

ai-ml data

pip install sparkxgb

package ↗

INSTALL

IMPORT

SIG · SPARKXGB

sparkxgb

ai-mlpythonv0.4

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

XGBoostEstimator

✓ from sparkxgb import XGBoostEstimator

Train an XGBoost classifier on a Spark DataFrame with feature vector column.

from pyspark.sql import SparkSession from sparkxgb import XGBoostClassifier spark = SparkSession.builder.appName('sparkxgb-example').getOrCreate() df = spark.createDataFrame([(1.0, [0.1, 0.2]), (0.0, [0.3, 0.4])], ['label', 'features']) classifier = XGBoostClassifier(max_depth=3, num_round=10) model = classifier.fit(df, params={'eval_metric': 'logloss'}) predictions = model.transform(df) predictions.show()

Debug

Known issues

breakingsparkxgb 0.4 requires PySpark <3.0 due to API changes in Spark 3.0+. Check your Spark version.

fix

Downgrade PySpark to 2.4.x or use XGBoost's built-in Spark integration (xgboost >=1.7).

affects: 0.4

deprecatedsparkxgb is no longer actively maintained; consider migrating to xgboost.spark (available from XGBoost 1.7+).

fix

Replace imports: from xgboost.spark import SparkXGBClassifier

affects: 0.4

gotchaFeature column must be of type VectorUDT (e.g., from VectorAssembler). Passing raw arrays fails silently with wrong predictions.

fix

Always use VectorAssembler to create the 'features' column: from pyspark.ml.feature import VectorAssembler; assembler = VectorAssembler(inputCols=[...], outputCol='features')

affects: all

Upgrade

Version history

0.4latest on PyPI · released Oct 8, 2025

Audit

Dependencies

pysparkrequiredRequired for Spark DataFrame operations

xgboostrequiredCore XGBoost library

Agent activity

9 hits · last 30 days

node

OpenAI (training)

Resources

packagehttps://pypi.org/project/sparkxgb/ ↗