PyFlink is the Python API for Apache Flink, a powerful open-source stream-processing framework. It enables users to write Flink jobs using Python's DataStream and Table APIs, leveraging Flink's robust capabilities for stateful computations, fault tolerance, and high throughput. Version 2.2.0 is compatible with Flink 1.18+ and requires Python 3.9+. New releases generally align with major Flink core updates and maintain a consistent release cadence.
pip install apache-flinkVerified import paths — ran on the pinned version, not inferred.
This quickstart demonstrates a simple PyFlink Table API job. It creates a streaming table environment, defines an in-memory source, performs a group-by aggregation, and prints the results to the console. This showcases basic setup, data ingestion, transformation, and execution for local development.
Upgrade your Python environment to 3.9+ or use an older PyFlink version compatible with your Python interpreter.
Refer to the official PyFlink documentation for the current Table API patterns, specifically for defining `TableDescriptor` and `Schema` directly under `pyflink.table` and using format classes from `pyflink.table.formats`.
Always use `import pyflink` or `from pyflink import ...` when importing components from the library.
Download the necessary Flink connector JARs from Maven Central or the official Flink downloads page and make them available to your Flink cluster (e.g., via `bin/flink run -pyfs ... -j ...` or by placing them in the `lib` directory of your Flink installation).
Ensure `java` is in your system's PATH. For local development, download a Flink binary distribution and set the `FLINK_HOME` environment variable, or use `StreamExecutionEnvironment.create_local_environment()` which can automatically manage a mini-cluster.
The correct import name is `pyflink`. Use `import pyflink` or `from pyflink.table import TableEnvironment` etc.
Download the required Flink connector JAR (e.g., `flink-connector-kafka_2.12-1.18.jar`) from Maven Central or Flink's download page. When running your job, specify the JAR using the `-j` flag: `bin/flink run -pyfs your_job.py -j path/to/flink-connector-kafka.jar`.
Review your `TableDescriptor` and `Schema` definitions. Ensure all required properties for the source/sink format are correctly set and that the schema matches the data. Check Flink's JobManager logs for more detailed error messages.
Upgrade your Python environment to 3.9 or higher. If using a virtual environment, ensure it's created with a compatible Python version.