Registry / database / django-postgres-copy

django-postgres-copy

JSON →
library2.8.0pypypi✓ verified 24d ago

django-postgres-copy is a Django package that facilitates fast and efficient bulk import and export of delimited data (like CSV files) to and from PostgreSQL databases, leveraging PostgreSQL's native `COPY` command. It is significantly faster than using Django's ORM for large datasets. The library is actively maintained, currently at version 2.8.0, and regularly updates to support recent Django and Python versions.

pip install django-postgres-copy
INSTALL
IMPORT
SIG · DJANGO-POSTGRES-CO
D
django-postgres-copy
databasepythonv2.8.0
Install
1.5s avg
Import
Disk
16MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v2.8.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.95 runs
installs and imports cleanly · install 0.0s · import 0.000s · 17.9MB
glibc
py 3.103.95 runs
installs and imports cleanly · install 1.5s · import 0.000s · 18MB
16MB installed
● package 16MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

CopyManager
from postgres_copy import CopyManager
from postgres_copy import CopyManager

To use `django-postgres-copy`, first attach `CopyManager` to your Django model. You can then use the `from_csv()` method on your model's manager to import data from a CSV file, providing a path to the file and a dictionary mapping model fields to CSV headers. Similarly, `to_csv()` exports data. The library handles creating temporary tables and inserting data efficiently.

import os from django.db import models from postgres_copy import CopyManager # Define a simple Django model with CopyManager class Person(models.Model): name = models.CharField(max_length=500) number = models.IntegerField(null=True) date = models.DateField(null=True) objects = CopyManager() class Meta: app_label = 'myapp' # Replace with your app's name # Example CSV content for 'data.csv' # name,number,date # ben,1,2024-01-01 # joe,2,2024-01-02 # jane,3,2024-01-03 # Create a dummy CSV file for the example (in a real scenario, this would be an existing file) csv_content = "name,number,date\nben,1,2024-01-01\njoe,2,2024-01-02\njane,3,2024-01-03" with open("data.csv", "w") as f: f.write(csv_content) # --- Import data from CSV --- # In a real Django setup, ensure your database connection is configured and migrations are run. # Example: Person.objects.from_csv( # "./data.csv", # dict(name="name", number="number", date="date") # Mapping model fields to CSV headers # ) print("To import from CSV:") print("Person.objects.from_csv('./data.csv', dict(name='name', number='number', date='date'))") # --- Export data to CSV (assuming some data exists in the model) --- # Example: Person.objects.to_csv("./export.csv") print("\nTo export to CSV:") print("Person.objects.to_csv('./export.csv')") # Clean up dummy file os.remove("data.csv") # if os.path.exists("export.csv"): os.remove("export.csv") # Uncomment if testing export
Debug
Known issues
gotchaThe library added support for `psycopg` (v3) in version 2.8.0, while maintaining `psycopg2` compatibility. However, `psycopg3` introduces several breaking changes compared to `psycopg2` in its underlying API. Although `django-postgres-copy` has a compatibility layer, users with existing direct `psycopg2` database interactions or complex connection pooling setups may need to adjust their project configuration or code during migration to `psycopg3`.
fix
Review `psycopg3` documentation for breaking changes if migrating from `psycopg2`. Ensure your project's database configuration is compatible with the `psycopg` version installed.
affects: 2.8.0+
breaking`django-postgres-copy` is regularly updated for Django version compatibility. Older versions of `django-postgres-copy` might not be fully compatible with newer Django versions, or vice-versa. For example, version 2.7.3 included a fix for 'Django 4.2+ compatible `setup_query`'. Always check the PyPI classifiers or GitHub releases for specific Django version support when upgrading either library.
fix
Upgrade `django-postgres-copy` to the latest version compatible with your Django version. Consult the project's `pyproject.toml` or release notes for specific compatibility information.
affects: <2.7.3 with Django 4.2+
gotchaThe library allows for powerful SQL-based data transformations during import using methods like `copy_field_template` or `setup_query`. While flexible, if these SQL statements are constructed using unsanitized user-provided input, they can introduce SQL injection vulnerabilities.
fix
Always sanitize or strictly validate any user-provided input used to construct dynamic SQL statements within `copy_field_template` or `setup_query` methods. Prefer fixed, predefined transformations where possible.
affects: All versions
gotchaBy default, for performance optimization during large imports, the `from_csv` method (via `CopyMapping`) temporarily drops database constraints and indexes, then recreates them. While this speeds up the process, it means data integrity checks are momentarily suspended, which could potentially expose data to inconsistencies if an unexpected error occurs during the import process.
fix
Be aware of this behavior for critical data. Consider performing imports within a robust transaction (though the library itself may operate outside one for performance) or backing up your database before large-scale imports. The `drop_constraints` and `drop_indexes` parameters can be set to `False` if strict integrity is needed throughout, though with a performance penalty.
affects: All versions
gotchaWhen importing data that includes auto-incrementing primary keys (like Django's default `id` field) using `COPY FROM`, PostgreSQL's sequence generator for that table is not automatically updated. If you subsequently insert new records via Django's ORM, `IntegrityError` or `UniqueViolation` exceptions may occur if the ORM tries to insert an ID that conflicts with a manually `COPY`-inserted ID.
fix
If importing custom primary keys, consider manually updating the PostgreSQL sequence for the table after the `from_csv` operation, or set the `columns` parameter in `from_csv` to exclude the primary key column to let PostgreSQL assign it.
affects: All versions
Upgrade
Version history
2.8.0latest on PyPI · released Jun 22, 2025
Audit
Dependencies
DjangorequiredCore framework dependency. Compatible with Django 4.2, 5.1, 5.2.
psycopgrequiredPostgreSQL adapter. Version 2.8.0 introduced support for `psycopg` (v3), alongside continued compatibility with `psycopg2`. `psycopg[binary]` or `psycopg[c]` are common installation choices.
psycopg2requiredPostgreSQL adapter (older version). Still supported but `psycopg` (v3) is recommended for new projects and newer Django versions.
Agent activity
10 hits · last 30 days
node
8
OpenAI (training)
1
Resources
django-postgres-copy — pip install django-postgres-copy · libregistry