torch-ort

library1.19.2pypypiunverified

torch-ort accelerates PyTorch models using ONNX Runtime. Version 1.19.2 supports PyTorch 2.x and ONNX Runtime 1.19.x. Released monthly in sync with ONNX Runtime.

ai-ml aws

pip install torch-ort

github ↗package ↗

INSTALL

IMPORT

SIG · TORCH-ORT

torch-ort

ai-mlpythonv1.19.2

Install

82.8s avg

Import

—

Disk

5786MB

Pass rate

2/ 10

Env Coverage2 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v1.19.2 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

glibc

py 3.10

✕ build_error

✓ 94.8s

py 3.11

✕ build_error

✓ 70.7s

py 3.12

✕ build_error

py 3.13

✕ build_error

py 3.9

✕ build_error

✕ timeout

5786MB installed

● package 5786MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

ORTModule

✓ from torch_ort import ORTModule

✗ from torch_ort import ORTModule as ORT

Common alias is not needed; just use ORTModule directly.

enable_training

✓ from torch_ort import enable_training

✗ from torch_ort import ort_enable_training

Function name changed in v1.10.0.

Wrap a PyTorch model with ORTModule to accelerate inference/training via ONNX Runtime.

import torch from torch_ort import ORTModule model = torch.nn.Linear(10, 5) ort_model = ORTModule(model) x = torch.randn(3, 10) output = ort_model(x) print(output)

Debug

Known issues

breakingBreaking change: torch-ort v1.14+ requires ONNX Runtime 1.14+. Older ONNX Runtime installations will cause import errors.

fix

Upgrade onnxruntime to 1.14+ or pin torch-ort to <1.14.0.

affects: >=1.14.0

deprecatedThe 'enable_training' function is deprecated since v1.15.0. Training mode is automatically detected.

fix

Remove calls to enable_training; ORTModule now auto-detects training vs inference.

affects: >=1.15.0

gotchaORTModule does not support all PyTorch operations. Custom autograd.Function subclasses may fail silently.

fix

Test model with ORTModule and fallback to native PyTorch if unsupported ops are encountered. Use ort_model.debug() to trace unsupported operators.

affects: all

gotchaModels must be in eval mode before conversion? No, ORTModule works in both train and eval, but state dict keys may change if you convert mid-training.

fix

Convert model _before_ training or after final eval to avoid key mismatch. Use model.load_state_dict on the original model, not the ORTModule-wrapped one.

affects: all

Upgrade

Version history

1.19.2latest on PyPI · released Sep 4, 2024

Audit

Dependencies

torchrequiredRequired for model export and execution.

onnxruntimerequiredRuntime for executing ONNX models.

Agent activity

37 hits · last 30 days

node

Amazon

OpenAI (training)

Resources

githubhttps://github.com/pytorch/ort ↗

packagehttps://pypi.org/project/torch-ort/ ↗