torchscale

library0.3.0pypypiunverified

torchscale is a PyTorch library for building large-scale Transformer models, providing components like Multi-head Attention (MHA), Long Short-Term Memory (LSTM), and other scalable architectures. As of version 0.3.0, it supports Python >=3.8 and is maintained by Microsoft. Releases are infrequent.

ai-ml llm-agents

pip install torchscale

github ↗package ↗

INSTALL

IMPORT

SIG · TORCHSCALE

torchscale

ai-mlpythonv0.3.0

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

MHA

✓ from torchscale.component import MHA

✗ from torchscale.component import MHA

Initialize MHA and IncrementalDecoder with random input.

import torch from torchscale.component import MHA from torchscale.model import IncrementalDecoder # Example: Multi-head attention mha = MHA(embed_dim=512, num_heads=8) x = torch.randn(4, 10, 512) output = mha(x, x, x) print(output.shape) # Example: Decoder decoder = IncrementalDecoder( vocab_size=1000, embed_dim=512, num_heads=8, num_layers=6, ) tokens = torch.randint(0, 1000, (4, 20)) logits = decoder(tokens) print(logits.shape)

Debug

Known issues

deprecatedThe `torchscale.model.LongShortTerm` class is deprecated in 0.3.0; use `IncrementalDecoder` or `TemporalDecoder` instead.

fix

Replace `LongShortTerm` with `IncrementalDecoder` or `TemporalDecoder` depending on use case.

affects: >=0.3.0

breakingIn version 0.3.0, the `MHA` class no longer accepts `kdim` and `vdim` arguments; use `embed_dim` for all.

fix

Remove `kdim` and `vdim` from MHA constructor and ensure all dimensions match `embed_dim`.

affects: >=0.3.0

gotchatorchscale components expect batch-first tensors (batch, seq, dim), not sequence-first. Incorrect ordering may cause shape mismatches.

fix

Ensure input tensors have shape (batch, sequence, features) or use .transpose() if needed.

affects: all

Upgrade

Version history

0.3.0latest on PyPI · released Oct 20, 2023

Audit

Dependencies

torchrequiredCore dependency for all tensor operations and models.

einopsoptionalUsed for tensor rearrangement in attention modules.

transformersoptionalOptional for loading Hugging Face model weights.

Agent activity

30 hits · last 30 days

node

OpenAI (training)

Resources

githubhttps://github.com/microsoft/torchscale ↗

packagehttps://pypi.org/project/torchscale/ ↗