The PyTorch Basis, makers of the PyTorch machine studying framework, has launched torchao, a PyTorch native library that makes fashions quicker and smaller by leveraging low-bit dtypes, sparsity, and quantization. It’s a toolkit of strategies that span each coaching and inference, Workforce PyTorch mentioned.
Unveiled September 26, torchao works with torch.compile() and FSDP2 over most PyTorch fashions on Hugging Face. A library for customized information varieties and optimizations, torchao is positioned to make fashions smaller and quicker for coaching or inference out of the field. Customers can quantize and sparsify weights, gradients, optimizers, and activations for inference and coaching. The torchao library serves as an accessible toolkit of strategies principally written in easy-to-read PyTorch code spanning inference and coaching, in keeping with Workforce Pytorch. Featured is torchao.float8 for accelerating coaching with float8 in native PyTorch.
Launched below a BSD 3 license, torchao makes liberal use of latest options in PyTorch and is beneficial to be used with the present nightly or newest secure launch of PyTorch, Workforce PyTorch advises.
