|
1 | | -# TorchAO Claude Instructions |
| 1 | +# TorchAO |
2 | 2 |
|
3 | | -Fill me in |
| 3 | +PyTorch-native library for quantization, sparsity, and low-precision training. Works with `torch.compile()` and `FSDP2`. |
| 4 | + |
| 5 | +## Config Classes |
| 6 | + |
| 7 | +All configs inherit from `AOBaseConfig`. Defined in `torchao/quantization/quant_api.py`: |
| 8 | + |
| 9 | +| Config | Description | |
| 10 | +|--------|-------------| |
| 11 | +| `Int4WeightOnlyConfig` | int4 weight-only (most common for inference) | |
| 12 | +| `Int8WeightOnlyConfig` | int8 weight-only | |
| 13 | +| `Int8DynamicActivationInt8WeightConfig` | int8 weights + int8 dynamic activations | |
| 14 | +| `Int8DynamicActivationIntxWeightConfig` | int8 activations + arbitrary int weight width | |
| 15 | +| `Float8WeightOnlyConfig` | float8 weight-only | |
| 16 | +| `Float8DynamicActivationFloat8WeightConfig` | float8 weights + float8 dynamic activations | |
| 17 | +| `Float8DynamicActivationInt4WeightConfig` | float8 activations + int4 weights | |
| 18 | +| `IntxWeightOnlyConfig` | arbitrary bit-width for edge/ExecuTorch | |
| 19 | +| `FqnToConfig` | map module names to different configs for per-layer quantization | |
| 20 | + |
| 21 | +### Prototype configs (in `torchao/prototype/mx_formats/`) |
| 22 | +- `MXDynamicActivationMXWeightConfig` - MXFP8/MXFP4 (H100/B200/MI350x) |
| 23 | +- `NVFP4DynamicActivationNVFP4WeightConfig` - NVIDIA FP4 (B200 Blackwell only) |
| 24 | +- `NVFP4WeightOnlyConfig` - NVFP4 weight-only (B200 Blackwell only) |
| 25 | + |
| 26 | +## Stable vs Prototype |
| 27 | + |
| 28 | +- **Stable** (`torchao/quantization/`, `torchao/float8/`, `torchao/sparsity/`, `torchao/optim/`): API stability guaranteed. Breaking changes go through deprecation cycle. |
| 29 | +- **Prototype** (`torchao/prototype/`): Experimental features, API may change without notice. Includes: `mx_formats/` (MXFP8, MXFP4, NVFP4), `moe_training/` (MoE mixed-precision), `awq/`, `hqq/`, `autoround/`, `quantized_training/`. |
| 30 | + |
| 31 | +## Architecture and Contributing |
| 32 | + |
| 33 | +For architecture details, tensor subclass design, and contributor guides, see the in-repo docs: |
| 34 | +- [Quantization Overview](docs/source/contributing/quantization_overview.rst) - full stack walkthrough, tensor subclasses, quantization flows |
| 35 | +- [Contributor Guide](docs/source/contributing/contributor_guide.rst) - how to add tensors, kernels, configs |
| 36 | +- [Workflows Matrix](docs/source/workflows/index.md) - dtype x hardware status table |
| 37 | +- [PT2E Quantization](docs/source/pt2e_quantization/index.rst) - PyTorch 2 Export quantization for deployment backends (X86, XPU, ExecuTorch) |
| 38 | + |
| 39 | +These same files render at https://docs.pytorch.org/ao/main/contributing/index.html |
| 40 | + |
| 41 | +## Deprecated APIs |
| 42 | + |
| 43 | +Do not use or recommend these: |
| 44 | +- `AffineQuantizedTensor` (AQT) in `torchao/dtypes/` - old v1 system, being removed. New tensor types inherit from `TorchAOBaseTensor` in `torchao/utils.py` |
| 45 | +- `autoquant()` - deleted |
| 46 | +- Layout registration system (`PlainLayout`, `Float8Layout`, `TensorCoreTiledLayout`, etc.) - deleted |
| 47 | +- `TorchAODType` - deprecated |
| 48 | +- `change_linear_weights_to_int4_woqtensors` - deleted, use `quantize_(model, Int4WeightOnlyConfig())` |
| 49 | + |
| 50 | +## Development |
| 51 | + |
| 52 | +```bash |
| 53 | +# Setup |
| 54 | +USE_CPP=0 pip install -e . --no-build-isolation # CPU-only |
| 55 | +USE_CUDA=1 pip install -e . --no-build-isolation # With CUDA |
| 56 | + |
| 57 | +# Lint (ruff v0.11.6, rules: F and I) |
| 58 | +ruff check --fix && ruff format . |
| 59 | + |
| 60 | +# Test (mirrors source structure) |
| 61 | +pytest test/quantization/test_quant_api.py |
| 62 | +pytest test/float8/ |
| 63 | +pytest test/prototype/mx_formats/ |
| 64 | +``` |
| 65 | + |
| 66 | +## Coding Style |
| 67 | + |
| 68 | +- 2 spaces for indentation |
| 69 | +- 80 character line length |
| 70 | +- BSD 3-Clause license header required on all source files |
| 71 | +- Match existing patterns in the file you're editing |
| 72 | + |
| 73 | +## Commit Messages |
| 74 | + |
| 75 | +- Do not commit without explicit request from the user |
| 76 | +- Preserve ghstack trailers when amending commits |
| 77 | +- If Claude or another AI tool was used, disclose in the commit message |
0 commit comments