bitsandbytes
quantization library
Quantization library for training and inference at reduced precision
Supported languages
Pros and Cons
Ventajas
- + Significantly reduces memory usage
- + Enables training large models
- + HuggingFace integration
- + Easy to use
Desventajas
- - NVIDIA GPUs only
- - May affect precision
Casos de Uso
- QLoRA fine-tuning
- Limited memory inference
- Large model training