Microsoft Orca
llm-model model
Small models that learn from large models
Supported languages
Orca is a family of Microsoft Research models that learn through imitation of larger models like GPT-4. They use explanation tuning techniques to transfer reasoning capabilities to smaller, more efficient models.
Concepts
explanation-tuningsynthetic-dataknowledge-distillationreasoning-traceschain-of-thought
Pros and Cons
Ventajas
- + Excellent reasoning for its size
- + Learns from GPT-4 thought traces
- + Multiple sizes available
- + Backed by Microsoft Research
- + Innovative training techniques
- + Good benchmark performance
Desventajas
- - Dependent on GPT-4 synthetic data
- - Limited by teacher's knowledge
- - Less versatile than general models
- - Variable weight availability
Casos de Uso
- Efficient logical reasoning
- Step-by-step problem solving
- Analysis assistants
- Math tasks
- Concept explanation