WizardLM: Fully Open-source Automated Instruction Data Generator

Automate tedious steps of instruction-based training data generation

TLDR

Instruction tuning on open-domain LLMs (LLaMA, MPT, Falcon) has worked fantastically! But manually creating instruction data is really time-consuming and humans are lazy and are not consistent. Evol-Instruct is a methodology to develop complex instructions (evolving instructions from less complex to more). WizardLM was trained using a dataset generated using Evol-Instruct. And…