Modern large language models (LLMs) rely heavily on mathematical reasoning, which is the primary focus of this work. There is a clear divide between closed-source and open-source LLMs, even with the recent progress in this area; closed-source models like GPT-4, PaLM-2, and Claude 2 dominate popular mathematical reasoning benchmarks like GSM8K and MATH, while open-source models like Llama, Falcon, and OPT fall far behind.
There are two main approaches to closing this gap:
- Ongoing pre-training, like with Galactica and MINERVA, which is now training an LLM on more than 100B tokens of web data linked to mathematics. Although it is computationally expensive, this method increases a model’s capacity for scientific reasoning in general.
- Using trained data unique to each dataset, fine-tuning methods such as rejection sampling fine-tuning (RFT) and WizardMath are used to perfect LLMs. While these methods are effective within their domain, they are not transferable to other areas of mathematics where reasoning is required.
Recent research by the University of Waterloo, the Ohio State University, HKUST, the University of Edinburgh, and IN.AI explore a lightweight, yet generalizable, math instruction-tuning technique to improve LLMs’ mathematical reasoning abilities in general (i.e., not just the fine-tuning tasks).
Read More Here.
