This paper explores using reinforcement learning (RL) to enhance reasoning in small language models (LLMs) under strict resource limitations. The authors adapted the Group Relative Policy Optimization (GRPO) algorithm and curated a focused mathematical reasoning dataset to train a 1.5-billion-parameter model. Their experiments demonstrated that even with limited data and computational power, significant gains in mathematical reasoning accuracy could be achieved, sometimes surpassing larger, more expensive models. However, challenges like optimization instability and managing output length emerged with prolonged training. Ultimately, the study highlights RL-based fine-tuning as a promising, cost-effective approach for improving reasoning in resource-constrained small LLMs.

Sources: https://arxiv.org/abs/2503.16219