Author: Lorenzo Massone Academic Advisor: Prof. Mirko Viroli Co-Advisor: Prof. Gianluca Aguzzi University of Bologna – Cesena Campus
This project demonstrates how to fine-tune Small Language Models (SLMs) for generating library-specific code—in particular, Akka/Scala code. The approach allows you to locally adapt general-purpose language models to generate code that is executable and tailored for specific frameworks, without requiring huge computational resources.
- Parameter-Efficient Fine-Tuning (PEFT): Uses techniques like LoRA to specialize only a small subset of model parameters.
- Akka/Scala Focus: Trains models to generate code patterns for the Akka actor framework.
- Lightweight Setup: Experiments show models can be fine-tuned on local hardware (e.g., laptops or Colab).
- Data Pipeline: Includes scripts for dataset preparation, fine-tuning, and automatic code validation.
- Llama3.2 3B
- Qwen 2.5 7B
- Fine-tuned Llama3.2 3B: 70% executable code (vs 0% for base)
- Fine-tuned Qwen 2.5 7B: 88% executable code