Fine Tuning with Unsloth

Overview

• Unsloth allows fine-tuning of models (Mistral, Gemma, Lama 2) 5x faster with 70% less memory.
• 0% loss in accuracy, compatibility with Linux and Windows (WSL).
• Supports 4-bit, 16-bit, Q LoRa, and LoRa fine-tuning.
• Unsloth is twice as efficient as Hugging Face.

Goals of the Lecture

• Fine-tune the Mistral 7 billion parameter model.
• Compare model responses before and after fine-tuning.
• Use the OIG dataset for instruction follow-up.

Steps in the Process

1. Load Data

   • Use the OIG dataset (JSON-L format).
   • Example of dataset format:
     • Text key with human tag and question.
     • Data split: 90% training, 10% testing.

2. Load the Model

   • Define max sequence length.
   • Use fast language model with a 4-bit quantized version for efficiency.

3. Before Training Comparison

   • Create a function to generate text using previous model settings.
   • Example question: "What are the tips for a successful business plan?"

4. Install Required Packages

   • Install Unsloth colab package for necessary modules.

5. Training the Model

   • Model patching with fast lower weights.
   • Define SFT trainer for supervised fine-tuning.
   • Specify optimizer and training parameters (e.g., maximum steps).
   • Train the model and observe performance.

6. Saving the Model

   • Save the merged model (16-bit) for convenience.
   • Difference between adapter and merged model:
     • Adapter requires base model load each time.
     • Merged model combines both for easier use.
   • Utilize model.push_to_hub to upload to Hugging Face.

7. Testing the Fine-Tuned Model

   • Load the merged version of the model.
   • Generate text with the same question and check improvements.

Results

• Fine-tuned model outputs in point format as expected after training.
• Successfully uploaded the model to Hugging Face.

Conclusion

• Fine-tuning with Unsloth is efficient and reduces resource requirements.
• Encouragement to subscribe for more AI content.