Understanding Small Language Models (SLMs)

Introduction

• Host: Ana Hoffman
• Guest: Muzma, leader on the SQL team
• Topic: Small Language Models (SLMs) and their integration with Azure SQL database

Key Concepts

• SLMs (Small Language Models)
  • New family of models announced by Microsoft
  • Example: Microsoft's Phi-3 model
    • Phi-3 Mini, Small, and Medium variants
    • Phi-3 is a 3.8 billion parameter model

Considerations for SLMs

• Comparison with Large Language Models (LLMs)
  • Latency: SLMs offer lower latency, especially when hosted close to data
  • Cost: Generally cheaper due to lesser compute requirements
  • Sustainability: Lower carbon footprint
  • Flexibility: Lightweight, can be hosted on mobile devices

Working with SQL Database

• Hosting and Fine-tuning

  • Fine-tuning involves using Python scripts to customize the model with specific datasets
  • Results of fine-tuning can be stored back in SQL database

• Architecture

  • Use of orchestration engines like Ollama to host fine-tuned models
  • Example of integration with models like Mistral-7B

Demonstration

• Phi-3 Model Fine-tuned on SQL Documentation
  • Used open web UI with Docker-based front-end running on Olama
  • Demonstrated using a question about Azure SQL database

Implementation

• Fine-tuning Process
  • Complex and requires understanding of model workings
  • Once set up, deployment and running is quick, less than five minutes
  • Configuration involves model and adapter files, along with parameters such as model temperature

Getting Started

• Resources
  • Links:
    • aka.ms/azuresql-slm
    • aka/sqlai
    • sqlsamples

Conclusion

• SLMs are an exciting new frontier with benefits in latency, cost, and sustainability
• Potential for fine-tuning to specific datasets and quick deployment
• Encouragement to explore resources and get started with learning and implementing SLMs

---

• To Viewers: Engage with the episode by liking and commenting. Check links in the description for further learning.