Discover more from ML Under the Hood
Fine-tuning for GPT-3.5
It will not teach your GPT new tricks, but can make it faster and more predictable.
Have you seen the news about the fine-tuning of GPT-3.5?
- This isn't the fine-tuning we're all used to!
- Training costs are relatively low - $0.008 per 1K tokens.
- Using the fine-tuned model is 8x more expensive than the base GPT-3.5-Turbo.
The news can be found here: OpenAI.
There's no additional cost for the uptime of the fine-tuned model. This suggests it's some variant of LoRA adapters and token manipulation.
This fine-tuning tailors the model for a specific task, making it more specialized. You can't teach it new facts easily, and it doesn't replace information retrieval. For more details, see OpenAI's documentation on fine-tuning.
Why go for such tuning? It's to save on prompt tokens! If there's a standard task that requires a lengthy prompt, a specific output format, or a certain response style, you can fine-tune GPT-3.5 for that task. This means you won't have to send as many few-shot examples in the request.
The tuning pays off when the input prompt is compressed by more than 8x. Plus, the response time will be faster.