ML Under the Hood

Share this post

Fine-tuning for GPT-3.5

abdullin.substack.com

Discover more from ML Under the Hood

Let's talk about ChatGPT and building ML-driven software products. You'll hear from me once a month: relevant industry news, interesting cases and lessons learned.
Continue reading
Sign in

Fine-tuning for GPT-3.5

It will not teach your GPT new tricks, but can make it faster and more predictable.

Aug 23, 2023
Share this post

Fine-tuning for GPT-3.5

abdullin.substack.com
Share

Have you seen the news about the fine-tuning of GPT-3.5?

In summary:

- This isn't the fine-tuning we're all used to!
- Training costs are relatively low - $0.008 per 1K tokens.
- Using the fine-tuned model is 8x more expensive than the base GPT-3.5-Turbo.

The news can be found here: OpenAI.

There's no additional cost for the uptime of the fine-tuned model. This suggests it's some variant of LoRA adapters and token manipulation.

This fine-tuning tailors the model for a specific task, making it more specialized. You can't teach it new facts easily, and it doesn't replace information retrieval. For more details, see OpenAI's documentation on fine-tuning.

Why go for such tuning? It's to save on prompt tokens! If there's a standard task that requires a lengthy prompt, a specific output format, or a certain response style, you can fine-tune GPT-3.5 for that task. This means you won't have to send as many few-shot examples in the request.

The tuning pays off when the input prompt is compressed by more than 8x. Plus, the response time will be faster.

This also pushes libraries like Microsoft Guidance or Microsoft TypedChat further into obsoletion.

Share this post

Fine-tuning for GPT-3.5

abdullin.substack.com
Share
Comments
Top
New
Community

No posts

Ready for more?

© 2023 Rinat Abdullin
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing