OpenAI has announced that developers can now fine-tune its GPT-3.5 Turbo model using custom data.
Fine-tuning is the process of taking the base version of a model like GPT-3.5 Turbo, which underpins ChatGPT, and training it on a very specific custom set of data.
The base model of GPT-3.5 Turbo is pretty good at doing a lot of different things. Once a developer fine-tunes it on a data set focused on a specific use case, it becomes really good at performing a very specific set of tasks. It’s like taking a general handyman and training him to be a really good plumber.
Companies can now train a version of ChatGPT using GPT-3.5 Turbo so that it delivers outputs that are tailored to their business.
For example, a business might want all of ChatGPT’s responses to be in a specific language or consistently deliver shorter or longer responses. The responses can also be trained to always output in a specific format or to follow a consistent tone of corporate communication.
If you want to use ChatGPT to consistently deliver a specific type of code completion or just focus on composing API calls, you can now train it to do that with GPT-3.5 Turbo.
Up until now, developers have only been able to fine-tune less capable legacy variants of GPT-3. Using a fine-tuned version of GPT-3.5 Turbo delivers much better performance.
In its announcement, OpenAI said that “Early tests have shown a fine-tuned version of GPT-3.5 Turbo can match, or even outperform, base GPT-4-level capabilities on certain narrow tasks.”
GPT-3.5 Turbo has a 4k token context window which is twice that of previous fine-tunable models. A 16k version of the model will be made available for fine-tuning later this year.
We’ve just launched fine-tuning for GPT-3.5 Turbo! Fine-tuning lets you train the model on your company’s data and run it at scale. Early tests have shown that fine-tuned GPT-3.5 Turbo can match or exceed GPT-4 on narrow tasks: https://t.co/VaageW9Kaw pic.twitter.com/nndOyxS2xs
— OpenAI (@OpenAI) August 22, 2023
It’s not free, but it’s getting cheaper
OpenAI is still charging for API calls to its models but being able to fine-tune the model can reduce the overall costs of running the model.
The cost of using GPT-3.5 is $0.008/1k tokens for training, $0.012/1k tokens for input usage, and $0.016/1k tokens for output usage.
The amount of tokens in the prompts you send and the responses you get will determine the roundtrip cost. By fine-tuning a model you can train it to respond to shortened prompts. OpenAI says that some early testers were able to reduce prompt lengths by up to 90%.
This reduction in the amount of tokens sent reduces the associated costs but also makes the API calls a lot faster. The opportunity for cost reduction through fine-tuning is crucial for OpenAI as it competes with free models like Llama 2.
Even though OpenAI charges for API calls, the efficiencies and improved performance can make it cheaper to run than open-source free models like Llama 2. Free-to-use models still have to run somewhere and computing costs aren’t cheap.
The move to make GPT-3.5 Turbo available for fine-tuning is also timely considering how Microsoft has been making moves to pitch competing AI models on its Azure platform.
And as corporate concerns over ChatGPT data security persist, OpenAI was at pains to reaffirm that “As with all our APIs, data sent in and out of the fine-tuning API is owned by the customer and is not used by OpenAI, or any other organization, to train other models.”
Whether that will be enough to convince the skeptics remains to be seen. If it comes down to processing costs and performance then this latest move will see OpenAI’s offering become a more appealing prospect than even the leading free AI models.
