"You have to show [an AI model] tens
of thousands of examples of good behavior, and you have to fine-tune those into
the model," Suleyman said. "The
good news is that tens of thousands of examples are very accessible to many
niche domains or specific verticals. So that's an edge and I think there's
plenty of room for startups in doing high-quality fine-tuning of a pre-trained
model."
Small AI models will be the future of AI,
according to Suleyman.
"We're going to sort of compress
knowledge into smaller, cheaper models, which can live on a fridge
magnet," he said. Training a large AI model currently takes about $100 million, with more advanced
models expected to cost billions of dollars. The data that goes into training
these models is controversial though, with many copyright lawsuits pending
against companies like OpenAI.
In June, Suleyman
answered the question of whether AI companies have taken the world's
intellectual property for their own gain. He stated then that almost all
content on the Internet, except for news sites and publishers that have asked
not to be crawled, is open to AI training. "I think that
with respect to content that is already on the open web, the social contract of
that content since the '90s has been that it is fair use," he said at the time.