Nvidia says its novel AI sound generator can produce sounds that have never been heard before

Nvidia says its novel AI music editor can create “sounds that have never been heard before” – like a meowing trumpet. A tool called Fugattois able to generate music, sounds, and speech using text and audio input that he has never been trained on.

As shown in this video posted belowthis allows Fugatto to compose songs based on wild prompts such as “Create saxophone howls, barks, then electronic music with dogs barking.”

Other examples provided by the company include the ability to create unique sound effects based on a description, such as “Deep, rumbling bass pulses coupled with intermittent, high-pitched digital chirps, like the sound of a huge sentient machine waking up.”

It can even change the sound of someone’s voice by changing their accent or giving it a different tone, such as enraged or serene. There are also ways to edit the music, as Fugatto can isolate vocals in a song, add instruments, and even change the melody by replacing the piano with an opera singer.

AND paper was published alongside the announcement was a long list of all the datasets Nvidia said Fugatto was trained on, one of which includes a library of sound effects from the BBC.

Nvidia says that to build Fugatto, researchers had to collect a dataset of millions of audio samples. They then created instructions “that greatly expanded the range of tasks the model could perform while providing more accurate performance and enabling new tasks without the need for additional data.” Nvidia does not say when – or if – the tool will be widely available.