Have you ever wondered how your voice would sound if it came out of a musical instrument like a flute or violin? Google’s team–Magenta–explored the possibility of using machine learning to convert your voice into music with Tone Transfer.
Leveraging Differential Digital Signal Processing (DDSP), the team developed an open-source technology to convert your voice into music. “DDSP allows developers to combine interpretable structure of classical DSP elements (such as filters, oscillators, reverberation, etc.) with the expressivity of deep learning,” noted Magenta.
With DDSP, the researchers created complex realistic signals by controlling various parameters, which was done by training neural networks to adapt to a dataset through standard backpropagation. This novel approach was also presented in ICLR 2020, where the researcher demonstrated the conversion of the sound of Voice → Violin and Violin → Flute.
- D-Wave Launches 5,000+ Qubit Quantum Computing Platform
- Top AI Conferences That You Should Look Ahead In 2020
- Amazon Launches Amazon One For Contactless Payment
DDSP learns by extracting different characters of a musical instrument and maps it with different sounds, thereby creating appealing music. You can use a wide range of audio right from someone singing to dog’s barking and people talking to convert them into a sound from violin, flute or more.
Google last week released a portal where you can upload or record your audio and then transform it into a wide range of sounds of instruments.
Since researchers used western music to train the model, it might give unexpected results. However, DDSP transforms sound by modeling frequencies in the audio, ignoring the standards of western music. This reduces the possibility of distorted or undesired sound.
You can use the tool–Tone Transfer–to listen to how your voice sounds when matched with the sound of instruments. You can also download the audio to share across your network.
Besides, you can use Magenta, a distributed open-source library for Python and JavaScript, to experiment and make your own machine learning models while playing with music to generate resounding audio.
Watch the video.
