We want to hear from you. Take our short AI survey and let us know your thoughts about the current state of AI, how you’re implementing it, and your hopes for the future. learn more
Eleven LabThe AI voice startup known for its voice cloning, text-to-speech and speech-to-text models has added a new tool to its product portfolio. AI Voice Isolator.
Available today on the ElevenLabs platform, the service enables creators to remove unwanted background noise and sounds from any content, from movies to podcasts to YouTube videos.
This comes just a few days after the company released its Reader app, which is free to use (with some limitations). However, it’s important to note that this feature is not entirely new to the market; many other creative solution providers have Including Adobethere are tools available to improve the quality of the voices in your content. The only question that remains is how effective Voice Isolator is in comparison to them.
How does AI Voice Isolator work?
When recording content such as movies, podcasts, or interviews, creators often face the problem of background noise. Background noise is when unwanted sounds interfere with the content (such as people talking, wind blowing, cars driving on the road, etc.). These noises may not be noticeable during the shoot, but they can affect the quality of the final output. Primarily, they can suppress the speaker’s voice.
Countdown to VB Transform 2024
Join enterprise leaders at our flagship AI event in San Francisco July 9-11. Network with your peers, explore the opportunities and challenges of generative AI, and learn how to integrate AI applications in your industry. Register now
To solve this problem, many people tend to use microphones with ambient noise cancellation, which removes background noise during the recording stage. While these microphones can certainly help, they can often be hard to come by, especially for early-stage creators with limited resources. This is where AI-based tools like ElevenLabs’ new Voice Isolator can help.
Essentially, the product works at the post-production stage, where users simply upload the content they want to enhance. Once the file is uploaded, the underlying model processes it, detects and removes unwanted noise, and extracts clear speech as the output.
ElevenLabs claims that its product can extract audio with the same quality as content recorded in a studio, and the company’s head of design, Ammaar Reshi, also gave a demo, showing how the tool filtered out the noise of a leaf blower to extract crystal clear audio of the speaker.
To test the real-world applicability of the Voice Isolator, we conducted three tests. In the first test, we spoke three separate sentences that were interrupted by various noises in the background. In the other two tests, we spoke three sentences that were mixed with various noises. The noises occurred erratically at random times.
In all cases, the tool was able to process the audio within a few seconds. Most importantly, in almost all cases it removed noises associated with door openings, banging on tables, clapping, moving household items, etc., extracting clear, distortion-free audio. The only sounds it was unable to recognize and remove were banging on walls and finger snapping.
Sam Sklar, who is in charge of growth at the company, said that while it doesn’t currently support musical vocals, users can try it out for that use case and it may be successful with some songs.
There is a possibility of improvement
Voice Isolator’s ability to remove erratic background noise makes it better than most other tools that only deal with flat noise, but there is still room for improvement, and as with all tools, we hope to see ElevenLabs further improve its performance.
It’s worth noting that the company hasn’t revealed much about the underlying model that powers the tool, or whether the recordings fed into the tool are used to train the model. Scalar couldn’t reveal details about what goes into creating the model, but the company says shape It links to a privacy policy where users can opt out of having their personal data used in the training.
Currently, the company Voice Isolator is only available through the platformThe API access will be available within the next few weeks, though the exact timeline is unclear. For users wanting to test the tool on their websites or apps, ElevenLabs is offering free access with certain usage limitations.
“The Voice Isolator model is priced at 1,000 characters per minute of audio. Our site has a free plan for 10,000 characters per month, so you get 10 minutes of audio per month for free,” Sklar explains. This means that users who want to remove background noise from larger audio files will need to switch to a paid plan, which starts at $5 per month.
