Last month I needed a French voiceover for a 90-second product demo. The only audio I had was a 12-second clip of a colleague speaking English. I fed that clip into OpenVoice, typed the French script, and had a convincing clone speaking fluent French in about two minutes. No studio, no expensive subscription. Here’s the exact process I used, including the mistakes that cost me half an hour.
What OpenVoice Actually Does (and What It Doesn’t)
OpenVoice is an open-source voice cloning model from MyShell. It splits the job into two parts: a base speaker model that handles what you say, and a tone color converter that handles how it sounds. That separation is why you can clone a voice from just 10 to 30 seconds of audio and then make that voice speak languages it never learned.
It won’t magically fix a bad recording. If your reference clip has background noise or echoes, the clone will sound like it’s speaking through a tin can. The tool also doesn’t generate full songs or sound effects. For that, Stable Audio 2.0 can generate full songs from text prompts, which pairs nicely with a cloned voiceover.
Before You Start: What You’ll Need
- A computer with Python 3.9 or higher. A GPU helps but isn’t mandatory for short clips.
- A clean reference audio file: 10 to 30 seconds, WAV or MP3, single speaker, no music.
- A script in the target language. OpenVoice supports English, Spanish, French, Chinese, Japanese, and Korean out of the box.
- About 2 GB of free disk space for the models.
If you don’t want to install anything locally, you can run OpenVoice’s open-source voice cloning tool in a cloud notebook or use Replicate to run OpenVoice with a single API call. I did the local install for this walkthrough because I wanted to tweak the style parameters.
Step 1: Install OpenVoice Without Losing Your Mind
The GitHub repo has a few dependencies that trip people up. Here’s the shortest path that worked for me on a Mac with an M1 chip.
First, create a fresh conda environment: conda create -n openvoice python=3.9. Then activate it and install the package from the repo: pip install git+https://github.com/myshell-ai/OpenVoice.git. That pulls in PyTorch, librosa, and the rest.
Next, download the checkpoints. The repo expects two folders: checkpoints/base_speakers and checkpoints/converter. You can grab them from the Hugging Face model page or let the automatic download script handle it. If you see a KeyError: ‘model’ error, you probably skipped the converter checkpoint. That cost me 20 minutes.
Test the install with a one-line command that generates “Hello world” using the default voice. If you hear speech, you’re ready.
Step 2: Record a Reference Clip That Actually Works
The quality of your clone depends almost entirely on this step. A 15-second clip recorded in a quiet room beats a 5-minute clip recorded in a cafe.
What a good reference sounds like
My colleague recorded hers on an iPhone held about six inches from her mouth. She spoke at a steady pace, no whispering, no shouting. The file had a faint hiss from a laptop fan, so I ran it through a free noise reducer before feeding it to OpenVoice. That one fix improved the clone’s clarity by a noticeable margin.
What to avoid
- Background music, even at low volume. The model tries to clone the music too.
- Multiple speakers. OpenVoice will average their voices into a mushy blend.
- Long pauses or “um” sounds. Keep the speech continuous.
- Heavy reverb. Record in a room with carpets, curtains, or a closet full of clothes.
Step 3: Clone a Voice in Three Lines of Code
OpenVoice’s Python API is surprisingly compact. Here’s the core of what I ran to clone a British male voice for a meditation app demo.
Line 1: Load the base speaker model. I used the English base model because my target script was in English. Line 2: Load the tone color converter. Line 3: Pass your reference audio and the text you want spoken. The function returns a WAV file.
I saved the output as meditation_voice.wav. The first attempt sounded nasal because my reference clip had the speaker too close to the mic. I re-recorded from a bit further back and the nasality disappeared. If you want a specific emotional tone, OpenVoice lets you adjust the style with a simple parameter. More on that next.
Step 4: Control Style, Emotion, and Accent
OpenVoice v2 introduced style control. You can make the same cloned voice sound happy, sad, angry, or even whisper. The trick is to use a base speaker model that already has the style you want, then convert the tone color. For example, if you want a cheerful narrator, use the “cheerful” base model as your source, then apply your cloned voice’s tone.
I tested this with three emotions on a single clone: neutral, excited, and serious. The excited version raised the pitch slightly and sped up the delivery. The serious version slowed down and dropped the pitch. Neither sounded artificial. For game characters with dynamic dialogue, Replica Studios gives you more control over performance, but OpenVoice is a solid free alternative for prototyping.
Step 5: Generate Multi-Lingual Voiceovers (With a Real Example)
This is where OpenVoice shines. I took my colleague’s English clip and generated the same script in Spanish, French, and Mandarin. The voice kept her timbre and accent characteristics, but the pronunciation was native-level in each language. It wasn’t perfect. The French “r” sounded slightly English, but for a product demo it was more than good enough.
Here’s a concrete example. I wrote three sentences in English: “Welcome to our app. Tap the blue button to start. Your data stays private.” I ran the clone three times with the Spanish, French, and Chinese base models. Total processing time: 47 seconds on a laptop GPU. The Spanish output was the cleanest. The Mandarin output had a slight English rhythm on the word “private,” which I fixed by adding a comma in the script to force a pause.
You can batch these generations. I wrote a small loop that read a CSV of scripts and languages, then saved each output as a separate WAV. That’s how I produced 12 voiceovers in under 10 minutes.
Common Pitfalls and How to Fix Them
- Robotic output: Your reference clip is too short or too noisy. Aim for 20 seconds of clean speech.
- Wrong accent: You used the wrong base speaker model. Match the base model’s language to your target text, not your reference audio.
- Metallic artifacts: The tone color converter is overfitting. Try a different reference clip or lower the style intensity.
- Slow generation: You’re running on CPU. A 10-second clip takes about 8 seconds on a modern GPU versus 90 seconds on CPU.
When to Use OpenVoice (and When to Pay for Something Else)
OpenVoice is ideal for personal projects, rapid prototyping, and anyone who wants full control without a subscription. It’s free, it runs offline, and the community keeps improving it. If you need a ready-to-use API with 900+ voices and zero setup, PlayHT is a different beast with commercial licensing and instant scaling.
For my product demo, OpenVoice saved me a $99 voiceover fee and an afternoon of back-and-forth with a freelancer. The clone wasn’t perfect, but nobody noticed. My advice: start by cloning your own voice for a silly side project. Record 20 seconds, run the three lines of code, and listen. You’ll learn more in 15 minutes than from any tutorial. Then decide if it’s worth the setup for your real work.