Finetuning Your Voice

The Training tab fine-tunes VibeVoice on a dataset from the Data tab. The result is a LoRA adapter (a few hundred MB) that you load on top of the base model on the Generate tab.

Do you need to fine-tune?

Modern zero-shot cloning models (VoxCPM2, MOSS-TTS, Echo-TTS, Qwen3-TTS Base) often get close enough from a 10-second clip. Fine-tune when you have lots of audio of one voice and want the most consistent likeness over long content.

Prerequisites

• A dataset: see Preparing Audio Data
• An NVIDIA GPU with 16 GB+ VRAM for VibeVoice 1.5B. 7B needs far more (48 GB+). On other hardware, training is very slow or fails, and the tab shows a warning.
• The VibeVoice model installed: training runs in the VibeVoice environment. Install it on the Models tab first.

Training

1. Open the Training tab.
2. Dataset: pick your dataset. Click Refresh datasets if it isn't listed.
3. Base model: vibevoice/VibeVoice-1.5B (recommended), vibevoice/VibeVoice-7B, or a local path.
4. Training parameters (optional, see below).
5. Click Start training.

Training runs as a background process, so you can close the browser while it runs. Click Stop to end it early.

Parameters

Parameter	Default	Notes
Epochs	3	Passes over the dataset. Start with 3-5; too many can sound robotic (overfitting).
Batch size	8	The official recommendation. Lower it if you run out of memory.
Gradient accumulation steps	1	Raise it to keep the effective batch size when you lower the batch size.
Learning rate	2.5e-5	The default works for most datasets.
LoRA rank	8	Higher (16-32) can capture more detail but uses more memory. Alpha is set to 4× the rank.
Voice prompt drop rate	0.2	0.2 keeps voice cloning working with the adapter. 1.0 trains a single fixed voice (more natural, no cloning).
Gradient checkpointing	off	Saves memory at the cost of speed.

Monitoring

• Show TensorBoard: loss curves and learning rate. Loss should go down over time. TensorBoard appears once the first metrics are logged.
• Training log: the trainer's raw output. Click Refresh log to update it.
• Runs: every run, with its status, dataset and LoRA path. Click Refresh runs to update it.

TensorBoard only works on the machine running OpenVoiceLab, not through --share or from another device.

Using your adapter

Runs are saved in training_runs/run_<timestamp>/, and the adapter is in checkpoints/lora/.

1. On the Generate tab, set Model family to VibeVoice and choose the same base checkpoint you trained on.
2. Pick your run under LoRA adapter, or paste the path to its lora/ folder.
3. Click Load model and generate as usual.

With a voice prompt drop rate below 1.0, you can still combine the adapter with a reference clip. Try generating both with and without one.

From the command line:

openvoicelab tts vibevoice "Speaker 1: Hello from my fine-tuned voice." \
    --lora training_runs/run_20261019_143022/checkpoints/lora -o lora.wav

Troubleshooting

• Out of memory: lower the batch size and raise the gradient accumulation steps, turn on gradient checkpointing, or use a lower LoRA rank.
• Loss isn't going down: check the dataset (listen to clips and check the transcripts in metadata.csv), then train for more epochs.
• The result doesn't sound like the voice: train for more epochs, raise the LoRA rank, use more or cleaner data, or try a voice prompt drop rate of 1.0.
• The result sounds robotic: it's overfitting. Use fewer epochs or more varied data.
• Training crashes: open Training log (or training_runs/run_<timestamp>/train.log) for the error.

Sharing your model

To share a fine-tune with the community, join the vibevoice-community org on Hugging Face. Create a new model repo and upload the contents of your run's checkpoints/lora/ folder. Only share voices you have permission to share.