PhonePaint: Zero-Shot Frame-Synchronous Phoneme Editing

Examples of PhonePaint, Seed-VC*, a zero-shot voice conversion (VC) model, and VoiceCraft**, a zero-shot word-level editor.
No direct comparison with zero-shot open-source phone-based editing tools is possible as none exist***.
Edited phones are shown in this color.
Cosine similarity between speaker embeddings of original and edited files obtained using ECAPA_TDNN.
Project brief (PDF)

Audio clips were collected from YouTube. If you deem any of the clips to be your intellectual property and would like to have it removed, contact physicalchemistrytom{@g}mail.com (remove {}).

Unbatched inference × real-time with (warm startup) and without (cold startup) preloaded models on GTX Ti1080, averaged over demo files

* Largest Seed-VC v1 checkpoint (seed-uvit-whisper-base, 200M SVC); default settings with --f0-condition True and source = target
** VoiceCraft giga330M speech-editing model; default settings with contiguous word-span infilling over the edited text
*** As of Aug 2026, no zero-shot open-source English phone editor is available; the closest phoneme-level work is PPG2Speech (Li et al., SSW 2025: Finnish, vowel-focused single-phoneme edits; model and code)
**** Acoustic encoding & dynamic programming (MAPS)/HMM-GMM (MFA) run on CPU; MAPS scoring done on GPU

Audio failed to load. If you received this link privately, contact the sender.