Three.js Voice Visualizer (Formants, Pitch & Speech Recognition)
Speech-to-visuals in Three.js, tracking pitch by autocorrelation and the first two formants from the spectral envelope to place the voice on a 3D vowel chart, with optional microphone input and Web Speech API words floating up as text.
Key topics: voice visualization, formant tracking, speech recognition api, pitch detection, vowel chart, web audio microphone
Game Over
What this code does
- Source–Filter Model: vocal folds make a harmonic series at f0, and the vocal tract's resonances (formants F1, F2, F3) shape it.
- Synthetic Talker: 28 additive harmonics with gains from three resonance curves glide through vowel targets, so no microphone is needed.
- Formants: the dB spectrum is smoothed over ~200 Hz into an envelope (yellow) whose peaks in 220–900 Hz and 750–2600 Hz are F1 and F2.
- Vowel Chart: F2 runs front→back and F1 close→open, matching the IPA layout, so /i/ sits top-left and /a/ at the bottom.
- Pitch: autocorrelation finds the lag of strongest self-similarity, one glottal period, giving f0 = sampleRate / lag.
- Microphone: when enabled, analysis switches to your voice, and SpeechRecognition (where supported) floats recognised words up as sprites.
Related Demos
Continue Learning
JavaScript Implementation
// Formant tracking: smooth the spectrum into an envelope, take its peaks as F1 and F2
analyser.getFloatFrequencyData(db)
const env = movingAverage(db.map(v => 10 ** (v / 20)), 200 / binHz).map(a => 20 * Math.log10(a))
const F1 = peakHz(env, 220, 900), F2 = peakHz(env, F1 + 250, 2600)
marker.position.set(chartX(F2), chartY(F1), 0) // vowel chart: /i/ top-left, /a/ bottom
// Pitch by autocorrelation: the lag with the strongest self-similarity is one glottal period
let best = 0, lag0 = 0
for (let lag = sr / 400 | 0; lag < sr / 70; lag++) { const c = dot(buf, shift(buf, lag)); if (c > best) { best = c; lag0 = lag } }
const f0 = sr / lag0
// Words: new (webkit)SpeechRecognition() → onresult → spawn text sprites