WarpSonic

Getting started

WarpSonic is for describing what's audible in a video: you mark each sound on the timeline, describe it, then export the result. Three terms to know first:

• A range is a labeled stretch of time on a track marking that one kind of sound occurs — from a start point to an end. Ranges are what you create and describe; they can overlap (speech over music is two ranges at the same time, not one).
• A subject is a voice or performer — a speaker on speech ranges, a performer on music ranges. You define it once (a name plus stable traits like gender, accent, age) and assign it to that person's ranges, so those traits fill in automatically instead of being re-entered on every range.
• The five types — Speech, Music, Sound Effects, Ambience, Artifacts — are parallel layers; you choose which kind of sound you're marking.

Tips

• Loop a region by selecting a range and pressing ⇧L, or marking I/O points first.
• Undo works across all edits: ⌘Z / Ctrl Z.
• Field values set on a Subject become defaults for their ranges — override per-range only when the range differs from the subject.

The five range types

Type	Use for	Key fields
Speech 1	Any spoken dialogue, narration, voice-over, commentary, or announcements. Human voice as language.	Transcription, Subject, Emotion, Speech Dynamics, Intent, Timbre, Volume, Speaking rate
Music 2	Musical content — background score, featured performance, diegetic music (music that exists in the scene's world), jingles, stings.	Genre, Mood, Energy, Tempo, Instrumentation, Vocals, Lyrics
Sound Effects (SFX) 3	Discrete sound effects with a clear onset — things that happen at a moment in time.	Description (e.g. "glass shatter", "door slam", "gunshot")
Ambience 4	Continuous background texture without a discrete onset — the sonic environment of a scene.	Description (e.g. "city traffic", "forest birds", "office hum")
Artifacts 5	Audio problems and recording issues — clipping, distortion, dropouts, digital glitches, or excessive noise. Flags a quality concern rather than content.	Description (e.g. "clipping", "dropout", "background hiss", "codec artifacts")

Edge cases

• Singing: Use Music, not Speech — the voice is part of the music track. If the singer also speaks between sections, create separate Speech ranges for those moments.
• Voice-over / narration: Always Speech, even with no on-screen speaker.
• Crowd noise: Ambience (continuous texture). A single crowd cheer or reaction is a sound effect (SFX).
• Music with speech over it: Create two overlapping ranges — Music for the bed, Speech for the foreground speaker. Overlapping ranges are fully supported.
• Sustained sound effects: If a sound effect becomes continuous (e.g. rain that continues throughout a scene), use Ambience. If it has a discrete start, a sound effect (SFX) is more appropriate — use your judgement based on perceptual character.
• Non-verbal vocalisations (laughs, coughs, sighs): Use Speech with a note in the Vocal Events field, not a separate Events range — they are vocal in origin.

Timbre — CAPE-V & Laver (1980)

Axes derived from the Consensus Auditory-Perceptual Evaluation of Voice (ASHA, 2004) and Laver, J., The Phonetic Description of Voice Quality (Cambridge, 1980).

The timbre widget maps perceptual voice quality onto five axes. Brightness correlates with spectral centroid — the centre of gravity of the frequency spectrum. A bright voice carries more high-frequency energy; a dark voice more low-frequency energy. Breathiness reflects the ratio of aspiration noise to harmonic energy (the harmonics-to-noise ratio, HNR): incomplete vocal fold closure allows air to escape, raising the noise floor. Roughness corresponds to aperiodicity and irregular pitch perturbation — jitter and shimmer in acoustic measurement, the R dimension in the clinical GRBAS scale.

Nasality reflects the degree of velopharyngeal opening: when the soft palate lowers, the nasal cavity couples into the vocal tract, introducing anti-formants visible in the spectrum. Fullness describes the perceived body, warmth, and resonance of the voice — associated with lower formant frequencies and efficient cavity coupling; Laver links a fuller sound to a lowered larynx setting that lengthens the vocal tract. (Warmth and resonance are combined into this single Fullness axis.)

The descriptor chips (warm, breathy, raspy, velvety, etc.) are generated by combining axis positions: "velvety" requires fullness above mid, roughness below mid, and breathiness in a moderate band. Chips that match the current axis configuration are highlighted as suggestions; clicking one adds it to the stored text value.

Emotion — the Geneva Emotion Wheel v3.0

Scherer, K.R., Shuman, V., Fontaine, J.R.J. & Soriano, C., "The GRID meets the Wheel," in Components of Emotional Meaning (Oxford, 2013). Lineage: Russell, J.A., "A circumplex model of affect," J. Personality and Social Psychology 39(6), 1980.

The emotion vocabulary is the Geneva Emotion Wheel (GEW), a validated circumplex instrument. Like Russell's model it locates every emotion in a two-dimensional space, but its axes are valence (unpleasant → pleasant) and control/power (feeling overwhelmed → feeling in control), not arousal. The GEW uses control because arousal differentiates intensity within a family (irritation → anger → rage), whereas control separates the families themselves: high-control anger sits opposite low-control fear — states that valence and arousal alone would leave piled together. Control/power is what the literature also calls dominance.

The 20 emotion families were positioned empirically by the cross-cultural GRID study (10 languages, 10 countries); WarpSonic adds neutral (the wheel's "no-emotion" hub) plus surprise and boredom — which the standard GEW omits but which are common and audible in speech. An optional intensity modifier (mild / strong) captures how strongly the emotion is expressed: mild fear (unease) vs. strong fear (alarm).

The wheel beside the chips is a read-only orientation map: it shows where your pick sits relative to the other emotions and to its opposite, so you build a mental model of the space. Clicking a chip stores the value; the map never changes the annotation.

Speech Dynamics — speech prosody research

Cruttenden, A., Intonation (2nd ed., Cambridge, 1997). Bolinger, D., Intonation and Its Parts (Stanford, 1986). Crystal, D., The English Tone of Voice (Arnold, 1975).

Prosody encompasses the suprasegmental features of speech — the way something is said rather than what is said. The four axes capture the dimensions most consistently identified in speech research as perceptually salient and acoustically measurable.

Tempo (speech rate) is measured in syllables per second or words per minute. Slow delivery (under ~120 wpm) is associated with emphasis, careful explanation, or cognitive load; fast delivery (over ~180 wpm) with fluency, excitement, or urgency. Highly variable rate within an utterance is itself informative and may generate multiple tags.

Pitch range reflects the span of fundamental frequency (F0) variation across the utterance — the intonation span. A narrow range produces monotone, flat delivery; a wide range produces expressive, animated delivery. Bolinger's work established that English uses pitch range to signal involvement, emphasis, and contrast.

Stress reflects how forcefully syllables or phrases are emphasised. In English, nuclear stress placement is the primary carrier of information structure (given vs. new information, contrast, correction). Heavy stress produces emphatic or commanding delivery; light stress produces soft, even, or whispered speech.

Rhythm reflects fluency and the temporal regularity of delivery. Halting rhythm is characterised by frequent pauses, hesitations, and disfluencies; fluid rhythm by smooth, continuous production. This is distinct from tempo: speech can be slow and fluid (deliberate) or fast and halting (anxious or clipped).

Intent — Conversational purpose (grounded in Searle's Speech Act Theory, 1969)

Searle, J.R., Speech Acts: An Essay in the Philosophy of Language (Cambridge, 1969). Austin, J.L., How to Do Things with Words (Oxford, 1962). SWBD-DAMSL scheme: Jurafsky et al., ICSI Technical Report TR-97-02, 1997.

The core insight of speech act theory — first developed by Austin and systematised by Searle — is that utterances do not merely convey propositional content but perform actions: each carries an illocutionary force (the action being performed) on top of its literal meaning. The widget sorts that force into five conversational-purpose categories — plainer labels for annotators than Searle's original type names, which they map onto directly:

Informing — telling the listener something: state, explain, report, clarify, correct, warn, admit. (Searle's Assertive / Representative — the speaker commits to the truth of a proposition; word-to-world fit.)

Questioning — asking for information or confirmation: ask, inquire, challenge, probe, wonder. (The information-seeking half of Searle's Directive.)

Requesting — asking the listener to do something: request, demand, command, invite, suggest, persuade, advise. (The action-seeking half of Searle's Directive; world-to-word fit.)

Expressing — sharing a psychological or social reaction: thank, apologize, congratulate, complain, praise, joke, exclaim. (Searle's Expressive — no direction of fit.)

Committing — promising or threatening a future action: promise, offer, pledge, threaten, agree, refuse. (Searle's Commissive — the obligation falls on the speaker.)

Searle's fifth type, Declaration (the utterance itself changes reality — verdicts, appointments, dismissals), is omitted: it's vanishingly rare in natural recorded speech. The widget supports multi-select because utterances frequently carry mixed force — "You might want to close the window" is at once Informing and Requesting. Grounding the labels in Searle's taxonomy keeps WarpSonic annotations compatible with the SWBD-DAMSL dialogue-act tradition used in the Switchboard corpus.

Turn the automatic, on-load analyses on or off. These are workflow preferences (saved on this device), not per-video — disable what you don't need to skip the model downloads and computation.