How it works
Two models, one computer
Here is what happens between dropping a folder on ToneTrace and seeing a tagged library. In the current build every step runs on your own machine.
Decode
FFmpeg, bundled with the app, decodes the file to raw audio on your machine. Silence is set aside, so a two-second kick and a four-minute stem are both judged on the parts that actually contain sound.
Listen for instruments
YAMNet, an open model from Google trained on the AudioSet collection of everyday and musical sounds, listens across the whole active file rather than taking a single snapshot. Its verdict reflects what is present for most of the file, so a brief cymbal hit does not outrank the guitar underneath it.
Listen for character
CLAP, an open language-audio model from LAION, hears audio and reads text in the same space. ToneTrace uses it to turn the file into an audio fingerprint, then compares that fingerprint with a curated vocabulary of descriptions covering genre, mood, production technique and sonic quality. The closest matches become tags.
Settle close calls
When YAMNet is torn between close relatives, a guitar and a ukulele, say, CLAP's hearing is weighed in before the tag is chosen. Neither model can overrule the other on its own. A clear ukulele stays a ukulele; an ambiguous strum gets the guitar that CLAP hears in it.
Remember
Before any of this, the file's bytes are hashed. Tags, waveform and fingerprint are stored against that hash in a local database, so the same audio is never analysed twice, even after it has been moved or renamed.
Show your work
Each instrument tag shows the model's own probability for that sound. Each character tag shows how closely the audio fingerprint matched its description. The numbers are the models' honest opinions, not a score tuned to look confident.
Why tags carry a confidence
A tag without a number invites you to trust it blindly. A tag with a number tells you when to double-check. A kick at 90 % and a kick at 35 % are both worth showing, but you will treat them differently, and so will the export: by default only tags at or above 30 % are written into files.
The tag vocabulary for mood and genre is still being refined during early access. Some descriptions are more useful than others, and testers' feedback on which tags are wrong is exactly what we are asking for.
Illustrative values from a strummed acoustic chord in the test set. Instrument confidence is a probability; genre and mood confidence is a similarity.
Built on open components
YAMNet and CLAP are published open models. They run inside the app through ONNX Runtime, FFmpeg handles decoding, and the library lives in SQLite. Each is credited with its license on the credits page, and the full license texts ship inside the app.
Hear it on your own samples
Early access is free. Pricing will be announced at launch.