What SoundSpotter actually does: matching, not synthesis

People often file SoundSpotter under "granular synthesis", and then get confused when it behaves nothing like a granular synth. The confusion is understandable, but the design goal is different. SoundSpotter does not make sound from scratch. It matches.

Target in, grains out

You give SoundSpotter two things: a corpus of source audio, and a live or recorded target. Frame by frame, it extracts a compact timbral feature from the target, searches the corpus for the closest-sounding frames, and plays those back. The output is always made of real sound that already exists in your corpus, re-sequenced to follow the shape of the target.

That is why we call it a real-time music mashup. Hum into it with a corpus of orchestral recordings and it will chase your melody with whatever orchestral fragments sound most like each moment of your voice.

Why matching, not synthesis

Matching keeps the character of the source corpus intact. A concatenative synth stitches grains too, but SoundSpotter's emphasis is on the similarity search: the quality of the match, the features used, and the speed of retrieval. Those are information-retrieval problems, and they are the same problems that later grew into audioDB.

If you are new to the object, start with the download page and the live-patching notes.