Abstract:

We introduce a framework for audio source separation using em- beddings on a hyperbolic manifold that compactly represent the hi- erarchical relationship between sound sources and time-frequency features. Inspired by recent successes modeling hierarchical rela- tionships in text and images with hyperbolic embeddings, our algo- rithm obtains a hyperbolic embedding for each time-frequency bin of a mixture signal and estimates masks using hyperbolic softmax layers. On a synthetic dataset containing mixtures of multiple peo- ple talking and musical instruments playing, our hyperbolic model performed comparably to a Euclidean baseline in terms of source to distortion ratio, with stronger performance at low embedding dimen- sions. Furthermore, we find that time-frequency regions containing multiple overlapping sources are embedded towards the center (i.e., the most uncertain region) of the hyperbolic space, and we can use this certainty estimate to efficiently trade-off between artifact intro- duction and interference reduction when isolating individual sounds.