Abstract:

This paper proposes reverberation as supervision (RAS), a novel un- supervised loss function for single-channel reverberant speech sepa- ration. Prior methods for unsupervised separation required the syn- thesis of mixtures of mixtures or assumed the existence of a teacher model, making them difficult to consider as potential methods ex- plaining the emergence of separation abilities in an animal’s audi- tory system. We assume the availability of two-channel mixtures at training time, and train a neural network to separate the sources given one of the channels as input such that the other channel may be predicted from the separated sources. As the relationship be- tween the room impulse responses (RIRs) of each channel depends on the locations of the sources, which are unknown to the network, the network cannot rely on learning that relationship. Instead, our proposed loss function fits each of the separated sources to the mix- ture in the target channel via Wiener filtering, and compares the resulting mixture to the ground-truth one. We show that minimiz- ing the scale-invariant signal-to-distortion ratio (SI-SDR) of the pre- dicted right-channel mixture with respect to the ground truth implic- itly guides the network towards separating the left-channel sources. On a semi-supervised reverberant speech separation task based on the WHAMR! dataset, using training data where just 5% (resp., 10%) of the mixtures are labeled with associated isolated sources, we achieve 70% (resp., 78%) of the SI-SDR improvement obtained when training with supervision on the full training set, while a model trained only on the labeled data obtains 43% (resp., 45%).