Dynamically localizing multiple speakers based on the time-frequency domain

In this study, we present a deep neural network-based online multi-speaker localization algorithm based on a multi-microphone array. Following the W-disjoint orthogonality principle in the spectral domain, time-frequency (TF) bin is dominated by a single speaker and hence by a single direction of arrival (DOA). A fully convolutional network is trained with instantaneous spatial features to estimate the DOA for each TF bin. The high-resolution classification enables the network to accurately and simultaneously localize and track multiple speakers, both static and dynamic. Elaborated experimental study using simulated and real-life recordings in static and dynamic scenarios demonstrates that the proposed algorithm significantly outperforms both classic and recent deep-learning-based algorithms. Finally, as a byproduct, we further show that the proposed method is also capable of separating moving speakers by the application of the obtained TF masks.

DIRECTIONAL MAXIMAL OPERATORS AND RADIAL WEIGHTS ON THE PLANE

Let $\Omega $ be the set of unit vectors and $w$ be a radial weight on the plane. We consider the weighted directional maximal operator defined by $$\begin{eqnarray*}{M}_{\Omega , w} f(x): = \sup _{x\in R\in \mathcal{B} _{\Omega }}\frac{1}{w(R)} \int \nolimits \nolimits_{R} \vert f(y)\vert w(y)\hspace{0.167em} dy,\end{eqnarray*}$$ where ${ \mathcal{B} }_{\Omega } $ denotes the set of all rectangles on the plane whose longest side is parallel to some unit vector in $\Omega $ and $w(R)$ denotes $\int \nolimits \nolimits_{R} w$. In this paper we prove an almost-orthogonality principle for this maximal operator under certain conditions on the weight. The condition allows us to get the weighted norm inequality $$\begin{eqnarray*}\Vert {M}_{\Omega , w} f\mathop{\Vert }\nolimits_{{L}^{2} (w)} \leq C\log N\Vert f\mathop{\Vert }\nolimits_{{L}^{2} (w)} ,\end{eqnarray*}$$ when $w(x)= \vert x\hspace{-1.2pt}\mathop{\vert }\nolimits ^{a} $, $a\gt 0$, and when $\Omega $ is the set of unit vectors on the plane with cardinality $N$ sufficiently large.