A Null-space Algorithm for Overcomplete Independent Component Analysis

Abstract

Here we propose a novel null-space algorithm for solving the overcomplete independent component analysis problem. If we have multivariate data, the idea of independent component analysis is to find a suitable representation of multivariate data such that the components of the representation are as independent as possible. Here the representation is considered as a linear transformation of the original multivariate data which is performed by multiplying the multivariate data by a matrix. The overcomplete situation is that the dimension of the representation is more than the dimension of multivariate data. The key point of this work is to identify the representation in terms of the observed multivariate data via the null space of the matrix of this linear transformation. Under this representation, the problem can be posed as a latent variable or missing data problem. Then we can develop a new MCMC type algorithm, the null-space algorithm, to perform the overcomplete independent component analysis. There are two components in our algorithm. One is the inhibition algorithm for recovering the sources given the mixing matrix. Another is the Givens sampler for estimating the mixing matrix using Givens rotations. Finally we applied the null-space algorithm to separate the different sound signals from their mixtures, and demonstrate several examples to show the performance of our method.