Statistics How To

Independent Component Analysis: Definition

Multivariate Analysis >

Independent component analysis is used in statistics and signal processing to express a multivariate function by its hidden factors or subcomponents. These component signals are independent non-Gaussian signals, and the intention is that these independent subcomponents accurately represent the composite signal.

The ‘cocktail party problem‘ is an often cited example of independent component analysis at work. Suppose you have a loud cocktail party, with many conversations going on in the same room. Now suppose you have a number of microphones located at various locations throughout the room, picking up the sound data. With no prior knowledge of the speakers, can you separate the signal from each microphone (in each case, a composite of all the noise in the room) into its component parts, i.e., each speakers voice?

You can do a surprisingly good job, assuming you have enough observation points. If there are N sources of noise (N guests, for instance) you will need N microphones in order to fully determine the original signals.

Assumptions in Independent Component Analysis.

Independent component analysis only works if the sources are non-Gaussian (i.e. they have non-normal distributions), and so that is one of the first assumptions you make if you use this analysis on a multivariate function. Since you are attempting to break the function down into independent components, you’re also making the assumption that the original sources were in fact independent.

Mixing Effects in Independent Component Analysis

There are three principles of mixing signals which make up the foundation for independent component analysis.

  1. Mixing signals involves going from a series of independent signals to a series of signals that is dependent. Although your source signals are all independently generated, the composite signals are all made from the same source signals, and so cannot be independent of each other.
  2. Although our source signals are non-Gaussian (by the assumption we made to begin with), the composite signals are in fact Gaussian (normally distributed). This is by the Central Limit Theorem, which tells us the the probability distribution function of the sum of independent variables with finite variance (like our signals) will tend toward the Gaussian distribution.
  3. The complexity of a signal mixture must always be greater than, or equal to, the complexity of its simplest source signal.

These three principles form the foundation for Independent Component Analysis.

References

  1. Hyvärinen, Aapo. What is Independent Component Analysis?
    Retrieved from https://www.cs.helsinki.fi/u/ahyvarin/whatisica.shtml on April 10, 2018
  2. Hyvärinen, Karhunen, & Oja. Independent Component Analysis. Chapter 1.
    Retrieved from http://research.ics.aalto.fi/ica/book/intro.pdf on April 10, 2018
  3. Ng, Andrew. CS 229 Lecture Notes: Independent Component Analysis.
    Retrieved from http://cs229.stanford.edu/notes/cs229-notes11.pdf on April 10, 2018
  4. Stone, James. Independent Component Analysis. Encyclopedia of Statistics in Behavioral Science, Volume 2, pp. 907–912
    Retrieved from https://pdfs.semanticscholar.org/6cdc/d22d69479c6c19f1583a281a95bc4029631e.pdf on April 10, 2018
------------------------------------------------------------------------------

Need help with a homework or test question? With Chegg Study, you can get step-by-step solutions to your questions from an expert in the field. If you'd rather get 1:1 study help, Chegg Tutors offers 30 minutes of free tutoring to new users, so you can try them out before committing to a subscription.

If you prefer an online interactive environment to learn R and statistics, this free R Tutorial by Datacamp is a great way to get started. If you're are somewhat comfortable with R and are interested in going deeper into Statistics, try this Statistics with R track.

Comments? Need to post a correction? Please post a comment on our Facebook page.

Check out our updated Privacy policy and Cookie Policy

Independent Component Analysis: Definition was last modified: July 22nd, 2018 by Stephanie