I write these papers and Matlab code for presentations, teaching and (or) for my own personal interest. I try to keep these informal for the purpose of instruction, not reference. Of course, if you find any errors or have any suggestions, please let me know.
Tutorials
- Tutorial on Principal Component Analysis, Dec 2005 [ pdf ]
A full introduction, description, derivation, and discussion of principal component analysis. Concrete examples for intuition building, the mathematical relation to SVD, and new extensions of this algorithm. Version 2.0 - A Light Discussion and Derivation of Entropy, Aug 2007 [ pdf ]
A light discussion of the underlying assumptions behind entropy followed by a rigorous but simple derivation of the formula for entropy. - Notes on Kullback-Leibler Divergence and Likelihood, Aug 2007 [ pdf ]
An intuitive discussion about where Kullback-Leibler divergence arises and its relationship to likelihood theory. - Notes on Generalized Linear Models of Neurons, May 2008 [ pdf ]
An introduction to the application of GLMs to model neurons and networks of neurons. Brief discussion and derivation of primary equations pertaining to maximum likelihood estimation.
Sample Code
- Bayesian Entropy and Information Rate Estimation, 1 July 2005 [ matlab/fortran ]
Estimation of entropy rates and information rates with confidence intervals using Bayesian techniques mixed with ideas from compression technology (i.e. context tree weighting) as discussed in recent paper. - Independent Component Analysis Demo, 13 Feb 2003 [ matlab ]
A step-by-step tutorial and sofware demo that examines what ICA does, the advantage of ICA over PCA and the limits of ICA. - Cluster Analysis Demo, 13 Feb 2003 [ matlab ]
A simple but colorful Matlab demo that demonstrates what K-means cluster analysis is and how it works. Fun to play with. - Principal Component Analysis, 26 Aug 2002 [ matlab ]
Version 1 performs PCA by diagonlizing the covariance (a little easier to understand). Version 2 uses the "SVD trick." Both versions produce equivalent results - Independent Component Analysis, 26 Aug 2002 [ matlab ]
This code follows the infomax method developed by Bell and Sejnowski and subequently improved by Amari et al
