Parallel Sparse Coding

The goal of sparse coding is find out an efficient coding strategy  of information which is inspired by neural science experimental results. The original sparse coding (Olshausen & Field 1996) was achieved by optimizing the objective function which contains both residual error square  and sparsity using gradient descent. And the computation in this optimization are fully formulated in linear algebra. The previous simulations were usually done with small image patches (8*8). As the size of image patches and the data size increase, the computation can not be handled on a single computer. Now researcher gradually turned to parallel computation. And the goal here is to utilized modern parallel computation to try to do more challenging and attracting highly-overcomplete sparse coding simulation. We hope that this simulation will give people a deeper insight about the properties of the underlying information coding system of a human brain. Also, I'm personally want to make future simulation more efficient so that people can verify different theoretical model of sparse coding in a faster pace.

                  Olshausen & Field (2003)                                          Olshausen & Field (1996)                                             Hyvarinen & Hoyer (2001)