Site de Vincent Gripon

Blog sur mes recherches et mon enseignement

A GPU-based Associative Memory using Sparse Neural Networks

Z. Yao, V. Gripon et M. Rabbat, "A GPU-based Associative Memory using Sparse Neural Networks," dans Proceedings of the PCNN-14 conference, pp. 688--692, juillet 2014.

Associative memories, serving as building blocks for a variety of algorithms, store content in such a way that it can be later retrieved by probing the memory with a small portion of it, rather than with an address as in more traditional memories. Recently, Gripon and Berrou have introduced a novel construction which builds on ideas from the theory of error correcting codes, greatly outperforming the celebrated Hopfield Neural Networks in terms of the number of stored messages per neuron and the number of stored bits per synapse. The work of Gripon and Berrou proposes two retrieval rules, SUM-OF-SUM and SUM-OF-MAX. In this paper, we implement both rules on a general purpose graphical processing unit (GPU). SUM-OF-SUMuses only matrix-vector multiplication and is easily implemented on the GPU, whereas SUM-OF-MAX, which involves non-linear operations, is much less straightforward to fulfill. However, SUM-OF-MAX gives significantly better retrieval error rates. We propose a hybrid scheme tailored for implementation on a GPU which achieves a 880-fold speedup without sacrificing any accuracy.

Télécharger le manuscrit.

  author = {Zhe Yao and Vincent Gripon and Michael
  title = {A GPU-based Associative Memory using Sparse
Neural Networks},
  booktitle = {Proceedings of the PCNN-14 conference},
  year = {2014},
  pages = {688--692},
  month = {July},

Vous êtes le 2021624ème visiteur

Site de Vincent Gripon