Home

Info:

This is a placeholder at the moment. However, the project is an implementation of Restricted Boltzmann Machines on the GPU, specifically in CUDA.

Acronym-tastic.

It started as a research project, to see if it was a decent idea and to explore the various variables. Turns out, yeah, it is. On an 8600GT I was getting training rates of about 1k images per second on the MNIST dataset and a network size of 784-512-512-2048 including all file reading and transfer times, which is 1 minute per epoch. Recent papers put the training time as several hours per layer for a standard implementation. As work progresses onto larger networks (for such things as fMRI, or perhaps multi-scale/attribute images) it seems pretty useful, the main limitation being the amount of ram on the card.

Current status: Hacky code there. Getting it to compile is difficult (moving directories, etc), and currently pointless without modifying the code as it’s in the state I left it after running experiments. It’s got lots of hard coded values, now unused code, explicit file paths, it’s all in one file shudder. Still, if you want to see the basic idea then it’s there. I’ll be improving it and making it a lot more flexible, it currently doesn’t allow fine grained control of parameters during training (which I now want to experiment with).

Update on the status: The code is being re-written to ease the writing of experiments. It’s on the way, you can define your own file readers and parameter modifiers now (simplest ones are implemented now, with things like changing the layer that is being trained). Hopefully soon it should be basically functional.

You will be able to:

- Train multiple layers at once. Simply set the learning rates (individual ones for each layer), the system will only process the data as far as it needs to, and will update any weights when the learning rate is not 0.

- Learn as many labels as you like. Multiple layers labeled? No problem. Some softmax, some not? Why not! Want to change from one to the other mid-learning? Go for it.

- Modify learning rates, momentums and weight decays in any way, at every iteration.

- Read and write matlab files for easy experiment running, using this implementation for the fast greedy training, then using the weights in a matlab program for whatever further work you wish.

- Have enormous files. This one is really quite simple actually, just using mmap. If there’s a better way, then just implement your own file reader, it’s really quite simple.

If you have any questions, or want to try and compile it, then feel free to drop me an email and I’ll help out best I can.