Monthly Archives: May 2015

I’ve seen a bunch of tutorials on Neural Networks in Python, but I’ve never found any of them to be particularly good. They’re either riddled with errors or use simple, single-example training with basic arrays. I’ve always wanted something a little more robust than the simple C-ish implementation and something less mathematically terse than the average neural network paper. I made this implementation in the hopes that it will explain how neural networks work and how you’d use a matrix library to train multiple examples at the same time. It’s not optimized (since you can save your activation values and re-use them), but it should be easy to tune.

Here’s the source:

And here are some visualized examples as the different activation functions try to learn f(x) = sin(x).

A five layer neural network with sigmoid activation for the inner three layers.

The same network structure as above, but with softmax activation for the hidden layers.

The same network with Tanh activation.