Archive

Monthly Archives: October 2013

Restricted Boltzmann Machine

2013-10-25
Rant

As a largely unstructured academic exercise, I spent the last few weeks implementing a Restricted Boltzmann Machine trainer. I’ve tried to do so before with limited success. Most of the hang-ups happen in the form of nonsensical output or impossible to tweak hyperparameters. The latest iteration is slightly farther along than I usually get, and I’d ascribe this success to a few factors: first, amphetamines; second, visualization tools.

The visualization path started with a simple function to dump the weights out to a PNG file. It was far from gorgeous, but it worked. Mostly. Almost. Not really.

While it was great to see the features the network could grow at the end, it wasn’t enough. I needed information about the genesis of the connections — something that would at least give _some_ information with which to reason, just the tiniest hint about the workings of the black box. I turned to swing and JPanel, overriding the paintComponent function for a special visualizer class and having it randomly select an element and run one cycle of the visual -> hidden -> visual -> hidden dance. This has proven to be the ‘printf’ statement of debugging tools. It allows me to see, as the network learns, the behavior of the RBM. Still, despite this booming success I had issues with weights converging to the same value, nonsensical reconstruction, and general issues with weird-looking data. I added more pieces. I wrote a realtime weight visualizer, a delta weight visualizer, a hidden and a visible bias visualizer. Each of these has been indispensable at one point or another in the development process.

The application is still growing. My system is whirring away in the background as I write this, sampling the dreams of a learning network. Maybe when it’s developed enough I’ll throw it on the network and give everyone a chance to play.

Learning Network

Removing Blank and Hidden Layers from an SVG

2013-10-10
Programming

We discovered our former graphics designer was including all the hidden layers in his SVG icons. Each icon, that is, had a copy of every other icon stored inside invisibly. The practical upshot is our dashboard icons were in excess of 4MB to load. I wrote a simple script to iterate through and strip invisible layers. For anyone that might find utility:

def remove_blank_layers(image_name):
  tree = ET.ElementTree();
  tree.parse(image_name);
  root = tree.getroot();
  children_to_remove = list();
  for child in root.getchildren():
  if child.attrib.get('display') == 'none':
    children_to_remove.append(child);
  for child in children_to_remove:
    root.remove(child);
  tree.write(image_name);