hexahedria

Daniel Johnson's personal fragment of the web

Introducing the GGT-NN

Recently, I submitted a paper titled “Learning Graphical State Transitions” to the ICLR conference! In it, I describe a new type of neural network architecture, called a Gated Graph Transformer Neural Network, that is designed to use graphs as an internal state. I demonstrate its performance on the bAbI tasks as well as on some other tasks with complex rules. While the main technical details are provided in the paper, I figured it would be worthwhile to describe the motivation and basic ideas here.

Note: Before I get too far into this post, if you have read my paper and are interested in replicating or extending my experiments, the code for my model is available on GitHub.

Another thing that I’ve noticed is that almost all of the papers on machine learning are about successes. This is an example of an overall trend in science to focus on the positive results, since they are the most interesting. But it can also be very useful to discuss the negative results as well. Learning what doesn’t work is in some ways just as important as learning what does, and can save others from repeating the same mistakes. During my development of the GGT-NN, I had multiple iterations of the model, which all failed to learn anything interesting. The version of the model that worked was thus a product of an extended cycle of trial and error. In this post I will try to describe the failed models as well, and give my speculative theories for why they may not have been as successful.

Read more...

Spiral Clock for Pebble

I had some free time recently, and felt like making a new watchface for my Pebble Time. The basic idea is that the outer spiral makes one revolution every hour, but the previous path of the spiral remains. Over time, the whole watchface fills up with the growing spiral.

Spiral Clock

As with my previous watchface, the code is available on GitHub. You can also download it from the Pebble App Store if you have a Pebble (it should work on the original Pebble, the Pebble Time, and the Pebble Time Round).

Read more...

Summer Research on the HMC Intelligent Music Software Team

This summer, I had the chance to do research at Mudd as part of the Intelligent Music Software team. Every year, under the advisement of Prof. Robert Keller, members of the Intelligent Music Software team work on computational jazz improvisation, usually in connection with the Impro-Visor software tool. Currently, Impro-Visor uses a grammar-based generation approach for most of its generation ability. The goal this summer was to try to integrate neural networks into the generation process.

Read more...

"LEG Processor for Education" at EWME 2016

Last semester, I was part of the Clay Wolkin Fellowship at Harvey Mudd. The fellowship consists of a group of students (mostly Engineering majors, but some CS also) who work on interesting electrical-engineering-focused projects. The project I worked on was the “LEG Processor”, an open-source pipelined processor that implements the ARMv5 instruction set and can boot the Linux kernel (3.19) in simulation. We recently published a paper describing our work in the European Workshop for Microelectronics! You can read the paper here. Or read on for a high-level overview of the work I did on the project.

Read more...

From Wordpress to Jekyll

Until now, my blog has been happily running on WordPress, hosted on a VPS provided by DigitalOcean. Recently I decided I didn’t really need all of the complexity of running a VPS just to host my website. So instead, I’m migrating my blog over to Jekyll, a static site generator used by GitHub Pages.

Read more...

Discrete and Computational Geometry Projects

At Harvey Mudd, I’m taking a cool class called “Discrete and Computational Geometry”, a special topics course taught by Professor Satyan Devadoss. It’s a very interesting class. In lieu of normal problem sets, we instead do a bunch of group projects, each one very freeform. The basic instructions are “go make something related to this class”. Here are a couple of the projects my group made:

Read more...

© . All rights reserved. | Top