Update: San Francisco, Cruise, and NeurIPS 2018

It's been a while since I've posted here, so I figured I'd make a post giving a bit of an update on what I've been up to.

To start off, I graduated from Harvey Mudd College and am now living in San Francisco! Right now I'm working at Cruise Automation, a self-driving car company, developing machine-learning models for predicting what people will do next.

Also, a paper that I worked on with other students at HMC has been accepted to NeurIPS 2018! The project was based on a collaboration between HMC and Intel Corporation, with the goal being to design a method for separating and locating sound events in a large, noisy outdoor space. We ended up building a probabilistic model of sound propagation and inter-microphone phase differences, and performed separation and localization using approximate Bayesian maximum a posteriori inference under that model.

The basic idea that makes this work is that, when two microphones are placed very close to one another, sounds arrive at one microphone slightly before the other, with a delay that is dependent on the angle the source makes with the microphones. Since the microphones are close, this time delay turns into a phase change in the audio signal, which can be detected using the short-time Fourier transform. If you have multiple microphone pairs, you can figure out what combination of phase changes you might see based on each possible source location (the probabilistic model). You can then "work backwards" (Bayesian inference) to figure out what sources and locations are most likely (maximum a posteriori) to have produced the inputs you recorded, using some assumptions about source and location smoothness to ensure you get a good separation. If that sounds interesting to you, you can find more information here (which should have the paper and poster available in the next few weeks).

I'll be presenting this paper as a poster at NeurIPS 2018 in Montreal in December. Looking forward to meeting people there and talking about my work in more detail!