I am a computer scientist at Adobe's computational photography team led by Marc Levoy. I received my Ph.D. at Princeton University advised by Adam Finkelstein. Before Princeton, I also received 3 years of Ph.D. training at the University of Virginia advised by Connelly Barnes.
Please see my resume for more information.
This paper adapts and extends differentiable rendering to the problem of synthesizer parameter inference, where we search for a set of patch connections and parameters to generate audio that best matches a given target sound. We design novel gradient rules to directly differentiate the synth as a white box program to overcome the discrete nature of the synth programs. Using a simple FM synth, our method better approximates the target sound, and is preferred by AMT users when compared to baselines.
This paper describes a compiler-based framework that extends reverse mode automatic differentiation so as to provide accurate gradients for arbitrary programs, even at discontinuities. We demonstrate the effectiveness of this framework in the context of optimizing parameters of 2D/3D procedural shader programs so that they match target illustrations.
This paper proposes to learn from program traces of procedural fragment shaders -- programs that generate images. At each pixel, we collect the intermediate values computed at program execution, and these data form the input to the learned model. We investigate this learning task for a variety of applications: denoising, learning from partial computation, learning postprocessing filters and learning non-imagery simulations. Our method outperforms baselines both qualitatively and quantitatively. We also conduct a series of analyses that show certain features are important within the trace: these coincide with intuitively important aspects of the program. The important features can be helpful for selecting a good subset of trace features for learning, and even learning from a small subset of the trace already outperforms the baselines.
We present a fully automatic system to optimize the parameters of black-box hardware and software image processing pipelines according to any arbitrary (i.e., application-specific) metric. We leverage a differentiable mapping between the configuration space and evaluation metrics, parameterized by a convolutional neural network that we train in an end-to-end fashion with imaging hardware in-the-loop. Unlike prior art, our differentiable proxies allow for high-dimension parameter search with stochastic first-order optimizers, without explicitly modeling any lower-level image processing transformations.
This paper introduces a general method to approximate the convolution of an arbitrary program with a Gaussian kernel which facilitates smoothing the program. We give several approximations including a novel adaptive Gaussian approximation that is accurate up to the second order in the standard deviation of the smoothing kernel. We construct a compiler framework that automatically chooses approximations for different parts of the program. We then apply this framework to bandlimit procedural shaders.
This paper introduces a novel domain-specific compiler, which translates visual computing programs written in dynamic languages to highly efficient code. We define "dynamic" languages as those such as Python and MATLAB, which feature dynamic typing and can be useful for rapid prototyping, but introduces significant overheads in program execution time. We introduce a compiler framework for accelerating visual computing programs written in general-purpose dynamic languages. Our compiler allows frequently orders of magnitude performance gains over general compilers for dynamic languages by specializing the compiler for visual computation.
Traffic information can be collected from existing inexpensive roadside cameras but extracting it often entails manual work or costly commercial software. We proposed a method that tracks and counts vehicles in the video and to use the tracking information to compute a 3D model for the vehicles and visualise the 2D road situation into 3D. The 3D model can provide feedback on the tracking and counting model for future research.
I received my master degree at the University of Pennsylvania in May 2015. My thesis was "Road Intersecting Monitoring from Video with 3D Reconstruction", advised by Dr. Daniel Lee and Dr. Camillo Taylor.
I received my bachelor degree at Huazhong University of Science and Technology in June 2013. My thesis was "Motion Detection from Video Surveillance", advised by Dr. Hao Wen.