Creating data visualizations is a lossy process, a user's code and data is compiled in the form of a static image. These images are the default form of data visualizations, and are ubiquitous across both social media and academia.

As visualization researchers, this lossy compilation into images makes it very challenging to experiment and tweak with design decisions of the original author.

In this work, we present a novel deep-learning based algorithm that can decompile an image visualization, allowing researchers to easily edit and inspect its visual design. Our method and domain-specific language is more general than previous approaches and is extensible to more types of data visualizations without the need for expert-tuned heuristics.



I had always wanted to recreate the bullet-time effect from the Matrix. But this gets really expensive and doesn't scale well if you are using a $300 DSLR for every camera. PiShot makes use of $5 Raspberry Pi Camera modules to get a similar effect.

The main challenge was to hack the Pi camera drivers to work in global exposure mode instead of rolling shutter, and making sure that the 16 Raspberry Pi's were all timed just right to the external strobe light.

Technical Report


For performing large scale reinforcement learning experiments, researchers tend to replicate their environments across a distributed cluster. However, this doesn't allow agent-agent interactions across these replicated environments.

Chunky is a distributed, fault tolerant multiplayer game framework that can shard any spatial game-like simulation across a cluster of machines.

Technical Report

I think many amazing ideas have felt the wrath of PowerPoint. Don't get me wrong, it's a great tool for most things, but humans are visual creatures, we like pretty things move and whizz in snazzy ways.

I wrote KeyFrames to allow me to create animated presentations easily. It runs super fast thanks to WebGL in the browser. It has all the collaborative tools that one has come to expect of web apps.


  • First Place, MIT's Web Programming Competition (web.lab)

This project was done at the MIT Media Lab Design Innovation Workshop 2014. We used a Kinect to figure out where the paper is in 3D space, and transformed our graphics onto the sheet using a projector.


Shreyas Kapur's Projects