Video recording of ultrafast phenomena such as dynamic events in molecular biology would transform our understanding of a range of phenomena. However, using a detector array based on CCD or CMOS technologies is fundamentally limited by the sensor's on-chip storage and data transfer speed. To get around this problem, the most practical approach is to use a streak camera. In this ultrafast imaging device, the incident light first passes through a narrow entrance slit (usually 50μm wide) and is imaged onto the photocathode of a streak tube. Here, the incident light is converted to photoelectrons, which are accelerated by an accelerating mesh. A pair of electrodes then applies a sweeping (i.e., time-varying) voltage along the axis perpendicular to the device's entrance slit. Because of this sweeping voltage, electrons arriving at different times are deflected to different spatial positions, and these electrons are then multiplied by a microchannel plate. They subsequently bombard a phosphor screen and are converted back into light. The phosphor screen is imaged to a CCD, which records the image. However, the resultant image is normally 1D: only a single line of the scene can be seen at a time. Acquiring a 2D image requires mechanical scanning across the entire field of view, which poses severe restrictions on the recordable scenes because the event itself must be repetitive.

​

Previous approaches to enable 2D ultrafast imaging of nonrepetitive events include sequentially timed all-optical mapping photography and parallel streak imaging using a tilted lenslet array. However, most of them either rely on active illumination or suffer from significant throughput loss. To overcome these limitations, we have developed a new computational ultrafast imaging method, referred to as compressed ultrafast photography (CUP), which can capture 2D dynamic scenes at up to 100 billion frames per second. Akin to a conventional photographic camera, CUP is receive-only, thereby allowing high-speed video recording of a variety of luminescent—such as fluorescent or bioluminescent—objects.

​

Based on the concept of compressed sensing, CUP works by encoding the input scene with a random binary pattern in the spatial domain, followed by shearing the resultant image in a streak camera with a fully opened entrance slit. The random binary pattern, which encodes the scene as is standard in compressed sensing, also serves as the key to unlock and retrieve time information from the input scene in the subsequent image reconstruction. CUP forms images by successively applying three operators—a spatial encoding operator, temporal shearing operator, and spatiotemporal integration operator—on the input time-lapse event I(x, y, t), where x and y are spatial coordinates, and t is time. The image is reconstructed by solving the inverse problem of these three processes. A typical input scene is regarded to be sparse in the spatiotemporal domain, meaning that it can be represented as a matrix in which a large number of elements are zeros. Under this condition, the original event datacube can be reasonably estimated by employing a two-step iterative shrinkage/thresholding algorithm.

​

Figure 1 shows CUP's system schematic. The scene is first imaged onto an intermediate plane by a camera lens. Then this image is relayed to a spatial encoding unit, a digital micromirror device (DMD), which consists of tens of thousands of small MEMS (microelectromechanical systems) mirrors. Each micromirror can be individually turned ‘on’ and ‘off.’ The light reflected from the ‘on’ micromirrors is collected by a microscope objective and passed to the streak camera, while the light reflected from the ‘off’ micromirrors is out of the objective's collecting angle. In the streak camera, the entrance slit is opened to its maximum width, allowing the resultant 2D image to be temporally sheared along the vertical axis. The amount of shearing is dependent on the arrival time of incident photons. The final spatiotemporally multiplexed image is captured by a CCD within a single exposure. Using CUP, we can reconstruct an event datacube with 150×150×350 (x, y, t) voxels (volume elements).