Multi-Aperture Photography and Computational Focusing

Today I want to talk about a technology that's made some huge advancements over the past few years. It's known by an array of names/descriptions: plenoptic photography, lenticular lenses, polydioptric cameras, light-field photography, multi-aperture cameras, fly-eye lenses, programmable aperture photography, etc. No matter what name you choose to call the technology, the processes all bear a similar end-product by utilizing data from multiple lenses.

Before I get into details, I want to address some of the potential possibilities this technology has for photographers:
1) Shoot your photo as usual, but select your focal point on your computer afterwards.
2) Allow the viewer to manually toggle between different focal points.
3) Create precise depth maps using a single image.
4) Creation of 3D images/models and associated phototexturing.

A polydioptric/light-field camera works similarly to a stereoscopic camera, as the final image is constructed using information obtained from multiple lenses. Unlike stereo photography (which uses two lenses that mimic human vision), a polydioptric/light-field lens is more like the eye of a fly which allows for a single scene to be viewed from a number of slightly different angles.

The raw image result is a multitude of images taken at sequential angles; when merged using specialized computer processes, an accurate 3D depth map can be created. The raw image data and generated depth map can then be used to computationally select focus, and/or slightly alter the viewpoint. Furthermore, photographers can expect a much higher degree of quality as the final images combine information from multiple lenses.


Here's a sample from Refocus Imaging that allows viewers to toggle the focal point - test it out:



Adobe's Light Field prototype camera, for instance, uses 19 lenses positioned at varying angles to capture the same scene (see images above) - all 19 shots are projected on different areas of the sensor. When the images are blended an image is produced where literally nothing is out of focus. Further processing of the data generates a 3D image which can be focused selectively by the user. The camera has an output capacity equivalent to about 100 megapixels.

Here's a clip from a press conference where Adobe provides us with a quick look at this new technology in action:



There are a number of key players in the production of this type of photography. At the forefront are Stanford University, Adobe Systems Inc., and The National Taiwan University. Here's one more video that goes into a bit more detail put together by the NTU. Their efforts differ from those of Adobe, but the end-result is similar:


What we're looking at here is even more emphasis being placed on the software/computational side of photography. The fact that photographers in the future will not have to focus their photos is mind blowing - instead, the DOF and focal point will be selected in post in accordance with the editor's taste. This is another technology that poses to revolutionize the industry - I'll definitely be keeping a close eye on advancements.