Credit: MIT
Researchers from MIT, Stanford University, and Adobe Systems presented a new image processing system at the Siggraph Asia conference last week that reduces the bandwidth consumed by server-based image processing by as much as 98.5 percent, and the power consumption by as much as 85 percent.
This is great news not only for casual photographers using their smartphone as their primary camera, but also for every dedicated camera with built-in Wi-Fi. This means that high resolution images can be processed in-camera without a hit on either bandwidth or power consumption.
How the researchers accomplished this feat is pretty clever: the camera sends a highly compressed JPEG version of the image, the server does all the heavy computational processing, then sends back an even smaller file containing just the instructions for modifying the file.
The camera sends a very low-quality JPEG of the image, which saves on bandwidth. To prevent the system from relying on just one pixel (which may have been the result of compression of many subtle colored pixels), the system artificially introduces high-frequency noise: small, random, local variation of the pixel color.
Next, the system performs the desired image processing, such as increasing contrast, sharpening, etc.
The system then divides the image into chunks of 64 x 64 pixels and, for each chunk, it uses a machine-learning algorithm to generate 25 numbers to describe the alteration to apply to that chunk. So for each 64-by-64 pixel patch of the uploaded image, the server sends back just 25 numbers.
Your smartphone (camera) receives those numbers and performs the modifications described by those numbers on its high-resolution image.
This is still early research, and it seems almost impossible that highly detailed images could survive this type of processing, but it does look like cameras (on your smartphone or dedicated) will be relying more and more on central image processing servers. Could the camera of the future be not only mirrorless, but also fully connected, with the grunt of the work performed on powerful servers and images stored in the cloud and immediately accessible by various connected devices?
via MIT News