Apple’s new iPhone SE is the first – and so far only – iPhone to rely solely on machine learning for depth estimation in portrait mode.
IPhone SE can create depth maps from a single 2D image using machine learning. Credit: Halide
The iPhone SE, released in April, appears to be largely a copy of the iPhone 8, down to the monocular single-lens camera. But, under the hood, there is much more to be done for depth estimation than any other iPhone before it.
According to a blog by the creators of the Halide camera application, the iPhone SE is the first in the Apple range to use “monocular depth estimation by image”. This means that it is the first iPhone that can create a portrait blur effect using a single 2D image.
In previous iPhones, Portrait mode required at least two cameras. Indeed, the best source of depth informationon has long been to compare two images from two slightly different places. Once the system compares these images, it can separate the subject of a photo from the background, allowing a blurry or “bokeh effect”.
The iPhone XR has changed that, introducing support for portrait mode through the use of “focus pixels” from the sensor, which could produce an approximate depth map. But while the new iPhone SE has focus pixels, its old hardware doesn’t have the required coverage for deep mapping purposes.
“The new iPhone SE cannot use focus pixels because its old sensor does not have sufficient coverage,” wrote Ben Sandofsky of Halide. An iFixit teardown revealed on Monday that the camera sensor on the iPhone SE is essentially interchangeable with the iPhone 8.
Instead, the entry-level iPhone produces depth maps entirely through machine learning. It also means that he can produce portrait mode photos from his front and rear cameras. This is undoubtedly possible thanks to the high-end A13 Bionic chipset under its hood.
The depth informationon isn’t perfect, says Halide, but it’s an impressive feat considering the relative hardware limitations of a three-year-old single-sensor camera configuration. Likewise, Portrait mode on the iPhone SE only works on people, but Halide says that the new version of its application allows bokeh effects on non-human subjects on the iPhone SE.