Facebook has announced that it focus its future in Virtual Reality through its massive multi-billion dollar acquisition of Oculus. Although millions of users are experimenting with other inexpensive means like Google Cardboard, Facebook has reverted to creating more content for those devices. Facebook has taken a major step by announcing that it has made the best 360-degree camera rig.
Some technical problems in making great 3D 360-degree content
Besides the technical challenges in creating immersive video, the creative challenges themselves are also daunting. They affect both the software and the hardware — putting the hardware right is the right approach, as the software’s task become much more tractable. Facebook had some major challenging areas that it faced in ensuring that Surround 360 video would be high-quality. However, it has invested to create a hardware design that addressed the issues.
The major reason that head movement and binocular vision are such powerful tools for humans and other animals to establish depth is that objects at varying depths move differently in our field. However, it isn’t possible to capture video simultaneously from every eye position and possible angle, as we are limited to a discrete number of cameras, and finite bandwidth.
The Surround has 50% overlap between two adjacent cameras, it gets two varying looks at each object, making it possible to construct a stereo view. Utilizing a technology known as optical flow to study the captured images and make a depth map of objects. Surround’s software is able to determine the correct stereo disparity for respective point of the image sphere, and use that to create an appropriate spherical video frame for screening by each eye.
This special feature that allows 3D experience may be the biggest difference between high-end capture rigs like GoPro Odyssey, Jaunt VR and Surround, when compared with consumer-oriented
The optical flow operates by analyzing many image captured over time to make a model of the most likely motion for objects in the scene. Although the camera may be moving, and different objects may be moving at varying speeds, the process is a complex one. In some cases it is not possible to determine with certainty what object motion caused a particular shift of that object in the subsequent frames. To allow the software a chance to accurately build these stereo models, it is crucial that all cameras in the rig capture images at precisely the same time, a feature that is referred to as gen-lock. For Surround 360, it therefore means that 17 sensors all have to fire at once.
Mis-aligned cameras results to messy images and additional stitching headaches, prompted by needing the software to try to align frames after the fact- This issue arises when looking at a panorama taken using your mobile phone or other handheld camera. Some thick metal plates and quite rigid connections enable the Surround 360 make sure its cameras are well aligned, and that they stay aligned.
Facebook’s Surround 360 features
The camera rig features 17 4.1MP industrial-grade imagers from Point Grey. Therefore fourteen are arranged precisely with a 50% overlapping field of view around a circle. Each of them has a wide-angle of 7mm f/2.4 lens. They are supported by three fisheye lenses — one on top and two on the bottom. The fisheye’s have a 2.7mm focal length and an adjustable aperture of f/1.8-f/16. The rig has two lenses on the bottom so that they can shoot around a support pole, making it disappear in the final image. These cameras feature global shutters to reduce the tearing art-facts typical of less-expensive rolling shutter cameras.
Thick aluminum top and bottom plates both insure rigidity — needed for appropriate alignment — and serve as heat reduce to allow longer operation. The structural plates are covered with an 18-gauge powder-coated steel cover.
The camera’s software which is controlled by a web interface to the unit’s Linux-powered CPU is used to control frame rate, exposure, and other settings. To enable all the cameras fire at the same time, Hirose GPIO cables are used, with the top camera serving as the master, and the other 16 slaved off of it. The system’s software runs overnight on servers to produce stereo 8K (and 4K and 6K) 360-degree output, with the option to also generate monoscopic versions — yet another example of why GPU makers like AMD and are so excited about Virtual Reality.
What makes it trend?
There are quite a few other high-end rigs for capturing 360-degree stereo video, like Samsung’s Beyond, JauntVR’s proprietary rig, and Nokia’s OZO. Google’s Jump project uses the GoPro Odyssey, a custom rig built from off-the-shelf GoPro cameras. All of these are expensive. One of the cost drivers for the Surround 360 is its 8K output capability. That drives the use of expensive RAID array of SSDs and 4MP sensors to offload the raw data. A 4K-only version could likely be much less expensive, but allow for immersive experiences appropriate for many viewing conditions.