Benefit:

WorldViewer is an AI-based SLAM system App and Hardware solution using a Nvidia TX1 which can create a visual "portal" to the past! Running on a tablet, the user can hold it up and see through the tablet into yesteryears.

High-level:

Incoming 3D point cloud data is collected and parsed by the TX1 with OpenCV where Deep Learning algorithms based on pytorch and previously trained on 3D objects detects and replaces the environment with historical versions of the objects! This means that, for example, if you are looking at a computer in a house it could be replaced with a slide ruler or similar.

Low-level:

The WorldWindow platform is a Machine Vision real-time simultaneous localization and mapping system constructed from primarily Intel and Nvidia hardware with primarily open source software except for the sensor drivers and firmware of the Intel r200 cameras. Opencv capable devices such as the TX1 connect to the r200 cameras with a single conductor for both power and communication, and they can be worn singly or in an array with the cables threaded through backpack straps or molle gear. The signal is aggregated back to a power source and the edge computer then provides synchronize stereo imaging by overlapping 3D video screens and point Cloud information onto the stream video of the world. Pytorch based deep learning systems perform classification on the point cloud after reducing the 3D data into 2D and the perform standard CNN image classification. Fusion of openCV camera orientation detection and the resulting 3D object of a corresponding item from the past which had been selected by the classifier, is then combined with the video stream. The resulting video stream is thus a fusion of live video and 3D rendering, and appears to show what the world would have looked like in years past. Furthermore, further image filters can be applied such as adding septa toning, random grain spots, and syncing errors to further provide the feeling of old video.

In addition to running on a tablet, the WorldWindow can also be used in conjunction with head mounted displays such as Google Glass or haptic devices. Here the viewing device connects to the TX1 computer and the camera data is passed from the edge, processed, and returned for displaying.