Neural network based 3D tracking with a graphene transparent focal stack imaging system

Recent years have seen the rapid growth of new approaches to optical imaging, with an emphasis on extracting three-dimensional (3D) information from what is normally a two-dimensional (2D) image capture. Perhaps most importantly, the rise of computational imaging enables both new physical layouts of optical components and new algorithms to be implemented. This paper concerns the convergence of two advances: the development of a transparent focal stack imaging system using graphene photodetector arrays, and the rapid expansion of the capabilities of machine learning including the development of powerful neural networks. This paper demonstrates 3D tracking of point-like objects with multilayer feedforward neural networks and the extension to tracking positions of multi-point objects. Computer simulations further demonstrate how this optical system can track extended objects in 3D, highlighting the promise of combining nanophotonic devices, new optical system designs, and machine learning for new frontiers in 3D imaging.

Feature Detection of Focused Plenoptic Camera Based on Central Projection Stereo Focal Stack

Fast and accurate feature extraction can lay a solid foundation for scene reconstruction and visual odometry. However, this has been rather a difficult problem for the focused plenoptic camera. In this paper, to the best of our knowledge, we first introduce an accurate and fast feature extraction algorithm based on central projection stereo focal stack (CPSFS). Specifically, we propose a refocusing algorithm that conforms to the central projection with regard to the center of main lens, which is more accurate than traditional one. On this basis, the feature is extracted on the CPSFS without calculating dense depth maps and total focus images. We verify the precision and efficiency of the proposed algorithm through simulated and real experiments, and give an example of scene reconstruction based on the proposed method. The experimental results show that our feature extraction algorithm is able to support the feature-based scene reconstruction via focused plenoptic camera.

Neural Network Based 3D Tracking with a Graphene Transparent Focal Stack Imaging System

Recent years have seen the rapid growth of new approaches to optical imaging, with an emphasis on extracting three-dimensional (3D) information from what is normally a two-dimensional (2D) image capture. Perhaps most importantly, the rise of computational imaging, defined as the synergistic design of optical systems in conjunction with image reconstruction algorithms, enables both new physical layouts of optical components and new algorithms to be implemented. This paper concerns the convergence of two advances: the development of transparent photodetectors with high responsivity, and the rapid expansion of the capabilities of machine learning including the development of powerful neural networks. In particular, we demonstrate that the use of transparent photodetector arrays stacked vertically along the optical axis of an imaging system, called a focal stack, together with a feedforward neural network, provides a powerful new approach to real-time 3D optical imaging including object tracking. The focal stack imaging system is realized through the development of graphene transparent photodetector arrays. As a proof-of concept, 3D tracking of point-like objects was successfully demonstrated with multilayer feedforward neural networks, which was then extended for tracking of multi-point objects in position. Our computer model further demonstrates how this optical system can track extended objects in 3D, highlighting the promise of combining nanophotonic devices, new optical system designs, and machine learning for new frontiers in 3D imaging.