Integrating Scene Semantic Knowledge into Image Captioning

Most existing image captioning methods use only the visual information of the image to guide the generation of captions, lack the guidance of effective scene semantic information, and the current visual attention mechanism cannot adjust the focus intensity on the image. In this article, we first propose an improved visual attention model. At each timestep, we calculated the focus intensity coefficient of the attention mechanism through the context information of the model, then automatically adjusted the focus intensity of the attention mechanism through the coefficient to extract more accurate visual information. In addition, we represented the scene semantic knowledge of the image through topic words related to the image scene, then added them to the language model. We used the attention mechanism to determine the visual information and scene semantic information that the model pays attention to at each timestep and combined them to enable the model to generate more accurate and scene-specific captions. Finally, we evaluated our model on Microsoft COCO (MSCOCO) and Flickr30k standard datasets. The experimental results show that our approach generates more accurate captions and outperforms many recent advanced models in various evaluation metrics.

Single-Shot Through-Focus Image Acquisition and Phase Retrieval From Chromatic Aberration and Multi-Angle Illumination

Phase recovery from a stack of through-focus intensity images is an effective non-interference quantitative phase imaging strategy. Nevertheless, the implementations of these methods are expensive and time-consuming because the distance between each through-focus plane has to be guaranteed by precision mechanical moving devices, and the multiple images must be acquired sequentially. In this article, we propose a single-shot through-focus intensity image stack acquisition strategy without any precision movement. Isolated LED units are used to illuminate the sample in different colors from different angles. Due to the chromatic aberration characteristics of the objective, the color-channel defocus images on the theoretical imaging plane are mutually laterally shifted. By calculating the shift amount of each sub-image area in each color channel, the distances between each through-focus image can be obtained, which is a critical parameter in transport of intensity equation (TIE) and alternating projection (AP). Lastly, AP is used to recover the phase distribution and realize the 3D localization of different defocus distances of the sample under test as an example. Both simulation and experiments are conducted to verify the feasibility of the proposed method.

Backward flow of energy for an optical vortex with arbitrary integer topological charge

We analyze the sharp focusing of an arbitrary optical vortex with the integer topological charge m and circular polarization in an aplanatic optical system. Explicit formulas to describe all projections of the electric and magnetic fields near the focal spot are derived. Expressions for the near-focus intensity (energy density) and energy flow (projections of the Pointing vector) are also derived. The expressions derived suggest that for a left-hand circularly polarized optical vortex with m > 2, the on-axis backward flow is equal to zero, growing in the absolute value as a power 2(m – 2) of the radial coordinate. These relations also show that upon the negative propagation, the energy flow rotates around the optical axis.