In a study addressing future use of video-telephone systems, the ability of 52 young adults with normal hearing to shadow verbal passages was assessed when they could both hear and observe the speaker. This performance was compared to performance in an audio-alone condition. The passages were presented against an irrelevant background message. Effects of varying the video frame rate (i.e., the rate at which frames were sampled) were examined, using rates of 8.3, 12.5, 16.7, and 25 Hz. The presence of the visual image of the relevant speaker always improved performance when compared with a baseline audio-alone condition. The motion of the speaker’s face may generally support the focusing of attention on the target message. However, effects of video frame rate were also apparent, suggesting that specific visual cues became available as the temporal resolution improved. When frame rates of 8.3 Hz and the maximum available rate of 25 Hz were compared, shadowing performance was significantly better across the subject group at the higher frame rate. The comparison of frame rates of 12.5 and 25 Hz did not show reliably improved performance across the whole subject group at 25 Hz, although a small number of subjects seemed to benefit. This suggests there may be some differences in the visual cues used by subjects and consequent differences in the way individuals perform under different frame rates. Performance at 16.7 and 25 Hz did not differ, and this is consistent with previous research that tested people with hearing loss. A frame rate of 16.7 Hz may therefore be adequate for the transmission of facial images via a video communication link to a broad range of users; at the lower frame rates, the performance of users is likely to suffer.
Video Frame Rate Up-Conversion via Spatio-Temporal Generative Adversarial Networks
