Planning the potential future during multi-item visual working memory

Working memory allows us to retain visual information to guide upcoming future behavior. In line with this future-oriented purpose of working memory, recent studies have shown that action planning occurs during encoding and retention of a single visual item, for which the upcoming action is certain. We asked whether and how this extends to multi-item visual working memory, when visual representations serve the potential future. Human participants performed a visual working memory task with a memory-load manipulation (one/two/four items), and a delayed orientation-reproduction report (of one item). We measured EEG to track 15-25 Hz beta activity in electrodes contralateral to the required response hand - a canonical marker of action planning. We show an attenuation of beta activity, not only in load one (with one certain future action), but also in load two (with two potential future actions), compared to load four (with low prospective-action certainty). Moreover, in load two, potential action planning occurs regardless whether both visual items afford similar or dissimilar manual responses; and it predicts the speed of ensuing memory-guided behavior. This shows that potential action planning occurs during multi- item visual working memory, and brings the perspective that working memory helps us prepare for the potential future.

Time for Action: Neural Basis of the Costs and Benefits of Temporal Predictability for Competing Response Choices

The brain can anticipate the time of imminent events to optimize sensorimotor processing. Yet, there can be behavioral costs of temporal predictability under situations of response conflict. Here, we sought to identify the neural basis of these costs and benefits by examining motor control processes in a combined electroencephalography–EMG study. We recorded electrophysiological markers of response activation and inhibition over motor cortex when the onset-time of visual targets could be predicted, or not, and when responses necessitated conflict resolution, or not. If stimuli were temporally predictable but evoked conflicting responses, we observed increased intertrial consistency in the delta range over the motor cortex involved in response implementation, perhaps reflecting increased response difficulty. More importantly, temporal predictability differentially modulated motor cortex activity as a function of response conflict before the response was even initiated. This effect occurred in the hemisphere ipsilateral to the response, which is involved in inhibiting unwanted actions. If target features all triggered the same response, temporal predictability increased cortical inhibition of the incorrect response hand. Conversely, if different target features triggered two conflicting responses, temporal predictability decreased inhibition of the incorrect, yet prepotent, response. This dissociation reconciles the well-established behavioral benefits of temporal predictability for nonconflicting responses as well as its costs for conflicting ones by providing an elegant mechanism that operates selectively over the motor cortex involved in suppressing inappropriate actions just before response initiation. Taken together, our results demonstrate that temporal information differentially guides motor activity depending on response choice complexity.

Dissociating the Neural Correlates of Planning and Executing Hierarchical Task Sets

Task processing and task representation, two facets of cognitive control, are both supported by lateral frontal cortex (LFC). However, processing and representation have largely been investigated separately, so it is unknown if they are distinguishable aspects of control or if they are complementary descriptions of the same mechanism. Here, we explored this by combining a hierarchical task mapping with a pre-cueing procedure. Participants made match/non-match judgments on features of pairs of stimuli. Cues presented at the start of each trial indicated the judgment domain (spatial/non-spatial), the response hand, both, or neither, giving variable amounts of information to the subject at each time point in the trial. Our results demonstrated that regions throughout LFC supported task processing, indicated by an influence of time point on their BOLD activity levels. A subset of regions in left caudal LFC also supported task representation, indicated by an interaction between time point and cue information; we termed this subgroup the "CuexTime" group. This interaction effect was not seen in the remaining LFC regions, which only showed a main effect of time consistent with involvement in task processing; we termed this subgroup the "Time" group. These results suggest that task representation is one component of task processing, confined to the "CuexTime group" in left caudal LFC, while other regions in our task support other aspects of task processing. We further conducted an exploratory investigation of connectivity between regions in the "CuexTime" and "Time" groups and their potential relationship to networks that support distinct cognitive control functions.

Enhanced Multimodal Biometric Recognition Based upon Intrinsic Hand Biometrics

In the proposed study, we examined a multimodal biometric system having the utmost capability against spoof attacks. An enhanced anti-spoof capability is successfully demonstrated by choosing hand-related intrinsic modalities. In the proposed system, pulse response, hand geometry, and finger–vein biometrics are the three modalities of focus. The three modalities are combined using a fuzzy rule-based system that provides an accuracy of 92% on near-infrared (NIR) images. Besides that, we propose a new NIR hand images dataset containing a total of 111,000 images. In this research, hand geometry is treated as an intrinsic biometric modality by employing near-infrared imaging for human hands to locate the interphalangeal joints of human fingers. The L2 norm is calculated using the centroid of four pixel clusters obtained from the finger joint locations. This method produced an accuracy of 86% on the new NIR image dataset. We also propose finger–vein biometric identification using convolutional neural networks (CNNs). The CNN provided 90% accuracy on the new NIR image dataset. Moreover, we propose a robust system known as the pulse response biometric against spoof attacks involving fake or artificial human hands. The pulse response system identifies a live human body by applying a specific frequency pulse on the human hand. About 99% of the frequency response samples obtained from the human and non-human subjects were correctly classified by the pulse response biometric. Finally, we propose to combine all three modalities using the fuzzy inference system on the confidence score level, yielding 92% accuracy on the new near-infrared hand images dataset.

Sidestepping spatial confounds in object-based correspondence effects: The Bimanual Affordance Task (BMAT)

Tucker and Ellis found that when participants made left/right button-presses to indicate whether objects were upright or inverted, responses were faster when the response hand aligned with the task-irrelevant handle orientation of the object. The effect of handle orientation on response times has been interpreted as evidence that individuals perceive grasp affordances when viewing briefly presented objects, which in turn activate grasp-related motor systems. Although the effect of handle alignment has since been replicated, there remains doubt regarding the extent to which the effect is indeed driven by affordance perception. Objects that feature in affordance-compatibility paradigms are asymmetrical and have laterally protruding handles (e.g., mugs) and thus confound spatial and affordance properties. Research has attempted to disentangle spatial compatibility and affordance effects with varying results. In this study, we present a novel paradigm with which to study affordance perception while sidestepping spatial confounds. We use the Bimanual Affordance Task (BMAT) to test whether object affordances in symmetrical objects facilitate response times. Participants ( N = 36) used one of three (left unimanual/right unimanual/bimanual) responses to indicate the colour of presented objects. Objects afforded either a unimanual (e.g., handbag) or a bimanual (e.g., laundry hamper) grasp. Responses were faster when the afforded grasp corresponded with the response type (unimanual vs. bimanual), suggesting that affordance effects exist independent of spatial compatibility.

Compatibility between object size and response side in grasping: the left hand prefers smaller objects, the right hand prefers larger objects

It has been proposed that the brain processes quantities such as space, size, number, and other magnitudes using a common neural metric, and that this common representation system reflects a direct link to motor control, because the integration of spatial, temporal, and other quantity-related information is fundamental for sensorimotor transformation processes. In the present study, we examined compatibility effects between physical stimulus size and spatial (response) location during a sensorimotor task. Participants reached and grasped for a small or large object with either their non-dominant left or their dominant right hand. Our results revealed that participants initiated left hand movements faster when grasping the small cube compared to the large cube, whereas they initiated right hand movements faster when grasping the large cube compared to the small cube. Moreover, the compatibility effect influenced the timing of grip aperture kinematics. These findings indicate that the interaction between object size and response hand affects the planning of grasping movements and supports the notion of a strong link between the cognitive representation of (object) size, spatial (response) parameters, and sensorimotor control.

Neural Dynamics of Reward-Induced Response Activation and Inhibition

Reward-predictive stimuli can increase an automatic response tendency, which needs to be counteracted by effortful response inhibition when this tendency is inappropriate for the current task. Here we investigated how the human brain implements this dynamic process by adopting a reward-modulated Simon task while acquiring EEG and fMRI data in separate sessions. In the Simon task, a lateral target stimulus triggers an automatic response tendency of the spatially corresponding hand, which needs to be overcome if the activated hand is opposite to what the task requires, thereby delaying the response. We associated high or low reward with different targets, the location of which could be congruent or incongruent with the correct response hand. High-reward targets elicited larger Simon effects than low-reward targets, suggesting an increase in the automatic response tendency induced by the stimulus location. This tendency was accompanied by modulations of the lateralized readiness potential over the motor cortex, and was inhibited soon after if the high-reward targets were incongruent with the correct response hand. Moreover, this process was accompanied by enhanced theta oscillations in medial frontal cortex and enhanced activity in a frontobasal ganglia network. With dynamical causal modeling, we further demonstrated that the connection from presupplementary motor area (pre-SMA) to right inferior frontal cortex (rIFC) played a crucial role in modulating the reward-modulated response inhibition. Our results support a dynamic neural model of reward-induced response activation and inhibition, and shed light on the neural communication between reward and cognitive control in generating adaptive behaviors.

Compatibility between object size and response side in grasping: The left hand prefers smaller objects, the right hand prefers larger objects

It has been proposed that the brain processes quantities such as space, size, number, and other magnitudes using a common neural metric, and that this common representation system reflects a direct link to motor control, because the integration of spatial, temporal, and other quantity-related information is fundamental for sensorimotor transformation processes. In the present study, we examined compatibility effects between physical stimulus size and spatial (response) location during a sensorimotor task. Participants reached and grasped for a small or large object with either their non-dominant left or their dominant right hand. Our results revealed that participants initiated left hand movements faster when grasping the small cube compared to the large cube, whereas they initiated right hand movements faster when grasping the large cube compared to the small cube. Moreover, the compatibility effect influenced the timing of grip aperture kinematics. These findings indicate that the interaction between object size and response hand affects the planning of grasping movements and supports the notion of a strong link between the cognitive representation of (object) size, spatial (response) parameters, and sensorimotor control.

Compatibility between object size and response side in grasping: The left hand prefers smaller objects, the right hand prefers larger objects

It has been proposed that the brain processes quantities such as space, size, number, and other magnitudes using a common neural metric, and that this common representation system reflects a direct link to motor control, because the integration of spatial, temporal, and other quantity-related information is fundamental for sensorimotor transformation processes. In the present study, we examined compatibility effects between physical stimulus size and spatial (response) location during a sensorimotor task. Participants reached and grasped for a small or large object with either their non-dominant left or their dominant right hand. Our results revealed that participants initiated left hand movements faster when grasping the small cube compared to the large cube, whereas they initiated right hand movements faster when grasping the large cube compared to the small cube. Moreover, the compatibility effect influenced the timing of grip aperture kinematics. These findings indicate that the interaction between object size and response hand affects the planning of grasping movements and supports the notion of a strong link between the cognitive representation of (object) size, spatial (response) parameters, and sensorimotor control.

The influence of number magnitude on continuous swiping movements

There is accumulating evidence that numerical information influences the way in which we perform bodily movements. Specifically, the idea that our cognitive representations of numbers and space interact is supported by systematic associations of space with both number magnitude (SNARC effect) and number parity (MARC effect). However, whether this influence is bound to the left or right side of space or to the hand with which we perform the movement remains debated. One novel and interesting way to disentangle these factors is to use movement responses in which hand and movement direction can be dissociated. In the present study, participants moved a central object to the left or right side on a touchscreen with their index fingers as response to a parity judgment and magnitude classification task. We observed significant SNARC effects in both tasks. Number magnitude and response direction interacted, but magnitude and response hand did not. This indicated that the SNARC effect can be independent of the responding hand. Importantly, however, a MARC effect was observed not only in an interaction between response direction and parity, but also in an interaction between response hand and parity, suggesting that response hand plays a role in the interaction between physical space and parity. Additionally, number magnitude influenced the amplitude of participants’ response movements, with larger numbers eliciting longer movements. These results indicate that space, magnitude and parity interact on different levels that can be unraveled in a paradigm utilizing continuous movements such as swiping.