Deep reinforcement learning of cell movement in the early stage of C.elegans embryogenesis

An adaptive nutrient control method for a plant factory is proposed. The method is based on a Reinforcement Learning (RL) modified for a target in which the same state never comes back during a single episode and a reward is given after a very long delay. In application such as plant growth control, one episode takes a very long time period, and a rapid convergence to a prospective control solution is essential while an extensive exploration is needed since there is usually no precise model available. A method like reinforcement learning is useful for a problem having no reference model. But a necessity of exploration does not match the need for rapid convergence, and a new balancing method is needed. In this research, an average reward distribution method is proposed, which is similar to the profit sharing method but effects more extensively on finding much prospective early solutions, while guaranteeing to converge into a rational solution in a long run. An experiment is conducted in a simple plant factory system, which shows that at least standard reinforcement learning is insufficient for this type of problem. Computer simulations show that the method has good effects on acquiring prospective control policy at early stage comparing to a standard reinforcement learning and a profit sharing method.

Download Full-text

Embedding High-Level Knowledge into DQNs to Learn Faster and More Safely

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i09.7091 ◽

2020 ◽

Vol 34 (09) ◽

pp. 13608-13609

Author(s):

Zihang Gao ◽

Fangzhen Lin ◽

Yi Zhou ◽

Hao Zhang ◽

Kaishun Wu ◽

...

Keyword(s):

Decision Making ◽

Reinforcement Learning ◽

Early Stage ◽

Training Process ◽

Learning Efficiency ◽

Embedded Knowledge ◽

High Level

Deep reinforcement learning has been successfully applied in many decision making scenarios. However, the slow training process and difficulty in explaining limit its application. In this paper, we attempt to address some of these problems by proposing a framework of Rule-interposing Learning (RIL) that embeds knowledge into deep reinforcement learning. In this framework, the rules dynamically effect the training progress, and accelerate the learning. The embedded knowledge in form of rule not only improves learning efficiency, but also prevents unnecessary or disastrous explorations at early stage of training. Moreover, the modularity of the framework makes it straightforward to transfer high-level knowledge among similar tasks.

Download Full-text

Non-Parametric Stakeholder Discovery

Advances in Data Mining and Database Management - Emerging Methods in Predictive Analytics ◽

10.4018/978-1-4666-5063-3.ch006 ◽

2014 ◽

pp. 97-126

Author(s):

John Benjamin Cassel

Keyword(s):

Machine Learning ◽

Reinforcement Learning ◽

Early Stage ◽

Parametric Models ◽

Risk Governance ◽

Initial Reaction ◽

Strategic Assessment ◽

Initial Interaction ◽

Risk Situation ◽

Non Parametric

This chapter provides a stakeholder discovery model for distributed risk governance suitable to machine learning and decision-theoretic planning. Distributed risk governance concerns when the underlying risk is not localized or has unknown locality so that any initial interaction with stakeholders is limited and educational and participatory initiatives are costly. Therefore, expecting the initial reaction to communications is critical. To capture this initial reaction, the authors sample the population of potential stakeholders to discover both their concerns and knowledge while handling inaccuracies and contradictions. This chapter provides a stakeholder discovery model that can accommodate these inconsistencies. Stakeholder discovery provides a timely strategic assessment of the risk situation. This assessment forecasts projected stakeholder actions to find if those actions are in line with their strategic interests or if there are better choices using reinforcement learning. Unlike other reinforcement learning formulations, it does not take the state space, criteria, potential observations, other agents, actions, or rewards for granted, but discovers these factors non-parametrically. Overall, this chapter introduces machine learning researchers and risk governance professionals to the compatibility between non-parametric models and early-stage stakeholder discovery problems and addresses widely known biases and deficits within risk governance and intelligence practices.

Download Full-text

S82. REINFORCEMENT LEARNING IMPAIRMENT IN PATIENTS WITH EARLY-STAGE PSYCHOTIC BIPOLAR DISORDER

Schizophrenia Bulletin ◽

10.1093/schbul/sbaa031.148 ◽

2020 ◽

Vol 46 (Supplement_1) ◽

pp. S65-S65

Author(s):

Joe Kwun Nam Chan ◽

Mary Chung Man Ng ◽

Cheuk Fei Wong ◽

Sui Fung Wo ◽

Corine Sau Man Wong ◽

...

Keyword(s):

Bipolar Disorder ◽

Reinforcement Learning ◽

Negative Reinforcement ◽

Early Stage ◽

Reward Sensitivity ◽

Learning Performance ◽

Loss Avoidance ◽

Learning Measures ◽

Learning Impairment ◽

Psychotic Bipolar Disorder

Abstract Background Abnormal reward sensitivity is a biosignature to mood disorders spectrum. Recent data suggested either elevated or preserved positive but impaired negative reinforcement learning in patients with bipolar disorder. Functional MRI studies provided extra evidence on heightened reward sensitivity in manic patients. Of note, these investigations mostly rest on chronically ill samples, conditions of whom may have been confounded by prolonged exposure to medications. This study aims to examine reinforcement learning performance and its relationship with symptomology in patients with early-stage psychotic bipolar disorder (BDP). Methods This study is based on 38 patients with early-stage BDP (defined by having received psychiatric treatment for first-episode BDP within 3 years since service entry) who have been euthymic for at least eight weeks and 40 demographically-matched controls. Reinforcement learning performance was evaluated using Gain-vs-Loss-Avoidance Task (GLAT), which measured the correct responses in both gain and loss-avoidance pairs with reinforcement probability at either 90% or 80% across four blocks in the training phase and one block in the test/transfer phase. Comparison analyses on reinforcement learning performance were conducted on two groups. Associations of reinforcement learning measures with symptom scores, cognitive functions and functioning measures were also tested. Results There was no group difference in gender, age or education level. Repeated-measures analysis of variance (ANOVA) showed significant main effects of group (F=6.52, p=0.013), block (F=43.71, p<0.001), probability (F= 5.58, p<0.001), and block x group (F=2.87, p=0.040) interaction. Post-hoc tests revealed that controls performed better than patients across blocks (p<0.05). Patients also showed a lower lose-shift rate (t= 2.21, p=0.03) and punishment-driven learning accuracy rates (t=2.42, p=0.018) than controls. Marginally significant main effect of stimulus pair (F=3.98, p=0.05) was revealed in the test phase, with controls showing a significantly higher preference in Frequent Winner vs Frequent Loser (FWFL) pair than patients (t=-2.25, p=0.028). No significant correlations between learning measures and any of the symptom dimensions in patient sample. Discussion Our preliminary findings provided a brief evidence on the negative reinforcement learning impairment in early-stage BDP patients. Further investigation is required to verify and confirm our results of impaired negative reinforcement learning in the initial course of bipolar disorder.

Download Full-text

Early Rumor Detection Based on Deep Recurrent Q-Learning

Security and Communication Networks ◽

10.1155/2021/5569064 ◽

2021 ◽

Vol 2021 ◽

pp. 1-13

Author(s):

Wei Wang ◽

Yuchen Qiu ◽

Shichang Xuan ◽

Wu Yang

Keyword(s):

Reinforcement Learning ◽

Online Social Networks ◽

Social Life ◽

Negative Impact ◽

Early Stage ◽

Optimization Strategy ◽

Q Learning ◽

Detection Model ◽

Model Based ◽

Rumor Detection

Online social networks provide convenient conditions for the spread of rumors, and false rumors bring great harm to social life. Rumor dissemination is a process, and effective identification of rumors in the early stage of their appearance will reduce the negative impact of false rumors. This paper proposes a novel early rumor detection (ERD) model based on reinforcement learning. In the rumor detection part, a dual-engine rumor detection model based on deep learning is proposed to realize the differential feature extraction of original tweets and their replies. A double self-attention (DSA) mechanism is proposed, which can eliminate data redundancy in sentences and words at the same time. In the reinforcement learning part, an ERD model based on Deep Recurrent Q-Learning Network (DRQN) is proposed, which uses LSTM to learn the state sequence features, and the optimization strategy of the reward function is to take into account the timeliness and accuracy of rumor detection. Experiments show that, compared with existing methods, the ERD model proposed in this paper has a greater improvement in the timeliness and detection rate of rumor detection.

Download Full-text

Correlated Studies of Cellular Interactions Using TEM, Freeze Etching, and SEM

Proceedings, annual meeting, Electron Microscopy Society of America ◽

10.1017/s042482010005161x ◽

1974 ◽

Vol 32 ◽

pp. 320-321

Author(s):

J. P. Revel

Keyword(s):

Developmental Stages ◽

Three Dimensional ◽

Cell Movement ◽

Cellular Interactions ◽

Serial Sections ◽

Cell Sheets ◽

Freeze Etching ◽

Early Developmental

Movement of individual cells or of cell sheets and complex patterns of folding play a prominent role in the early developmental stages of the embryo. Our understanding of these processes is based on three- dimensional reconstructions laboriously prepared from serial sections, and from autoradiographic and other studies. Many concepts have also evolved from extrapolation of investigations of cell movement carried out in vitro. The scanning electron microscope now allows us to examine some of these events in situ. It is possible to prepare dissections of embryos and even of tissues of adult animals which reveal existing relationships between various structures more readily than used to be possible vithout an SEM.

Download Full-text

Quantitative Freeze Fracture Studies of Gap Junctions in Somatic Cell Hybrids, Their Parents, and Revertants

Proceedings, annual meeting, Electron Microscopy Society of America ◽

10.1017/s042482010009107x ◽

1976 ◽

Vol 34 ◽

pp. 218-219

Author(s):

W. J. Larsen ◽

R. Azarnia ◽

W. R. Loewenstein

Keyword(s):

Gap Junctions ◽

Striated Muscle ◽

Freeze Fracture ◽

Cell Movement ◽

Dye Molecules ◽

Somatic Cell Hybrids ◽

Cell Coupling ◽

Normal Cells ◽

Mediate Cell ◽

Almost All

Although the physiological significance of the gap junction remains unspecified, these membrane specializations are now recognized as common to almost all normal cells (excluding adult striated muscle and some nerve cells) and are found in organisms ranging from the coelenterates to man. Since it appears likely that these structures mediate the cell-to-cell movement of ions and small dye molecules in some electrical tissues, we undertook this study with the objective of determining whether gap junctions in inexcitable tissues also mediate cell-to-cell coupling.To test this hypothesis, a coupling, human Lesh-Nyhan (LN) cell was fused with a non-coupling, mouse cl-1D cell, and the hybrids, revertants, and parental cells were analysed for coupling with respect both to ions and fluorescein and for membrane junctions with the freeze fracture technique.

Download Full-text

Alterations in the number of motoneurons containing immunoreactive calcitonin gene-related peptide(CGRP)& choline acetyltransferase(ChAT) in the cervical spinal cord of the wobbler mouse during the development of the motoneuron disease

Proceedings, annual meeting, Electron Microscopy Society of America ◽

10.1017/s0424820100141226 ◽

1995 ◽

Vol 53 ◽

pp. 970-971

Author(s):

L. Vacca-Galloway ◽

Y.Q. Zhang ◽

P. Bose ◽

S.H. Zhang

Keyword(s):

Spinal Cord ◽

Early Stage ◽

Cervical Spinal Cord ◽

Wild Type Mouse ◽

Reflex Response ◽

Mouse Strains ◽

Behavioral Tests ◽

Wobbler Mouse ◽

Stage 4 ◽

Stage 1

The Wobbler mouse (wr) has been studied as a model for inherited human motoneuron diseases (MNDs). Using behavioral tests for forelimb power, walking, climbing, and the “clasp-like reflex” response, the progress of the MND can be categorized into early (Stage 1, age 21 days) and late (Stage 4, age 3 months) stages. Age-and sex-matched normal phenotype littermates (NFR/wr) were used as controls (Stage 0), as well as mice from two related wild-type mouse strains: NFR/N and a C57BI/6N. Using behavioral tests, we also detected pre-symptomatic Wobblers at postnatal ages 7 and 14 days. The mice were anesthetized and perfusion-fixed for immunocytochemical (ICC) of CGRP and ChAT in the spinal cord (C3 to C5).Using computerized morphomety (Vidas, Zeiss), the numbers of IR-CGRP labelled motoneurons were significantly lower in 14 day old Wobbler specimens compared with the controls (Fig. 1). The same trend was observed at 21 days (Stage 1) and 3 months (Stage 4). The IR-CGRP-containing motoneurons in the Wobbler specimens declined progressively with age.

Download Full-text

Automated high-resolution microscopy: the approaches so far

Proceedings, annual meeting, Electron Microscopy Society of America ◽

10.1017/s0424820100125312 ◽

1987 ◽

Vol 45 ◽

pp. 58-61

Author(s):

W. O. Saxton

Keyword(s):

High Resolution ◽

Early Stage ◽

Automatic Recording ◽

Microprocessor Control ◽

Alternative Systems ◽

Electron Image ◽

Complete Automation ◽

Analysis System ◽

High Resolution Microscopy

Recent commercial microscopes with internal microprocessor control of all major functions have already demonstrated some of the benefits anticipated from such systems, such as continuous magnification, rotation-free diffraction and magnification, automatic recording of mutually registered focal series, and fewer control knobs. Complete automation of the focusing, stigmating and alignment of a high resolution microscope, allowing focal series to be recorded at preselected focus values as well, is still imminent rather than accomplished, however; some kind of image pick-up and analysis system, fed with the electron image via a TV camera, is clearly essential for this, but several alternative systems and algorithms are still being explored. This paper reviews the options critically in turn, and stresses the need to consider alignment and focusing at an early stage, and not merely as an optional extension to a basic proposal.

Download Full-text