Reinforcement Learning in Healthcare: A Survey

As a subfield of machine learning, reinforcement learning (RL) aims at optimizing decision making by using interaction samples of an agent with its environment and the potentially delayed feedbacks. In contrast to traditional supervised learning that typically relies on one-shot, exhaustive, and supervised reward signals, RL tackles sequential decision-making problems with sampled, evaluative, and delayed feedbacks simultaneously. Such a distinctive feature makes RL techniques a suitable candidate for developing powerful solutions in various healthcare domains, where diagnosing decisions or treatment regimes are usually characterized by a prolonged period with delayed feedbacks. By first briefly examining theoretical foundations and key methods in RL research, this survey provides an extensive overview of RL applications in a variety of healthcare domains, ranging from dynamic treatment regimes in chronic diseases and critical care, automated medical diagnosis, and many other control or scheduling problems that have infiltrated every aspect of the healthcare system. In addition, we discuss the challenges and open issues in the current research and highlight some potential solutions and directions for future research.

Use of Pyriproxyfen to Induce Oogenesis in Diapausing Megacopta cribraria (Heteroptera: Plataspidae), and Evaluation of Pyriproxyfen-Induced Eggs for Rearing the Parasitoid Paratelenomus saccharalis (Hymenoptera: Scelionidae)

The mass rearing of hymenopteran egg parasitoids requires an abundant supply of host eggs. The onset of reproductive diapause and subsequent decline in egg production poses a challenge for parasitoid rearing when using host colonies augmented by field-collected insects. We investigated the application of pyriproxyfen, a juvenile hormone analog, to induce oviposition in diapausing adult kudzu bugs, Megacopta cribraria (Fabricius) (Heteroptera: Plataspidae), and the use of eggs produced by pyriproxyfen-treated kudzu bugs to rear the egg parasitoid, Paratelenomus saccharalis (Dodd) (Hymenoptera: Scelionidae). The effects of pyriproxyfen and photoperiod treatments on host mortality, egg production, and rates of parasitoid eclosion from the eggs were used to calculate the parasitoid yield for the different treatment regimes. A combination of pyriproxyfen and a long-day photoperiod increased the parasitoid yield by 87% compared to acetone and a long-day photoperiod. The general applicability of JH-analog mediated egg production for parasitoid rearing is discussed.

Te Kēmu Hauora - Designing a mobile game to facilitate education and improve healthcare engagement

<p>This thesis addresses the research question “How could a mobile game be designed to facilitate education and improve healthcare engagement around skin sores in children?”. Health behaviour issues such as lack of education and low adherence to treatment regimes impact the success rates of treatments in children for common ailments such as skin sores. Skin sores are a particular health issue in New Zealand where the rates of admission to hospital for serious skin infections in 2006 were double that of USA and Australia (Craig et al., 2007, p. 278-282). Hospitalisation can be prevented by ensuring treatment regimes for skin sores are correctly completed after early diagnosis (Gray et al., 2013, p. 2). Literature states that game design is a viable solution to healthcare issues, as it can be used alongside persuasive strategies to engage and educate children around their treatments. However, there is a gap in the literature and existing precedents for health games addressing treatment of common ailments. This thesis addresses the research question through developing design criteria for a health game by identifying suitable theories to encourage positive health behaviors and educate children. These include: simulation, personalisation, reward and flow theory, as well as the information, motivation, strategy model to increase engagement with treatment. These criteria are used alongside user personas and journey maps methods to create the design output of a mobile health game to educate and engage New Zealand children around the treatment of skin sores. The game was tested for education, engagement and usability during the design process. Methods for user testing included observation, an adaption of the System Usability Scale and semi-structured interview questions. Thematic analysis of the testing results showed that most participants were engaged with the game and gained education around treatment steps. Insights on user testing with children for education, engagement, and usability are reported. The final output was refined and accessed against the design criteria. Findings from this thesis discuss how game design techniques including simulation, flow theory and reward can be used to educate and engage children with treatments of common ailments.</p>

Te Kēmu Hauora - Designing a mobile game to facilitate education and improve healthcare engagement

<p>This thesis addresses the research question “How could a mobile game be designed to facilitate education and improve healthcare engagement around skin sores in children?”. Health behaviour issues such as lack of education and low adherence to treatment regimes impact the success rates of treatments in children for common ailments such as skin sores. Skin sores are a particular health issue in New Zealand where the rates of admission to hospital for serious skin infections in 2006 were double that of USA and Australia (Craig et al., 2007, p. 278-282). Hospitalisation can be prevented by ensuring treatment regimes for skin sores are correctly completed after early diagnosis (Gray et al., 2013, p. 2). Literature states that game design is a viable solution to healthcare issues, as it can be used alongside persuasive strategies to engage and educate children around their treatments. However, there is a gap in the literature and existing precedents for health games addressing treatment of common ailments. This thesis addresses the research question through developing design criteria for a health game by identifying suitable theories to encourage positive health behaviors and educate children. These include: simulation, personalisation, reward and flow theory, as well as the information, motivation, strategy model to increase engagement with treatment. These criteria are used alongside user personas and journey maps methods to create the design output of a mobile health game to educate and engage New Zealand children around the treatment of skin sores. The game was tested for education, engagement and usability during the design process. Methods for user testing included observation, an adaption of the System Usability Scale and semi-structured interview questions. Thematic analysis of the testing results showed that most participants were engaged with the game and gained education around treatment steps. Insights on user testing with children for education, engagement, and usability are reported. The final output was refined and accessed against the design criteria. Findings from this thesis discuss how game design techniques including simulation, flow theory and reward can be used to educate and engage children with treatments of common ailments.</p>

Leiomyosarcoma of urinary bladder with unusual recurrence in intestinal mucosa and peritoneum: a case report

Background Leiomyosarcomas of urinary bladder constitute rare malignant sarcomas with very few cases reported in literature. Case presentation Here, we present a case of bladder leiomyosarcoma in a well-preserved female. She failed to respond to standard chemotherapy and had a rapidly downhill course with unusual metastases in anastomotic site and peritoneum soon after surgery. Despite multimodality management including resection of primary and metastatic site, systemic therapy and pelvic radiotherapy, our patient had dismal prognosis with an overall survival of 1.7 years. Conclusion Leiomyosarcomas of bladder are aggressive tumors and have a very poor prognosis; thus, future research should focus on optimizing more effective treatment regimes.

Management of post-acute COVID-19 patients in geriatric rehabilitation: EuGMS guidance

Purpose To describe a guidance on the management of post-acute COVID 19 patients in geriatric rehabilitation. Methods The guidance is based on guidelines for post-acute COVID-19 geriatric rehabilitation developed in the Netherlands, updated with recent insights from literature, related guidance from other countries and disciplines, and combined with experiences from experts in countries participating in the Geriatric Rehabilitation Special Interest Group of the European Geriatric Medicine Society. Results This guidance for post-acute COVID-19 rehabilitation is divided into a section addressing general recommendations for geriatric rehabilitation and a section addressing specific processes and procedures. The Sect. “General recommendations for geriatric rehabilitation” addresses: (1) general requirements for post-acute COVID-19 rehabilitation and (2) critical aspects for quality assurance during COVID-19 pandemic. The Sect. “Specific processes and procedures”, addresses the following topics: (1) patient selection; (2) admission; (3) treatment; (4) discharge; and (5) follow-up and monitoring. Conclusion Providing tailored geriatric rehabilitation treatment to post-acute COVID-19 patients is a challenge for which the guidance is designed to provide support. There is a strong need for additional evidence on COVID-19 geriatric rehabilitation including developing an understanding of risk profiles of older patients living with frailty to develop individualised treatment regimes. The present guidance will be regularly updated based on additional evidence from practice and research.