The Simpson’s Paradox in the Offline Evaluation of Recommendation Systems

Recommendation systems are often evaluated based on user’s interactions that were collected from an existing, already deployed recommendation system. In this situation, users only provide feedback on the exposed items and they may not leave feedback on other items since they have not been exposed to them by the deployed system. As a result, the collected feedback dataset that is used to evaluate a new model is influenced by the deployed system, as a form of closed loop feedback. In this article, we show that the typical offline evaluation of recommender systems suffers from the so-called Simpson’s paradox. Simpson’s paradox is the name given to a phenomenon observed when a significant trend appears in several different sub-populations of observational data but disappears or is even reversed when these sub-populations are combined together. Our in-depth experiments based on stratified sampling reveal that a very small minority of items that are frequently exposed by the deployed system plays a confounding factor in the offline evaluation of recommendation systems. In addition, we propose a novel evaluation methodology that takes into account the confounder, i.e., the deployed system’s characteristics. Using the relative comparison of many recommendation models as in the typical offline evaluation of recommender systems, and based on the Kendall rank correlation coefficient, we show that our proposed evaluation methodology exhibits statistically significant improvements of 14% and 40% on the examined open loop datasets (Yahoo! and Coat), respectively, in reflecting the true ranking of systems with an open loop (randomised) evaluation in comparison to the standard evaluation.

De novo protein structure prediction by incremental inter-residue geometries prediction and model quality assessment using deep learning

Motivation: The successful application of deep learning has promoted progress in protein model quality assessment. How to use model quality assessment to further improve the accuracy of protein structure prediction, especially not reliant on the existing templates, is helpful for unraveling the folding mechanism. Here, we investigate whether model quality assessment can be introduced into structure prediction to form a closed-loop feedback, and iteratively improve the accuracy of de novo protein structure prediction. Results: In this study, we propose a de novo protein structure prediction method called RocketX. In RocketX, a feedback mechanism is constructed through the geometric constraint prediction network GeomNet, the structural simulation module, and the model quality evaluation network EmaNet. In GeomNet, the co-evolutionary features extracted from MSA that search from the sequence databases are sent to an improved residual neural network to predict the inter-residue geometric constraints. The structure model is folded based on the predicted geometric constraints. In EmaNet, the 1D and 2D features are extracted from the folded model and sent to the deep residual neural network to estimate the inter-residue distance deviation and per-residue lDDT of the model, which will be fed back to GeomNet as dynamic features to correct the geometries prediction and progressively improve model accuracy. RocketX is tested on 483 benchmark proteins and 20 FM targets of CASP14. Experimental results show that the closed-loop feedback mechanism significantly contributes to the performance of RocketX, and the prediction accuracy of RocketX outperforms that of the state-of-the-art methods trRosetta (without templates) and RaptorX. In addition, the blind test results on CAMEO show that although no template is used, the prediction accuracy of RocketX on medium and hard targets is comparable to the advanced methods that integrate templates.

Automated Irrigation and Fertigation System Applying Sensing Technology

Internet of Things (IoT) is the most discussed topic nowadays and it finds many applications in various industries. The development of efficient agricultural systems has become essential to address the challenges of feeding the enormous and ever-increasing population around the globe. One way of dealing with this is to expand agricultural production standard and quantity. An integrated IoT system with irrigation and finishing on the market was missing. To increase production efficiency, farmers shall collect massive data from various devices. The system's effectiveness depends on the performance of the sensor under various climate changes. The network and a closed-loop feedback system need to be connected to the system for maximum operational capacity. The module of the Soil Moisture Sensor and the MCU ESP8266 V2 MCU Node, the float switch and the 5V Optocoupler relay module are used in this project. The remote control of the system is a mobile application. If the moisture level in the soil reaches the desired value, it will stop water supply. The project could be expended on a larger scale in the future and the system may also be developed with solar energy rather than batteries.

Energy Efficient, Highly Precise Cascade Dryness Control for Fibrous Tape by Induction-Based Surface Heating of a Rotating Steel Cylinder with Moving Inductors

Drying various materials constitutes an essential component of several industrial processes, e.g., paper production. Typically, rotating cylinders heated internally by water steam are used for drying tape-shaped material in paper-making machines. Such an approach remains very energy-consuming, while the whole process is expensive and in conflict with the global policy of reducing energy consumption in heavy industry. One promising alternative method of drying fibrous tapes is the induction heating of drying cylinders. In this paper, we propose a drying system based on a set of inductors (electromagnetic field sources) that generate energy in the mantle of the cylinder and dry the running tape. By enabling the movement of the inductors, the system provides a high level of flexibility in terms of reacting to the varying humidity of the tape. Additionally, imaging the temperature field on the cylinder surface provides a supplementary source of information, enabling the temperature profile to be controlled. Two types of humidity control systems, a one-loop feedback control and a cascade control, were designed and analyzed. Simulation analysis and experimental verification performed using a semi-industrial setup proves that using the proposed cascade control ensures more than 30% faster response of the whole dryness control system.

Number and Distribution of Fiducial Fibres in a Spectroscopic Survey Telescope

To date, the Large Sky Area Multi-Object Fibre Spectroscopic Telescope (LAMOST) has been in operation for 12 years. To improve the telescope’s astronomical observation accuracy, the original open-loop fibre positioning system of LAMOST is in urgent need of upgrading. The upgrade plan is to locate several fibre view cameras (FVCs) around primary mirror B to build a closed-loop feedback control system. The FVCs are ~20 m from the focal surface. To reduce a series of errors when the cameras detect the positions of the optical fibres, we designed fiducial fibres on the focal surface to be fiducial points for the cameras. Increasing the number of fiducial fibres can improve the detection accuracy of the FVC system, but it will also certainly reduce the number of fibre positioners that can be used for observation. Therefore, the focus of this paper is how to achieve the quantity and distribution that meet the requirements of system detection. In this paper, we introduce the necessity of using fiducial fibres, propose a method for selecting their number, and present several methods for assessing the uniformity of their distribution. Finally, we use particle swarm optimization to find the best distribution of fiducial fibres.

Experimental Comparison of PID Based RPM Control for Long Horizontal vs. Vertical Drillstring

Drilling vibrations has been identified as a key performance limiter that prevents successful energy transfer from the surface to the bit. The resultant irregular bit rotation speed causes drilling challenges like poor hole quality and reduction in rate of penetration to name a few. Amongst other vibration suppression techniques, active control of vibrations is widely used and recommended. Active control system utilizes a closed loop feedback control to continuously adapt to downhole vibrations detected at the surface. The system requires regular tuning and upgrades which is done experimentally before field deployment. The paper presents a PID based experimental tuning of horizontal and vertical strings for irregular RPM. In doing so, the paper highlights the need of having horizontal experimental setups for state-of-the-art PID tuning for downhole RPM. The settling time of irregular RPM for horizontal setup is found to be lower than the vertical configuration due to multiple contact points and increased friction coefficient.

Video Capsule Endoscopy and Ingestible Electronics: Emerging Trends in Sensors, Circuits, Materials, Telemetry, Optics, and Rapid Reading Software

Real-time monitoring of the gastrointestinal tract in a safe and comfortable manner is valuable for the diagnosis and therapy of many diseases. Within this realm, our review captures the trends in ingestible capsule systems with a focus on hardware and software technologies used for capsule endoscopy and remote patient monitoring. We introduce the structure and functions of the gastrointestinal tract, and the FDA guidelines for ingestible wireless telemetric medical devices. We survey the advanced features incorporated in ingestible capsule systems, such as microrobotics, closed-loop feedback, physiological sensing, nerve stimulation, sampling and delivery, panoramic imaging with adaptive frame rates, and rapid reading software. Examples of experimental and commercialized capsule systems are presented with descriptions of their sensors, devices, and circuits for gastrointestinal health monitoring. We also show the recent research in biocompatible materials and batteries, edible electronics, and alternative energy sources for ingestible capsule systems. The results from clinical studies are discussed for the assessment of key performance indicators related to the safety and effectiveness of ingestible capsule procedures. Lastly, the present challenges and outlook are summarized with respect to the risks to health, clinical testing and approval process, and technology adoption by patients and clinicians.

Implementing an Anonymous Closed-Loop Feedback Process and Associations With ACGME Annual Survey Scores in a Family Medicine Residency

Background and Objectives: The annual Accreditation Council for Graduate Medical Education (ACGME) survey evaluates numerous variables, including resident satisfaction with the training program. We postulated that an anonymous system allowing residents to regularly express and discuss concerns would result in higher ACGME survey scores in areas pertaining to program satisfaction. Methods: One family medicine residency program implemented a process of quarterly anonymous closed-loop resident feedback and discussion in academic year 2012-2013. Data were tracked longitudinally from the 2011-2019 annual ACGME resident surveys, using academic year 2011-2012 as a baseline control. Results: For the survey item “Satisfied that evaluations of program are confidential,” years 2013-2014, 2014-2015, and 2018-2019 showed a significantly higher change from baseline. For “Satisfied that program uses evaluations to improve,” year 2018-2019 had a significantly higher percentage change from baseline. For “Satisfied with process to deal with problems and concerns,” year 2018-2019 showed significantly higher change. For “Residents can raise concerns without fear,” years 2013-2014 and 2018-2019 saw significantly higher changes. Conclusions: These results suggest that this feedback process is perceived by residents as both confidential and promoting a culture of safety in providing feedback. Smaller changes were seen in residents’ belief that the program uses evaluations to improve, and in satisfaction with the process to deal with problems and concerns.