On the Reproducibility and Replicability of Deep Learning in Software Engineering

Context: Deep learning (DL) techniques have gained significant popularity among software engineering (SE) researchers in recent years. This is because they can often solve many SE challenges without enormous manual feature engineering effort and complex domain knowledge. Objective: Although many DL studies have reported substantial advantages over other state-of-the-art models on effectiveness, they often ignore two factors: (1) reproducibility —whether the reported experimental results can be obtained by other researchers using authors’ artifacts (i.e., source code and datasets) with the same experimental setup; and (2) replicability —whether the reported experimental result can be obtained by other researchers using their re-implemented artifacts with a different experimental setup. We observed that DL studies commonly overlook these two factors and declare them as minor threats or leave them for future work. This is mainly due to high model complexity with many manually set parameters and the time-consuming optimization process, unlike classical supervised machine learning (ML) methods (e.g., random forest). This study aims to investigate the urgency and importance of reproducibility and replicability for DL studies on SE tasks. Method: In this study, we conducted a literature review on 147 DL studies recently published in 20 SE venues and 20 AI (Artificial Intelligence) venues to investigate these issues. We also re-ran four representative DL models in SE to investigate important factors that may strongly affect the reproducibility and replicability of a study. Results: Our statistics show the urgency of investigating these two factors in SE, where only 10.2% of the studies investigate any research question to show that their models can address at least one issue of replicability and/or reproducibility. More than 62.6% of the studies do not even share high-quality source code or complete data to support the reproducibility of their complex models. Meanwhile, our experimental results show the importance of reproducibility and replicability, where the reported performance of a DL model could not be reproduced for an unstable optimization process. Replicability could be substantially compromised if the model training is not convergent, or if performance is sensitive to the size of vocabulary and testing data. Conclusion: It is urgent for the SE community to provide a long-lasting link to a high-quality reproduction package, enhance DL-based solution stability and convergence, and avoid performance sensitivity on different sampled data.

Evaluating the use of absolute binding free energy in the fragment optimization process.

Key to the fragment optimization process is the need to accurately capture the changes in affinity that are associated with a given set of chemical modifications. Due to the weakly binding nature of fragments, this has proven to be a challenging task, despite recent advancements in leveraging experimental and computational methods. In this work, we evaluate the use of Absolute Binding Free Energy (ABFE) calculations in guiding fragment optimization decisions, retrospectively calculating binding free energies for 59 ligands across 4 fragment elaboration campaigns. We first demonstrate that ABFEs can be used to accurately rank fragment-sized binders with an overall Spearman’s r of 0.89 and a Kendall τ of 0.67, although often deviating from experiment in absolute free energy values with an RMSE of 2.75 kcal/mol. We then also show that in several cases, retrospective fragment optimization decisions can be supported by the ABFE calculations. Cases that were not supported were often limited by large uncertainties in the free energy estimates, however generally the right direction in ΔΔG is still observed. Comparing against cheaper endpoint methods, namely Nwat-MM/GBSA, we find that ABFEs offer better outcomes in ranking binders, improving correlation metrics, although a similar confidence in retrospective synthetic decisions is achieved. Our results indicate that ABFE calculations are currently at the level of accuracy that can be usefully employed to gauge which fragment elaborations are likely to offer the best gains in affinity.

Improved-Efficacy EM-Based Antenna Miniaturization by Multi-Fidelity Simulations and Objective Function Adaptation

The growing demand for the integration of surface mount design (SMD) antennas into miniaturized electronic devices has imposed increasing limitations on the structure dimensions. Examples include embedded antennas in applications such as on-board devices, picosatellites, 5G communications, or implantable and wearable devices. The demands for size reduction while ensuring a satisfactory level of electrical and field performance can be managed through constrained numerical optimization. The reliability of optimization-based size reduction requires utilization of full-wave electromagnetic (EM) analysis, which entails significant computational costs. This can be alleviated by incorporating surrogate modeling techniques, adjoint sensitivities, or the employment of sparse sensitivity updates. An alternative is the incorporation of multi-fidelity simulation models, normally limited to two levels, low and high resolution. This paper proposes a novel algorithm for accelerated antenna miniaturization, featuring a continuous adjustment of the simulation model fidelity in the course of the optimization process. The model resolution is determined by factors related to violation of the design constraints as well as the convergence status of the algorithm. The algorithm utilizes the lowest-fidelity model for the early stages of the optimization process; it is gradually refined towards the highest-fidelity model upon approaching convergence, and the constraint violations improve towards the preset tolerance threshold. At the same time, a penalty function approach with adaptively adjusted coefficients is applied to enable the precise control of constraints, and to increase the achievable miniaturization rates. The presented procedure has been validated using five microstrip antennas, including three broadband, and two circularly polarized structures. The obtained results corroborate the relevance of the implemented mechanisms from the point of view of improving the average computational efficiency of the optimization process by 43% as compared to the single-fidelity adaptive penalty function approach. Furthermore, the presented methodology demonstrates a performance that is equivalent or even superior to its single-fidelity counterpart in terms of an average constraint violation of 0.01 dB (compared to 0.03 dB for the reference), and an average size reduction of 25% as compared to 25.6%.

Eucalyptol, an All-Purpose Product

Eucalyptus plants have attracted the attention of researchers and environmentalists worldwide because they are a rapidly growing source of wood and a source of oil used for multiple purposes. The main and the most important oil component is 1,8-cineole (eucalyptol: 60%–85%). This review summarizes the literature reported to date involving the use of 1,8-cineole for the treatment of disorders. Additionally, we describe our efforts in the use of eucalyptol as a solvent for the synthesis of O,S,N-heterocycles. Solvents used in chemistry are a fundamental element of the environmental performance of processes in corporate and academic laboratories. Their influence on costs, safety and health cannot be neglected. Green solvents such as bio-based systems hold considerable additional promise to reduce the environmental impact of organic chemistry. The first section outlines the process leading to our discovery of an unprecedented solvent and its validation in the first coupling reactions. This section continues with the description of its properties and characteristics and its reuse as reported in the various studies conducted. The second section highlights the use of eucalyptol in a series of coupling reactions (i.e., Suzuki–Miyaura, Sonogashira–Hagihara, Buchwald–Hartwig, Migita–Kosugi–Stille, Hiyama and cyanation) that form O,S,N-heterocycles. We describe the optimization process applied to reach the ideal conditions. We also show that eucalyptol can be a good alternative to build heterocycles that contain oxygen, sulfur and nitrogen. These studies allowed us to demonstrate the viability and potential that bio solvents can have in synthesis laboratories.

Preparation of Nanoemulsion

Nanoemulsion is a versatile emulsion-based delivery system that can be structured or prepared with different compositions, methods, or processing variables. Although single nanoemulsion has been a promising delivery carrier in numerous industries, the innovation of double nanoemulsion is introduced to provide different benefits. However, the nanoemulsion must be optimized in terms of the composition and emulsification methods to maintain the integrity of the delivery system. Therefore, the optimization of nanoemulsion can range from deciding the type and concentration of compositions (aqueous, lipid, surfactants) to the processing conditions during emulsification. The current options of emulsification methods, processing parameters, and optimization process of nanoemulsion are highlighted and elaborated in this chapter. This allows readers to understand the fundamental principles of nanoemulsion preparation and encourage future studies and applications in the related field.

Optimization and Sensitivity Analysis of the Cutting Conditions in Rough, Semi-Finish and Finish Honing

Honing processes are currently employed to obtain a cross-hatched pattern on the internal surfaces of cylinders that favors oil flow in combustion engines or hydraulic cylinders. The main aim of the present paper is to optimize the machining conditions in honing processes with respect to surface roughness, material removal rate and tool wear by means of the desirability function. Five process variables are considered: grain size, density, pressure, linear speed and tangential speed. Later, a sensitivity analysis is performed to determine the effect of the variation of the importance given to each response on the results of the optimization process. In the rough and semi-finish honing steps, variations of less than 5% of the importance value do not cause substantial changes in the optimization process. On the contrary, in the finish honing step, small changes in the importance values lead to modifications in the optimization process, mainly regarding pressure. Thus, the finish honing phase is more sensitive to changes in the optimization process than the rough and the semi-finish honing phases. The present paper will help users of honing machines to select proper values for the process variables.

Gramian-constrained optimization process for the stiffness model identification of industrial manipulators

This work deals with the elastostatic identification of industrial manipulators. By reviewing the basics of the physical elastic properties of both links and joints in the framework of the lumped stiffness modeling techniques, the Gramian nature of the stiffness matrices has been found out adequate to do so. Then, a novel optimization method has been developed, which incorporates the Gramian matrix formulation along a non-linear optimization process, acting as an intrinsic constraint for the conservativeness of the elastostatic modeling. Numerical and experimental analyses evince the effectiveness of the proposed method, as the elastostatic models obtained by means of the proposed technique predict more than 93.7% of the compliance deviations of a real industrial robot. The proposed method is simple enough to be jointly applicable to the most recent elastostatic model reduction techniques.

Python-Based Open-Source Electro-Mechanical Co-Optimization System for MEMS Inertial Sensors

The surge in fabrication techniques for micro- and nanodevices gave room to rapid growth in these technologies and a never-ending range of possible applications emerged. These new products significantly improve human life, however, the evolution in the design, simulation and optimization process of said products did not observe a similarly rapid growth. It became thus clear that the performance of micro- and nanodevices would benefit from significant improvements in this area. This work presents a novel methodology for electro-mechanical co-optimization of micro-electromechanical systems (MEMS) inertial sensors. The developed software tool comprises geometry design, finite element method (FEM) analysis, damping calculation, electronic domain simulation, and a genetic algorithm (GA) optimization process. It allows for a facilitated system-level MEMS design flow, in which electrical and mechanical domains communicate with each other to achieve an optimized system performance. To demonstrate the efficacy of the methodology, an open-loop capacitive MEMS accelerometer and an open-loop Coriolis vibratory MEMS gyroscope were simulated and optimized—these devices saw a sensitivity improvement of 193.77% and 420.9%, respectively, in comparison to their original state.

Optimizing Main Process Parameters When Conducting Powder-Mixed Electrical Discharge Machining of Hardened 90CrSi

In the current study, an optimization process of powder-mixed electrical discharge machining (PMEDM) process when machining cylindrically shaped parts made of hardened 90CrSi steel is reported. In this study, SiC powder was mixed into the Diel MS 7000 dielectric solution. Additionally, graphite was chosen as the electrode material. The multi-objective functions were minimizing the surface roughness (SR) and electrode wear rate (EWR) and maximizing the material removal rate (MRR). The used input parameters of the optimization process included the powder concentration, the pulse-on time, the pulse-off time, the pulse current, and the servo voltage. A combination between the Taguchi method and the grey relation analysis (GRA) method with the support of Minitab R19 software was used to design the experiment and analyze the results. It was found that the optimal set of process parameters that can satisfy the above responses are Cp of 0.5 g/L, Ton of 8 µs, Toff of 8 µs, IP of 5 A, and SV of 4 V.