scholarly journals A Survey of Uncertainty Quantification in Machine Learning for Space Weather Prediction

Geosciences ◽  
2022 ◽  
Vol 12 (1) ◽  
pp. 27
Author(s):  
Talha Siddique ◽  
Md Mahmud ◽  
Amy Keesee ◽  
Chigomezyo Ngwira ◽  
Hyunju Connor

With the availability of data and computational technologies in the modern world, machine learning (ML) has emerged as a preferred methodology for data analysis and prediction. While ML holds great promise, the results from such models are not fully unreliable due to the challenges introduced by uncertainty. An ML model generates an optimal solution based on its training data. However, if the uncertainty in the data and the model parameters are not considered, such optimal solutions have a high risk of failure in actual world deployment. This paper surveys the different approaches used in ML to quantify uncertainty. The paper also exhibits the implications of quantifying uncertainty when using ML by performing two case studies with space physics in focus. The first case study consists of the classification of auroral images in predefined labels. In the second case study, the horizontal component of the perturbed magnetic field measured at the Earth’s surface was predicted for the study of Geomagnetically Induced Currents (GICs) by training the model using time series data. In both cases, a Bayesian Neural Network (BNN) was trained to generate predictions, along with epistemic and aleatoric uncertainties. Finally, the pros and cons of both Gaussian Process Regression (GPR) models and Bayesian Deep Learning (DL) are weighed. The paper also provides recommendations for the models that need exploration, focusing on space weather prediction.

2020 ◽  
Author(s):  
Abdur Rahman M. A. Basher ◽  
Steven J. Hallam

AbstractMachine learning methods show great promise in predicting metabolic pathways at different levels of biological organization. However, several complications remain that can degrade prediction performance including inadequately labeled training data, missing feature information, and inherent imbalances in the distribution of enzymes and pathways within a dataset. This class imbalance problem is commonly encountered by the machine learning community when the proportion of instances over class labels within a dataset are uneven, resulting in poor predictive performance for underrepresented classes. Here, we present leADS, multi-label learning based on active dataset subsampling, that leverages the idea of subsampling points from a pool of data to reduce the negative impact of training loss due to class imbalance. Specifically, leADS performs an iterative process to: (i)-construct an acquisition model in an ensemble framework; (ii) select informative points using an appropriate acquisition function; and (iii) train on selected samples. Multiple base learners are implemented in parallel where each is assigned a portion of labeled training data to learn pathways. We benchmark leADS using a corpora of 10 experimental datasets manifesting diverse multi-label properties used in previous pathway prediction studies, including manually curated organismal genomes, synthetic microbial communities and low complexity microbial communities. Resulting performance metrics equaled or exceeded previously reported machine learning methods for both organismal and multi-organismal genomes while establishing an extensible framework for navigating class imbalances across diverse real world datasets.Availability and implementationThe software package, and installation instructions are published on github.com/[email protected]


2020 ◽  
Vol 2020 ◽  
pp. 1-11
Author(s):  
Yanfei Xiang ◽  
Jianbing Ma ◽  
Xi Wu

Unpredicted precipitations, even mild, may cause severe economic losses to many businesses. Precipitation nowcasting is hence significant for people to make correct decisions timely. For traditional methods, such as numerical weather prediction (NWP), the accuracy is limited because the smaller scale of strong convective weather must be smaller than the minimum scale that the model can capture. And it often requires a supercomputer. Furthermore, the optical flow method has been proved to be available for precipitation nowcasting. However, it is difficult to determine the model parameters because the two steps of tracking and extrapolation are separate. In contrast, current machine learning applications are based on well-selected full datasets, ignoring the fact that real datasets quite often contain missing data requiring extra consideration. In this paper, we used a real Hubei dataset in which a few radar echo data are missing and proposed a proper mechanism to deal with the situation. Furthermore, we proposed a novel mechanism for radar reflectivity data with single altitudes or cumulative altitudes using machine learning techniques. From the experimental results, we conclude that our method can predict future precipitation with a high accuracy when a few data are missing, and it outperforms the traditional optical flow method. In addition, our model can be used for various types of radar data with a type-specific feature extraction, which makes the method more flexible and suitable for most situations.


2020 ◽  
Vol 120 (6) ◽  
pp. 1149-1174 ◽  
Author(s):  
K.H. Leung ◽  
Daniel Y. Mo ◽  
G.T.S. Ho ◽  
C.H. Wu ◽  
G.Q. Huang

PurposeAccurate prediction of order demand across omni-channel supply chains improves the management's decision-making ability at strategic, tactical and operational levels. The paper aims to develop a predictive methodology for forecasting near-real-time e-commerce order arrivals in distribution centres, allowing third-party logistics service providers to manage the hour-to-hour fast-changing arrival rates of e-commerce orders better.Design/methodology/approachThe paper proposes a novel machine learning predictive methodology through the integration of the time series data characteristics into the development of an adaptive neuro-fuzzy inference system. A four-stage implementation framework is developed for enabling practitioners to apply the proposed model.FindingsA structured model evaluation framework is constructed for cross-validation of model performance. With the aid of an illustrative case study, forecasting evaluation reveals a high level of accuracy of the proposed machine learning approach in forecasting the arrivals of real e-commerce orders in three different retailers at three-hour intervals.Research limitations/implicationsResults from the case study suggest that real-time prediction of individual retailer's e-order arrival is crucial in order to maximize the value of e-order arrival prediction for daily operational decision-making.Originality/valueEarlier researchers examined supply chain demand, forecasting problem in a broader scope, particularly in dealing with the bullwhip effect. Prediction of real-time, hourly based order arrivals has been lacking. The paper fills this research gap by presenting a novel data-driven predictive methodology.


2019 ◽  
Vol 141 (12) ◽  
Author(s):  
Gary M. Stump ◽  
Simon W. Miller ◽  
Michael A. Yukish ◽  
Timothy W. Simpson ◽  
Conrad Tucker

Abstract A novel method has been developed to optimize both the form and behavior of complex systems. The method uses spatial grammars embodied in character-recurrent neural networks (char-RNNs) to define the system including actuator numbers and degrees of freedom, reinforcement learning to optimize actuator behavior, and physics-based simulation systems to determine performance and provide (re)training data for the char-RNN. Compared to parametric design optimization with fixed numbers of inputs, using grammars and char-RNNs allows for a more complex, combinatorial infinite design space. In the proposed method, the char-RNN is first trained to learn a spatial grammar that defines the assembly layout, component geometries, material properties, and arbitrary numbers and degrees of freedom of actuators. Next, generated designs are evaluated using a physics-based environment, with an inner optimization loop using reinforcement learning to determine the best control policy for the actuators. The resulting design is thus optimized for both form and behavior, generated by a char-RNN embodying a high-performing grammar. Two evaluative case studies are presented using the design of the modular sailing craft. The first case study optimizes the design without actuated surfaces, allowing the char-RNN to understand the semantics of high-performing designs. The second case study extends the first by incorporating controllable actuators requiring an inner loop behavioral optimization. The implications of the results are discussed along with the ongoing and future work.


2017 ◽  
Author(s):  
Reuben Binns ◽  
Michael Veale ◽  
Max Van Kleek ◽  
Nigel Shadbolt

The internet has become a central medium through which 'networked publics' express their opinions and engage in debate. Offensive comments and personal attacks can inhibit participation in these spaces. Automated content moderation aims to overcome this problem using machine learning classifiers trained on large corpora of texts manually annotated for offence. While such systems could help encourage more civil debate, they must navigate inherently normatively contestable boundaries, and are subject to the idiosyncratic norms of the human raters who provide the training data. An important objective for platforms implementing such measures might be to ensure that they are not unduly biased towards or against particular norms of offence. This paper provides some exploratory methods by which the normative biases of algorithmic content moderation systems can be measured, by way of a case study using an existing dataset of comments labelled for offence. We train classifiers on comments labelled by different demographic subsets (men and women) to understand how differences in conceptions of offence between these groups might affect the performance of the resulting models on various test sets. We conclude by discussing some of the ethical choices facing the implementers of algorithmic moderation systems, given various desired levels of diversity of viewpoints amongst discussion participants.


2021 ◽  
Vol 15 (02) ◽  
pp. 215-240
Author(s):  
Tomoki Chiba ◽  
Yuichi Sei ◽  
Yasuyuki Tahara ◽  
Akihiko Ohsuga

In the modern world, several areas of our lives can be improved, in the form of diverse additional dimensions, in terms of quality, by machine learning. When building machine learning models, open data are often used. Although this trend is on the rise, the monetary losses since the attacks on machine learning models are also rising. Preparation is, thus, believed to be indispensable in terms of embarking upon machine learning. In this field of endeavor, machine learning models may be compromised in various ways, including poisoning attacks. Assaults of this nature involve the incorporation of injurious data into the training data rendering the models to be substantively less accurate. The circumstances of every individual case will determine the degree to which the impairment due to such intrusions can lead to extensive disruption. A modus operandi is proffered in this research as a safeguard for machine learning models in the face of the poisoning menace, envisaging a milieu in which machine learning models make use of data that emanate from numerous sources. The information in question will be presented as training data, and the diversity of sources will constitute a barrier to poisoning attacks in such circumstances. Every source is evaluated separately, with the weight of each data component assessed in terms of its ability to affect the precision of the machine learning model. An appraisal is also conducted on the basis of the theoretical effect of the use of corrupt data as from each source. The extent to which the subgroup of data in question can undermine overall accuracy depends on the estimated data removal rate associated with each of the sources described above. The exclusion of such isolated data based on this figure ensures that the standard data will not be tainted. To evaluate the efficacy of our suggested preventive measure, we evaluated it in comparison with the well-known standard techniques to assess the degree to which the model was providing accurate conclusions in the wake of the change. It was demonstrated during this test that when the innovative mode of appraisal was applied, in circumstances in which 17% of the training data are corrupt, the degree of precision offered by the model is 89%, in contrast to the figure of 83% acquired through the traditional technique. The corrective technique suggested by us thus boosted the resilience of the model against harmful intrusion.


2020 ◽  
Vol 12 (6) ◽  
pp. 2544
Author(s):  
Alice Consilvio ◽  
José Solís-Hernández ◽  
Noemi Jiménez-Redondo ◽  
Paolo Sanetti ◽  
Federico Papa ◽  
...  

The objective of this study is to show the applicability of machine learning and simulative approaches to the development of decision support systems for railway asset management. These techniques are applied within the generic framework developed and tested within the In2Smart project. The framework is composed by different building blocks, in order to show the complete process from data collection and knowledge extraction to the real-world decisions. The application of the framework to two different real-world case studies is described: the first case study deals with strategic earthworks asset management, while the second case study considers the tactical and operational planning of track circuits’ maintenance. Although different methodologies are applied and different planning levels are considered, both the case studies follow the same general framework, demonstrating the generality of the approach. The potentiality of combining machine learning techniques with simulative approaches to replicate real processes is shown, evaluating the key performance indicators employed within the considered asset management process. Finally, the results of the validation are reported as well as the developed human–machine interfaces for output visualization.


2020 ◽  
Vol 44 (7-8) ◽  
pp. 499-514
Author(s):  
Yi Zheng ◽  
Hyunjung Cheon ◽  
Charles M. Katz

This study explores advanced techniques in machine learning to develop a short tree-based adaptive classification test based on an existing lengthy instrument. A case study was carried out for an assessment of risk for juvenile delinquency. Two unique facts of this case are (a) the items in the original instrument measure a large number of distinctive constructs; (b) the target outcomes are of low prevalence, which renders imbalanced training data. Due to the high dimensionality of the items, traditional item response theory (IRT)-based adaptive testing approaches may not work well, whereas decision trees, which are developed in the machine learning discipline, present as a promising alternative solution for adaptive tests. A cross-validation study was carried out to compare eight tree-based adaptive test constructions with five benchmark methods using data from a sample of 3,975 subjects. The findings reveal that the best-performing tree-based adaptive tests yielded better classification accuracy than the benchmark method IRT scoring with optimal cutpoints, and yielded comparable or better classification accuracy than the best benchmark method, random forest with balanced sampling. The competitive classification accuracy of the tree-based adaptive tests also come with an over 30-fold reduction in the length of the instrument, only administering between 3 to 6 items to any individual. This study suggests that tree-based adaptive tests have an enormous potential when used to shorten instruments that measure a large variety of constructs.


2021 ◽  
Vol 21 (15) ◽  
pp. 11857-11887
Author(s):  
César Sauvage ◽  
Cindy Lebeaupin Brossier ◽  
Marie-Noëlle Bouin

Abstract. The western Mediterranean Sea area is frequently affected in autumn by heavy precipitation events (HPEs). These severe meteorological episodes, characterized by strong offshore low-level winds and heavy rain in a short period of time, can lead to severe flooding and wave-submersion events. This study aims to progress towards an integrated short-range forecast system via coupled modeling for a better representation of the processes at the air–sea interface. In order to identify and quantify the coupling impacts, coupled ocean–atmosphere–wave simulations were performed for a HPE that occurred between 12 and 14 October 2016 in the south of France. The experiment using the coupled AROME-NEMO-WaveWatchIII system was notably compared to atmosphere-only, coupled atmosphere–wave and ocean–atmosphere simulations. The results showed that the HPE fine-scale forecast is sensitive to both couplings: the interactive coupling with the ocean leads to significant changes in the heat and moisture supply of the HPE that intensify the convective systems, while coupling with a wave model mainly leads to changes in the low-level dynamics, affecting the location of the convergence that triggers convection over the sea. Result analysis of this first case study with the AROME-NEMO-WaveWatchIII system does not clearly show major changes in the forecasts with coupling and highlights some attention points to follow (ocean initialization notably). Nonetheless, it illustrates the higher realism and potential benefits of kilometer-scale coupled numerical weather prediction systems, in particular in the case of severe weather events over the sea and/or in coastal areas, and shows their affordability to confidently progress towards operational coupled forecasts.


Sign in / Sign up

Export Citation Format

Share Document