Bayesian Machine Learning

2005 ◽  
pp. 229-235
Author(s):  
Eitel J.M. Lauria
Keyword(s):  
Machine Learning ◽  
Observational Data ◽  
Bayesian Methods ◽  
Posterior Probability ◽  
Prior Probability ◽  
Probabilistic Approach ◽  
Bayes Theorem ◽  
Bayesian Framework ◽  
Probability Models ◽  
Bayesian Machine Learning

Bayesian methods provide a probabilistic approach to machine learning. The Bayesian framework allows us to make inferences from data using probability models for values we observe and about which we want to draw some hypotheses. Bayes theorem provides the means of calculating the probability of a hypothesis (posterior probability) based on its prior probability, the probability of the observations and the likelihood that the observational data fit the hypothesis.

scholarly journals Estimating statistical power, posterior probability and publication bias of psychological research using the observed replication rate

2018 ◽  
Vol 5 (9) ◽  
pp. 181190 ◽  
Author(s):  
Michael Ingre ◽  
Gustav Nilsonne
Keyword(s):  
Publication Bias ◽  
Statistical Power ◽  
Posterior Probability ◽  
Prior Probability ◽  
Bayes Theorem ◽  
Psychological Research ◽  
Open Science ◽  
Negative Evidence ◽  
Replication Rate ◽  
Science Collaboration

In this paper, we show how Bayes' theorem can be used to better understand the implications of the 36% reproducibility rate of published psychological findings reported by the Open Science Collaboration. We demonstrate a method to assess publication bias and show that the observed reproducibility rate was not consistent with an unbiased literature. We estimate a plausible range for the prior probability of this body of research, suggesting expected statistical power in the original studies of 48–75%, producing (positive) findings that were expected to be true 41–62% of the time. Publication bias was large, assuming a literature with 90% positive findings, indicating that negative evidence was expected to have been observed 55–98 times before one negative result was published. These findings imply that even when studied associations are truly NULL, we expect the literature to be dominated by statistically significant findings.

scholarly journals Detecting Symptom Errors in Neural Machine Translation of Patient Health Information on Depressive Disorders: Developing Interpretable Bayesian Machine Learning Classifiers

2021 ◽  
Vol 12 ◽  
Author(s):  
Wenxiu Xie ◽  
Meng Ji ◽  
Mengdan Zhao ◽  
Tianqi Zhou ◽  
Fan Yang ◽  
...  
Keyword(s):  
Machine Learning ◽  
Health Information ◽  
Machine Translation ◽  
Posterior Probability ◽  
Depressive Disorders ◽  
Diagnostic Utility ◽  
Neural Machine Translation ◽  
Machine Learning Classifiers ◽  
Patient Health ◽  
Bayesian Machine Learning

Background: Due to its convenience, wide availability, low usage cost, neural machine translation (NMT) has increasing applications in diverse clinical settings and web-based self-diagnosis of diseases. Given the developing nature of NMT tools, this can pose safety risks to multicultural communities with limited bilingual skills, low education, and low health literacy. Research is needed to scrutinise the reliability, credibility, usability of automatically translated patient health information.Objective: We aimed to develop high-performing Bayesian machine learning classifiers to assist clinical professionals and healthcare workers in assessing the quality and usability of NMT on depressive disorders. The tool did not require any prior knowledge from frontline health and medical professionals of the target language used by patients.Methods: We used Relevance Vector Machine (RVM) to increase generalisability and clinical interpretability of classifiers. It is a typical sparse Bayesian classifier less prone to overfitting with small training datasets. We optimised RVM by leveraging automatic recursive feature elimination and expert feature refinement from the perspective of health linguistics. We evaluated the diagnostic utility of the Bayesian classifier under different probability cut-offs in terms of sensitivity, specificity, positive and negative likelihood ratios against clinical thresholds for diagnostic tests. Finally, we illustrated interpretation of RVM tool in clinic using Bayes' nomogram.Results: After automatic and expert-based feature optimisation, the best-performing RVM classifier (RVM_DUFS12) gained the highest AUC (0.8872) among 52 competing models with distinct optimised, normalised features sets. It also had statistically higher sensitivity and specificity compared to other models. We evaluated the diagnostic utility of the best-performing model using Bayes' nomogram: it had a positive likelihood ratio (LR+) of 4.62 (95% C.I.: 2.53, 8.43), and the associated posterior probability (odds) was 83% (5.0) (95% C.I.: 73%, 90%), meaning that approximately 10 in 12 English texts with positive test are likely to contain information that would cause clinically significant conceptual errors if translated by Google; it had a negative likelihood ratio (LR-) of 0.18 (95% C.I.: 0.10,0.35) and associated posterior probability (odds) was 16% (0.2) (95% C.I: 10%, 27%), meaning that about 10 in 12 English texts with negative test can be safely translated using Google.

scholarly journals Some Applications of Bayes' Rule in Probability Theory to Electrocatalytic Reaction Engineering

2011 ◽  
Vol 2011 ◽  
pp. 1-5 ◽  
Author(s):  
Thomas Z. Fahidy
Keyword(s):  
Bayesian Methods ◽  
Posterior Probability ◽  
Prior Probability ◽  
Reaction Engineering ◽  
Bayes Rule ◽  
Reactor Performance ◽  
Tafel Slopes ◽  
Synthetic Metal ◽  
Electrocatalytic Reaction

Bayesian methods stem from the principle of linking prior probability and conditional probability (likelihood) to posterior probability via Bayes' rule. The posterior probability is an updated (improved) version of the prior probability of an event, through the likelihood of finding empirical evidence if the underlying assumptions (hypothesis) are valid. In the absence of a frequency distribution for the prior probability, Bayesian methods have been found more satisfactory than distribution-based techniques. The paper illustrates the utility of Bayes' rule in the analysis of electrocatalytic reactor performance by means of four numerical examples involving a catalytic oxygen cathode, hydrogen evolution on a synthetic metal, the reliability of a device testing the quality of an electrocatalyst, and the range of Tafel slopes exhibited by an electrocatalyst.

Bayesian Data Mining and Knowledge Discovery

Data Mining ◽  
2011 ◽  
pp. 260-277
Author(s):  
Eitel J.M. Lauria ◽  
Giri Kumar Tayi
Keyword(s):  
Data Mining ◽  
Bayesian Methods ◽  
Practical Method ◽  
Bayesian Belief Networks ◽  
Bayes Theorem ◽  
Theoretical Concept ◽  
Machine Learning Techniques ◽  
Probability Models ◽  
Simulation Techniques ◽  
Basic Probability

One of the major problems faced by data-mining technologies is how to deal with uncertainty. The prime characteristic of Bayesian methods is their explicit use of probability for quantifying uncertainty. Bayesian methods provide a practical method to make inferences from data using probability models for values we observe and about which we want to draw some hypotheses. Bayes’ Theorem provides the means of calculating the probability of a hypothesis (posterior probability) based on its prior probability, the probability of the observations, and the likelihood that the observational data fits the hypothesis. The purpose of this chapter is twofold: to provide an overview of the theoretical framework of Bayesian methods and its application to data mining, with special emphasis on statistical modeling and machine-learning techniques; and to illustrate each theoretical concept covered with practical examples. We will cover basic probability concepts, Bayes’ Theorem and its implications, Bayesian classification, Bayesian belief networks, and an introduction to simulation techniques.

scholarly journals Policy relevance of Bayesian statistics overestimated?

2004 ◽  
Vol 20 (4) ◽  
pp. 488-492 ◽  
Author(s):  
Gert Jan van der Wilt ◽  
Maroeska Rovers ◽  
Huub Straatman ◽  
Sjoukje van der Bij ◽  
Paul van den Broek ◽  
...  
Keyword(s):  
Health Care Professionals ◽  
Posterior Probability ◽  
Prior Probability ◽  
Otitis Media With Effusion ◽  
Probability Distributions ◽  
Bayes Theorem ◽  
Policy Makers ◽  
Subjective Beliefs ◽  
Probability Estimates ◽  
New Evidence

Objectives: The observed posterior probability distributions regarding the benefits of surgery for otitis media with effusion (OME) with expected probability distributions, using Bayes' theorem are compared.Methods: Postal questionnaires were used to assess prior and posterior probability distributions among ear-nose-throat (ENT) surgeons in the Netherlands.Results: In their prior probability estimates, ENT surgeons were quite optimistic with respect to the effectiveness of tube insertion in the treatment of OME. The trial showed no meaningful benefit of tubes on hearing and language development. Posterior probabilities calculated on the basis of prior probability estimates and trial results differed widely from those, elicited empirically 1 year after completion of the trial and dissemination of the results.Conclusions: ENT surgeons did not adjust their opinion about the benefits of surgical treatment of glue ears to the extent that they should have done according to Bayes' theorem. Users of the results of Bayesian analyses, notably policy-makers, should realize that Bayes' theorem is prescriptive and not necessarily descriptively correct. Health policy decisions should not be based on the untested assumption that health-care professionals use new evidence to adjust their subjective beliefs in a Bayesian manner.

scholarly journals Are Multi-Armed Bandits Susceptible to Peeking?

2018 ◽  
Vol 21 (1) ◽  
pp. 95-104
Author(s):  
Markus Loecher
Keyword(s):  
Standard Method ◽  
Bayesian Methods ◽  
Web Sites ◽  
Multiple Testing ◽  
Posterior Probability ◽  
Prior Probability ◽  
Strong Dependence ◽  
P Value ◽  
Repeated Testing ◽  
Parameters Of Interest

Abstract A standard method to evaluate new features and changes to e.g. Web sites is A/B testing. A common pitfall in performing A/B testing is the habit of looking at a test while it’s running, then stopping early. Due to the implicit multiple testing, the p-value is no longer trustworthy and usually too small. We investigate the claim that Bayesian methods, unlike frequentist tests, are immune to this “peeking” problem. We demonstrate that two regularly used measures, namely posterior probability and value remaining are severely affected by repeated testing. We further show a strong dependence on the prior probability of the parameters of interest.

Bayesian Inference

2019 ◽  
pp. 37-47
Author(s):  
Therese M. Donovan ◽  
Ruth M. Mickey
Keyword(s):  
Bayesian Inference ◽  
Prior Knowledge ◽  
Scientific Method ◽  
Posterior Probability ◽  
Prior Probability ◽  
Bayes Theorem ◽  
Scientific Inference ◽  
Alternative Hypotheses

Chapter 4 introduces the concept of Bayesian inference. The chapter discusses the scientific method, and illustrates how Bayes’ Theorem can be used for scientific inference. Bayesian inference is the use of Bayes’ Theorem to draw conclusions about a set of mutually exclusive and exhaustive alternative hypotheses by linking prior knowledge about each hypothesis with new data. The result is updated probabilities for each hypothesis of interest. By the end of this chapter, the reader will understand the concepts of induction and deduction, prior probability of a hypothesis, likelihood of the observed data, and posterior probability of a hypothesis, given the data.

A simple Bayesian method of inferring extinction

Paleobiology ◽  
10.1666/13074 ◽  
2014 ◽  
Vol 40 (4) ◽  
pp. 584-607 ◽  
Author(s):  
John Alroy
Keyword(s):  
Posterior Probability ◽  
Bayesian Method ◽  
Prior Probability ◽  
Bayes Theorem ◽  
Costa Rican ◽  
Time Interval ◽  
Reasonable Person ◽  
Critically Endangered Species ◽  
The Status ◽  
Sampling Probability

Determining whether a species has gone extinct is a central problem in both paleobiology and conservation biology. Past literature has mostly employed equations that yield confidence intervals around the endpoints of temporal ranges. These frequentist methods calculate the chance of not having seen a species lately given that it is still alive (a conditional probability). However, any reasonable person would instead want to know the chance that a species is extinct given that it has not been seen (the posterior probability). Here, I present a simple Bayesian equation that estimates posteriors. It uninterestingly assumes that the sampling probability equals the frequency of sightings within the range. It interestingly sets the prior probability of going extinct during any one time interval (E) by assuming that extinction is an exponential decay process and there is a 50% chance a species has gone extinct by the end of its observed range. The range is first adjusted for undersampling by using a routine equation. Bayes' theorem is then used to compute the posterior for interval 1 (ε1), which becomes the prior for interval 2. The following posterior ε2again incorporates E because extinction might have happened instead during interval 2. The posteriors are called “creeping-shadow-of-a-doubt values” to emphasize the uniquely iterative nature of the calculation. Simulations show that the method is highly accurate and precise given moderate to high sampling probabilities and otherwise conservative, robust to random variation in sampling, and able to detect extinction pulses after a short lag. Improving the method by having it consider clustering of sightings makes it highly resilient to trends in sampling. Example calculations involving recently extinct Costa Rican frogs and Maastrichtian ammonites show that the method helps to evaluate the status of critically endangered species and identify species likely to have gone extinct below some stratigraphic horizon.

Bayesian Applications in Auditory Research

2019 ◽  
Vol 62 (3) ◽  
pp. 577-586 ◽  
Author(s):  
Garnett P. McMillan ◽  
John B. Cannon
Keyword(s):  
Monte Carlo ◽  
Markov Chain ◽  
Bayesian Methods ◽  
Prior Distribution ◽  
Interim Analysis ◽  
Iterative Process ◽  
Bayes Theorem ◽  
Sound Therapy ◽  
The Many ◽  
Auditory Research

Purpose This article presents a basic exploration of Bayesian inference to inform researchers unfamiliar to this type of analysis of the many advantages this readily available approach provides. Method First, we demonstrate the development of Bayes' theorem, the cornerstone of Bayesian statistics, into an iterative process of updating priors. Working with a few assumptions, including normalcy and conjugacy of prior distribution, we express how one would calculate the posterior distribution using the prior distribution and the likelihood of the parameter. Next, we move to an example in auditory research by considering the effect of sound therapy for reducing the perceived loudness of tinnitus. In this case, as well as most real-world settings, we turn to Markov chain simulations because the assumptions allowing for easy calculations no longer hold. Using Markov chain Monte Carlo methods, we can illustrate several analysis solutions given by a straightforward Bayesian approach. Conclusion Bayesian methods are widely applicable and can help scientists overcome analysis problems, including how to include existing information, run interim analysis, achieve consensus through measurement, and, most importantly, interpret results correctly. Supplemental Material https://doi.org/10.23641/asha.7822592

Sign in / Sign up

Export Citation Format

Share Document