scholarly journals The no-free-lunch theorems of supervised learning

Synthese ◽  
2021 ◽  
Author(s):  
Tom F. Sterkenburg ◽  
Peter D. Grünwald
Keyword(s):  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Data Driven ◽  
Free Lunch ◽  
Inductive Bias ◽  
No Free Lunch Theorems ◽  
No Free Lunch ◽  
Skeptical Conclusion ◽  
Better Than ◽  
Standard Learning

AbstractThe no-free-lunch theorems promote a skeptical conclusion that all possible machine learning algorithms equally lack justification. But how could this leave room for a learning theory, that shows that some algorithms are better than others? Drawing parallels to the philosophy of induction, we point out that the no-free-lunch results presuppose a conception of learning algorithms as purely data-driven. On this conception, every algorithm must have an inherent inductive bias, that wants justification. We argue that many standard learning algorithms should rather be understood as model-dependent: in each application they also require for input a model, representing a bias. Generic algorithms themselves, they can be given a model-relative justification.

scholarly journals No Free Lunch for Noise Prediction

2000 ◽  
Vol 12 (3) ◽  
pp. 547-564 ◽  
Author(s):  
Malik Magdon-Ismail
Keyword(s):  
Learning Algorithms ◽  
Target Function ◽  
Learning Systems ◽  
Superior Performance ◽  
Noise Prediction ◽  
Noise Distribution ◽  
Free Lunch ◽  
Data Set ◽  
No Free Lunch Theorems ◽  
No Free Lunch

No-free-lunch theorems have shown that learning algorithms cannot be universally good. We show that no free funch exists for noise prediction as well. We show that when the noise is additive and the prior over target functions is uniform, a prior on the noise distribution cannot be updated, in the Bayesian sense, from any finite data set. We emphasize the importance of a prior over the target function in order to justify superior performance for learning systems.

scholarly journals Algorithmic and human prediction of success in human collaboration from visual features

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Martin Saveski ◽  
Edmond Awad ◽  
Iyad Rahwan ◽  
Manuel Cebrian
Keyword(s):  
Machine Learning ◽  
Visual Cues ◽  
Success Factors ◽  
Group Performance ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Adventure Game ◽  
Group Success ◽  
The Relationship ◽  
Better Than

AbstractAs groups are increasingly taking over individual experts in many tasks, it is ever more important to understand the determinants of group success. In this paper, we study the patterns of group success in Escape The Room, a physical adventure game in which a group is tasked with escaping a maze by collectively solving a series of puzzles. We investigate (1) the characteristics of successful groups, and (2) how accurately humans and machines can spot them from a group photo. The relationship between these two questions is based on the hypothesis that the characteristics of successful groups are encoded by features that can be spotted in their photo. We analyze >43K group photos (one photo per group) taken after groups have completed the game—from which all explicit performance-signaling information has been removed. First, we find that groups that are larger, older and more gender but less age diverse are significantly more likely to escape. Second, we compare humans and off-the-shelf machine learning algorithms at predicting whether a group escaped or not based on the completion photo. We find that individual guesses by humans achieve 58.3% accuracy, better than random, but worse than machines which display 71.6% accuracy. When humans are trained to guess by observing only four labeled photos, their accuracy increases to 64%. However, training humans on more labeled examples (eight or twelve) leads to a slight, but statistically insignificant improvement in accuracy (67.4%). Humans in the best training condition perform on par with two, but worse than three out of the five machine learning algorithms we evaluated. Our work illustrates the potentials and the limitations of machine learning systems in evaluating group performance and identifying success factors based on sparse visual cues.

Application of machine learning algorithms to predict permeability in tight sandstone formations

Nafta-Gaz ◽  
2021 ◽  
Vol 77 (5) ◽  
pp. 283-292
Author(s):  
Tomasz Topór ◽  
Keyword(s):  
Machine Learning ◽  
Oil And Gas ◽  
Learning Algorithms ◽  
Confining Pressure ◽  
Machine Learning Algorithms ◽  
Core Material ◽  
Confining Stress ◽  
Tight Sandstone ◽  
Oil And Gas Exploration ◽  
Better Than

The application of machine learning algorithms in petroleum geology has opened a new chapter in oil and gas exploration. Machine learning algorithms have been successfully used to predict crucial petrophysical properties when characterizing reservoirs. This study utilizes the concept of machine learning to predict permeability under confining stress conditions for samples from tight sandstone formations. The models were constructed using two machine learning algorithms of varying complexity (multiple linear regression [MLR] and random forests [RF]) and trained on a dataset that combined basic well information, basic petrophysical data, and rock type from a visual inspection of the core material. The RF algorithm underwent feature engineering to increase the number of predictors in the models. In order to check the training models’ robustness, 10-fold cross-validation was performed. The MLR and RF applications demonstrated that both algorithms can accurately predict permeability under constant confining pressure (R2 0.800 vs. 0.834). The RF accuracy was about 3% better than that of the MLR and about 6% better than the linear reference regression (LR) that utilized only porosity. Porosity was the most influential feature of the models’ performance. In the case of RF, the depth was also significant in the permeability predictions, which could be evidence of hidden interactions between the variables of porosity and depth. The local interpretation revealed the common features among outliers. Both the training and testing sets had moderate-low porosity (3–10%) and a lack of fractures. In the test set, calcite or quartz cementation also led to poor permeability predictions. The workflow that utilizes the tidymodels concept will be further applied in more complex examples to predict spatial petrophysical features from seismic attributes using various machine learning algorithms.

A note on no-free-lunch theorem

2021 ◽  
Author(s):  
Lidong Wu
Keyword(s):  
Machine Learning ◽  
Theoretical Result ◽  
Data Driven ◽  
Np Hard ◽  
Free Lunch ◽  
No Free Lunch Theorem ◽  
Data Driven Approach ◽  
Hard Problems ◽  
No Free Lunch ◽  
Np Hard Problems

The No-Free-Lunch theorem is an interesting and important theoretical result in machine learning. Based on philosophy of No-Free-Lunch theorem, we discuss extensively on the limitation of a data-driven approach in solving NP-hard problems.

Sign in / Sign up

Export Citation Format

Share Document