scholarly journals Parsimony hierarchies for inductive inference

2004 ◽  
Vol 69 (1) ◽  
pp. 287-327 ◽  
Author(s):  
Andris Ambainis ◽  
John Case ◽  
Sanjay Jain ◽  
Mandayam Suraj
Keyword(s):  
Inductive Inference ◽  
Computable Function ◽  
Programming System ◽  
Minimal Size ◽  
Scientific Inference ◽  
Learning Programs ◽  
Independent Criterion ◽  
Total Procedure ◽  
Computable Functions ◽  
Ordinal Notation

AbstractFreivalds defined an acceptable programming system independent criterion for learning programs for functions in which the final programs were required to be both correct and “nearly” minimal size. i.e.. within a computable function of being purely minimal size. Kinber showed that this parsimony requirement on final programs limits learning power. However, in scientific inference, parsimony is considered highly desirable. Alim-computable functionis (by definition) one calculable by a total procedure allowed to change its mind finitely many times about its output. Investigated is the possibility of assuaging somewhat the limitation on learning power resulting from requiring parsimonious final programs by use of criteria which require the final, correct programs to be “not-so-nearly” minimal size, e.g., to be within a lim-computable function of actual minimal size. It is shown that some parsimony in the final program is thereby retained, yet learning power strictly increases. Considered, then, are lim-computable functions as above but for whichnotations forconstructive ordinals are used to bound the number of mind changes allowed regarding the output. This is a variant of an idea introduced by Freivalds and Smith. For this ordinal notation complexity bounded version of lim-computability, the power of the resultant learning criteria form finely graded, infinitely ramifying, infinite hierarchies intermediate between the computable and the lim-computable cases. Some of these hierarchies, for the natural notations determining them, are shown to be optimally tight.

SIZES OF ORDERED DECISION TREES

2002 ◽  
Vol 13 (03) ◽  
pp. 445-458 ◽  
Author(s):  
HANS ZANTEMA ◽  
HANS L. BODLAENDER
Keyword(s):  
Machine Learning ◽  
Decision Tree ◽  
Decision Trees ◽  
Blow Up ◽  
Inductive Inference ◽  
Hard Problem ◽  
Minimal Size ◽  
Knowledge Based ◽  
Np Hard Problem ◽  
Decision Tables

Decision tables provide a natural framework for knowledge acquisition and representation in the area of knowledge based information systems. Decision trees provide a standard method for inductive inference in the area of machine learning. In this paper we show how decision tables can be considered as ordered decision trees: decision trees satisfying an ordering restriction on the nodes. Every decision tree can be represented by an equivalent ordered decision tree, but we show that doing so may exponentially blow up sizes, even if the choice of the order is left free. Our main result states that finding an ordered decision tree of minimal size that represents the same function as a given ordered decision tree is an NP-hard problem; in earlier work we obtained a similar result for unordered decision trees.

scholarly journals Denjoy, Demuth and density

2014 ◽  
Vol 14 (01) ◽  
pp. 1450004 ◽  
Author(s):  
Laurent Bienvenu ◽  
Rupert Hölzl ◽  
Joseph S. Miller ◽  
André Nies
Keyword(s):  
Computable Function ◽  
Density Theorem ◽  
Positive Density ◽  
Random Real ◽  
Open Problems ◽  
Covering Problems ◽  
Closed Class ◽  
Random Reals ◽  
Upper And Lower Derivatives ◽  
Computable Functions

We consider effective versions of two classical theorems, the Lebesgue density theorem and the Denjoy–Young–Saks theorem. For the first, we show that a Martin-Löf random real z ∈ [0, 1] is Turing incomplete if and only if every effectively closed class 𝒞 ⊆ [0, 1] containing z has positive density at z. Under the stronger assumption that z is not LR-hard, we show that every such class has density one at z. These results have since been applied to solve two open problems on the interaction between the Turing degrees of Martin-Löf random reals and K-trivial sets: the noncupping and covering problems. We say that f : [0, 1] → ℝ satisfies the Denjoy alternative at z ∈ [0, 1] if either the derivative f′(z) exists, or the upper and lower derivatives at z are +∞ and -∞, respectively. The Denjoy–Young–Saks theorem states that every function f : [0, 1] → ℝ satisfies the Denjoy alternative at almost every z ∈ [0, 1]. We answer a question posed by Kučera in 2004 by showing that a real z is computably random if and only if every computable function f satisfies the Denjoy alternative at z. For Markov computable functions, which are only defined on computable reals, we can formulate the Denjoy alternative using pseudo-derivatives. Call a real zDA-random if every Markov computable function satisfies the Denjoy alternative at z. We considerably strengthen a result of Demuth (Comment. Math. Univ. Carolin.24(3) (1983) 391–406) by showing that every Turing incomplete Martin-Löf random real is DA-random. The proof involves the notion of nonporosity, a variant of density, which is the bridge between the two themes of this paper. We finish by showing that DA-randomness is incomparable with Martin-Löf randomness.

Computational speed-up by effective operators

1972 ◽  
Vol 37 (1) ◽  
pp. 55-68 ◽  
Author(s):  
Albert R. Meyer ◽  
Patrick C. Fischer
Keyword(s):  
Turing Machine ◽  
Computable Function ◽  
Turing Machines ◽  
Complexity Measures ◽  
Computational Speed ◽  
Partial Recursive Functions ◽  
Speed Up ◽  
Recursive Equations ◽  
Computable Functions ◽  
Effective Operators

The complexity of a computable function can be measured by considering the time or space required to compute its values. Particular notions of time and space arising from variants of Turing machines have been investigated by R. W. Ritchie [14], Hartmanis and Stearns [8], and Arbib and Blum [1], among others. General properties of such complexity measures have been characterized axiomatically by Rabin [12], Blum [2], Young [16], [17], and McCreight and Meyer [10].In this paper the speed-up and super-speed-up theorems of Blum [2] are generalized to speed-up by arbitrary total effective operators. The significance of such theorems is that one cannot equate the complexity of a computable function with the running time of its fastest program, for the simple reason that there are computable functions which in a very strong sense have no fastest programs.Let φi be the ith partial recursive function of one variable in a standard Gödel numbering of partial recursive functions. A family Φ0, Φ1, … of functions of one variable is called a Blum measure on computation providing(1) domain (φi) = domain (Φi), and(2) the predicate [Φi(x) = m] is recursive in i, x and m.Typical interpretations of Φi(x) are the number of steps required by the ith Turing machine (in a standard enumeration of Turing machines) to converge on input x, the space or number of tape squares required by the ith Turing machine to converge on input x (with the convention that Φi(x) is undefined even if the machine fails to halt in a finite loop), and the length of the shortest derivation of the value of φi(x) from the ith set of recursive equations.

Computability and λ-definability

1937 ◽  
Vol 2 (4) ◽  
pp. 153-163 ◽  
Author(s):  
A. M. Turing
Keyword(s):  
Computable Function ◽  
General Recursive Function ◽  
Technical Details ◽  
Intuitive Idea ◽  
Short Space ◽  
Definition Of ◽  
Definable Functions ◽  
Positive Integers ◽  
Computable Functions ◽  
Definable Function

Several definitions have been given to express an exact meaning corresponding to the intuitive idea of ‘effective calculability’ as applied for instance to functions of positive integers. The purpose of the present paper is to show that the computable functions introduced by the author are identical with the λ-definable functions of Church and the general recursive functions due to Herbrand and Gödel and developed by Kleene. It is shown that every λ-definable function is computable and that every computable function is general recursive. There is a modified form of λ-definability, known as λ-K-definability, and it turns out to be natural to put the proof that every λ-definable function is computable in the form of a proof that every λ-K-definable function is computable; that every λ-definable function is λ-K-definable is trivial. If these results are taken in conjunction with an already available proof that every general recursive function is λ-definable we shall have the required equivalence of computability with λ-definability and incidentally a new proof of the equivalence of λ-definability and λ-K-definability.A definition of what is meant by a computable function cannot be given satisfactorily in a short space. I therefore refer the reader to Computable pp. 230–235 and p. 254. The proof that computability implies recursiveness requires no more knowledge of computable functions than the ideas underlying the definition: the technical details are recalled in §5.

Complexity for partial computable functions over computable Polish spaces

2016 ◽  
Vol 28 (3) ◽  
pp. 429-447 ◽  
Author(s):  
MARGARITA KOROVINA ◽  
OLEG KUDINOV
Keyword(s):  
Crucial Role ◽  
Polish Space ◽  
Computable Function ◽  
Topological Spaces ◽  
Computable Numbering ◽  
Polish Spaces ◽  
The Real ◽  
Computable Functions

In the framework of effectively enumerable topological spaces, we introduce the notion of a partial computable function. We show that the class of partial computable functions is closed under composition, and the real-valued partial computable functions defined on a computable Polish space have a principal computable numbering. With respect to the principal computable numbering of the real-valued partial computable functions, we investigate complexity of important problems such as totality and root verification. It turns out that for some problems the corresponding complexity does not depend on the choice of a computable Polish space, whereas for other ones the corresponding choice plays a crucial role.

Questions of decidability and undecidability in Number Theory

1994 ◽  
Vol 59 (2) ◽  
pp. 353-371 ◽  
Author(s):  
B. Mazur
Keyword(s):  
Number Theory ◽  
Upper Bound ◽  
Diophantine Equations ◽  
Computable Function ◽  
Integral Solution ◽  
Survey Article ◽  
Image Position ◽  
Parameter Values ◽  
Computable Functions ◽  
Integral Solutions

Davis, Matijasevic, and Robinson, in their admirable survey article [D-M-R], interpret the negative solution of Hilbert's Tenth Problem as a resounding positive statement about the versatility of Diophantine equations (that any listable set can be coded as the set of parameter values for which a suitable polynomial possesses integral solutions).One can also view the Matijasevic result as implying that there are families of Diophantine equations parametrized by a variable t, which have integral solutions for some integral values t = a > 0, and yet there is no computable function of t which provides an upper bound for the smallest integral solution for these values a. The smallest integral solutions of the Diophantine equation for these values are, at least sporadically, too large to be bounded by any computable function. This is somewhat difficult to visualize, since there is quite an array of computable functions. But let us take an explicit example. Consider the functionMatijasevic's result guarantees the existence of parametrized families of Diophantine equations such that even this function fails to yield an upper bound for its smallest integral solutions (for all values of the parameter t for which there are integral solutions).Families of Diophantine equations in a parameter t, whose integral solutions for t = 1, 2, 3,… exhibit a certain arythmia in terms of their size, have fascinated mathematicians for centuries, and this phenomenon (the size of smallest integral solution varying wildly with the parameter-value) is surprising, even when the equations are perfectly “decidable”.

scholarly journals Machine learning of higher-order programs

1994 ◽  
Vol 59 (2) ◽  
pp. 486-500 ◽  
Author(s):  
Ganesh Baliga ◽  
John Case ◽  
Sanjay Jain ◽  
Mandayam Suraj
Keyword(s):  
Machine Learning ◽  
Infinite Sequence ◽  
Computable Function ◽  
Higher Order ◽  
Global Properties ◽  
Computable Functions

AbstractA generator program for a computable function (by definition) generates an infinite sequence of programs all but finitely many of which compute that function. Machine learning of generator programs for computable functions is studied. To motivate these studies partially, it is shown that, in some cases, interesting global properties for computable functions can be proved from suitable generator programs which cannot be proved from any ordinary programs for them. The power (for variants of various learning criteria from the literature) of learning generator programs is compared with the power of learning ordinary programs. The learning power in these cases is also compared to that of learning limiting programs, i.e., programs allowed finitely many mind changes about their correct outputs.

Learning Programs with an Easy to Calculate Set of Errors1

1992 ◽  
Vol 16 (3-4) ◽  
pp. 355-370
Author(s):  
William I. Gasarch ◽  
Ramesh K. Sitaraman ◽  
Carl H. Smith ◽  
Mahendran Velauthapillai
Keyword(s):  
Learning Process ◽  
Inductive Inference ◽  
Learning Programs ◽  
The Difference ◽  
Qualitative Measure ◽  
The One

Within the study of inductive inference a recurring theme has been to investigate the learning of programs that are not exactly correct. Previous work attempted to quantify the difference between the function to be learned and the one computed by the result of a learning process. In this paper we study a qualitative measure of approximate correctness of the result of attempting to learn a program for a given function. What we require is that the set of errors be somehow easy to describe.

Undecidable complexity statements in -arithmetic

1989 ◽  
Vol 54 (2) ◽  
pp. 415-427
Author(s):  
Ron Sigal
Keyword(s):  
Computational Complexity ◽  
Free Variable ◽  
Theoretical Computer Science ◽  
Programming System ◽  
Theoretical Computer ◽  
Independence Result ◽  
Constructive Version ◽  
Proof Techniques ◽  
Definition Of ◽  
Ordinal Notation

The failure of a large and diverse body of work to settle some of the now-classical questions of computational complexity (notably P =? NP) suggests that they might not, in fact, be resolvable by established proof techniques.Hartmanis and Hopcroft [HH] raised the issue of independent statements about computational complexity in 1976, constructing, for any consistent r.e. theory T capable of expressing statements about Turing machines, a Turing machine MT such that statements which intuitively express the computational complexity of MT are independent of T. Their technique involves a simple diagonalizing search over the theorems of T. In this paper we prove a constructive version of their independence result in the context of a generalization of a hierarchy of free variable logics defined by Rose [R1]. These logics are based on an axiomatized treatment of an extension by Löb and Wainer [LW] of Grzegorczyk's [G] hierarchy into the transfinite. Associated with each extended Grzegorczyk class (relative to an ordinal notation system S satisfying certain conditions) is the logic -arithmetic.Free variable logics are interesting from the perspective of theoretical computer science. We may construe the equational definition of functions as a form of programming system. Each -arithmetic, then, has the nice property of containing both a programming system and a logic for stating and proving facts about programs.

Sign in / Sign up

Export Citation Format

Share Document