Modeling Missing Data in Distant Supervision for Information Extraction

Distant supervision algorithms learn information extraction models given only large readily available databases and text collections. Most previous work has used heuristics for generating labeled data, for example assuming that facts not contained in the database are not mentioned in the text, and facts in the database must be mentioned at least once. In this paper, we propose a new latent-variable approach that models missing data. This provides a natural way to incorporate side information, for instance modeling the intuition that text will often mention rare entities which are likely to be missing in the database. Despite the added complexity introduced by reasoning about missing data, we demonstrate that a carefully designed local search approach to inference is very accurate and scales to large datasets. Experiments demonstrate improved performance for binary and unary relation extraction when compared to learning with heuristic labels, including on average a 27% increase in area under the precision recall curve in the binary case.

Extensions of ordered theories by generic predicates

Given a theory T extending that of dense linear orders without endpoints (DLO), in a language ℒ ⊇ {<}, we are interested in extensions T′ of T in languages extending ℒ by unary relation symbols that are each interpreted in models of T′ as sets that are both dense and codense in the underlying sets of the models.There is a canonically “wild” example, namely T = Th(〈ℝ, <, +, ·〉) and T′ = Th(〈ℝ, <, +, · ℚ 〉). Recall that T is o-minimal, and so every open set definable in any model of T has only finitely many definably connected components. But it is well known that 〈ℝ, <, +, · ℚ 〉 defines every real Borel set, in particular, every open subset of any finite cartesian power of ℝ and every subset of any finite cartesian power of ℚ. To put this another way, the definable open sets in models of T are essentially as simple as possible, while T′ has a model where the definable open sets are as complicated as possible, as is the structure induced on the new predicate.In contrast to the preceding example, if ℝalg is the set of real algebraic numbers and T′ Th(〈ℝ, <, +, ·, 〈alg〉), then no model of T′ defines any open set (of any arity) that is not definable in the underlying model of T.

Low5 Boolean Subalgebras and Computable Copies

It is known that the spectrum of a Boolean algebra cannot contain a low4 degree unless it also contains the degree 0; it remains open whether the same holds for low5 degrees. We address the question differently, by considering Boolean subalgebras of the computable atomless Boolean algebra . For such subalgebras , we show that it is possible for the spectrum of the unary relation on to contain a low5 degree without containing 0.

On spectra of sentences of monadic second order logic with counting

We show that the spectrum of a sentence ϕ in Counting Monadic Second Order Logic (CMSOL) using one binary relation symbol and finitely many unary relation symbols, is ultimately periodic, provided all the models of ϕ are of clique width at most k, for some fixed k. We prove a similar statement for arbitrary finite relational vocabularies τ and a variant of clique width for τ-structures. This includes the cases where the models of ϕ are of tree width at most k. For the case of bounded tree-width, the ultimate periodicity is even proved for Guarded Second Order Logic GSOL. We also generalize this result to many-sorted spectra, which can be viewed as an analogue of Parikh's Theorem on context-free languages, and its analogues for context-free graph grammars due to Habel and Courcelle.Our work was inspired by Gurevich and Shelah (2003), who showed ultimate periodicity of the spectrum for sentences of Monadic Second Order Logic where only finitely many unary predicates and one unary function are allowed. This restriction implies that the models are all of tree width at most 2, and hence it follows from our result.

A uniform method for proving lower bounds on the computational complexity of logical theories

In this chapter we present a method for obtaining lower bounds on the computational complexity of logical theories, and give several illustrations of its use. This method is an extension of widely used procedures for proving the recursive undecidability of logical theories. (See Rabin [1965] and Eršov et al. [1965].) One important aspect of this method is that it is based on a family of inseparability results for certain logical problems, closely related to the well-known inseparability result of Trakhtenbrot (as refined by Vaught), that no recursive set separates the logically valid sentences from those which are false in some finite model, as long as the underlying language has at least one non-unary relation symbol. By using these inseparability results as a foundation, we are able to obtain hereditary lower bounds, i.e., bounds which apply uniformly to all subtheories of the theory. The second important aspect of this method is that we use interpretations to transfer lower bounds from one theory to another. By doing this we eliminate the need to code machine computations into the models of the theory being studied. (The coding of computations is done once and for all in proving the inseparability results.) By using interpretations, attention is centred on much simpler definability considerations, viz., what kinds of binary relations on large finite sets can be defined using short formulas in models of the theory. This is conceptually much simpler than other approaches that have been proposed for obtaining lower bounds, such as the method of bounded concatenations of Fleischmann et al. [1977]. We will deal primarily with theories in first-order logic and monadic second-order logic.

Processing capacity defined by relational complexity: Implications for comparative, developmental, and cognitive psychology

Working memory limits are best defined in terms of the complexity of the relations that can be processed in parallel. Complexity is defined as the number of related dimensions or sources of variation. A unary relation has one argument and one source of variation; its argument can be instantiated in only one way at a time. A binary relation has two arguments, two sources of variation, and two instantiations, and so on. Dimensionality is related to the number of chunks, because both attributes on dimensions and chunks are independent units of information of arbitrary size. Studies of working memory limits suggest that there is a soft limit corresponding to the parallel processing of one quaternary relation. More complex concepts are processed by “segmentation” or “conceptual chunking.” In segmentation, tasks are broken into components that do not exceed processing capacity and can be processed serially. In conceptual chunking, representations are “collapsed” to reduce their dimensionality and hence their processing load, but at the cost of making some relational information inaccessible. Neural net models of relational representations show that relations with more arguments have a higher computational cost that coincides with experimental findings on higher processing loads in humans. Relational complexity is related to processing load in reasoning and sentence comprehension and can distinguish between the capacities of higher species. The complexity of relations processed by children increases with age. Implications for neural net models and theories of cognition and cognitive development are discussed.

Some characterization theorems for infinitary universal Horn logic without equality

In this paper we mainly study preservation theorems for two fragments of the infinitary languages Lκκ, with κ regular, without the equality symbol: the universal Horn fragment and the universal strict Horn fragment. In particular, when κ is ω, we obtain the corresponding theorems for the first-order case.The universal Horn fragment of first-order logic (with equality) has been extensively studied; for references see [10], [7] and [8]. But the universal Horn fragment without equality, used frequently in logic programming, has received much less attention from the model theoretic point of view. At least to our knowledge, the problem of obtaining preservation results for it has not been studied before by model theorists. In spite of this, in the field of abstract algebraic logic we find a theorem which, properly translated, is a preservation result for the strict universal Horn fragment of infinitary languages without equality which, apart from function symbols, have only a unary relation symbol. This theorem is due to J. Czelakowski; see [5], Theorem 6.1, and [6], Theorem 5.1. A. Torrens [12] also has an unpublished result dealing with matrices of sequent calculi which, properly translated, is a preservation result for the strict universal Horn fragment of a first-order language. And in [2] of W. J. Blok and D. Pigozzi we find Corollary 6.3 which properly translated corresponds to our Corollary 19, but for the case of a first-order language that apart from its function symbols has only one κ-ary relation symbol, and for strict universal Horn sentences. The study of these results is the basis for the present work. In the last part of the paper, Section 4, we will make these connections clear and obtain some of these results from our theorems. In this way we hope to make clear two things: (1) The field of abstract algebraic logic can be seen, in part, as a disguised study of universal Horn logic without equality and so has an added interest. (2) A general study of universal Horn logic without equality from a model theoretic point of view can be of help in the field of abstract algebraic logic.

The elementary theory of free pseudo p-adically closed fields of finite corank

Let be p-adic closures of a countable Hilbertian field K. The main result of [EJ] asserts that the field has the following properties for almost all σ1,…,σe + m ϵ G(K) (in the sense of the unique Haar measure on G(K)e+m):(a) Kσ is pseudo p-adically closed (abbreviation: PpC), i.e., each nonempty absolutely irreducible variety defined over Kσ has a Kσ-rational point, provided that it has a simple rational point in each p-adic closure of Kσ.(b) G(Kσ) ≅ De,m, where De,m is the free profinite product of e copies Γ1,…, Γe of G(ℚp) and a free profinite group of rank m.(c) Kσ has exactly e nonequivalent p-adic valuation rings. They are the restrictions Oσ1,…, Oσe of the unique p-adic valuation rings on , respectively.In this paper we show that this result is in a certain sense the best possible. More precisely, we first show that the class of fields which satisfy (a)–(c) above is elementary in the appropriate language e(K), which is the ordinary first-order language of rings augmented by constant symbols for the elements of K and by e new unary relation symbols (interpreted as e p-adic valuation rings).

Completeness of two theories on ordered abelian groups and embedding relations

The first order language ℒ that we consider has two nullary function symbols 0, 1, a unary function symbol –, a binary function symbol +, a unary relation symbol 0 <, and the binary relation symbol = (equality). Let ℒ′ be the language obtained from ℒ, by adding, for each integer n > 0, the unary relation symbol n| (read “n divides”).

An inelastic model with indiscernibles

Let L be a countable language including the unary relation symbol U. Let and be L-structures such that is a proper elementary U-extension of ; i.e., , and . Under what conditions will have a proper elementary U-extension? In [2], it was shown that this is not always the case, even if and are countable. However, the examples given are completely artificial, and it still seems that in most cases will have a proper elementary U-extension.Lascar asked whether will necessarily have a proper elementary U-extension whenever it contains an infinite set of indiscernibles over . This paper gives a counterexample for Lascar's question. The example is produced by modifying one of the examples in [2], using an idea of Marcus [5].Models containing an infinite set of indiscernibles can often be “stretched” to produce larger models that share some desired nonelementary property with the original [1], [6]. However, the mere presence of indiscernibles in a model does not guarantee that it can be used in this way.If the model is not completely determined by the indiscernibles, the nonelementary property may not carry over to larger models. An example of this is given in [3]. The example for Lascar's question is further evidence that models with indiscernibles need not be “elastic”.