Combinatorics of Genotype-Phenotype Maps: An RNA Case Study

The fundamental mechanisms of biological evolution have fascinated generations of researchers and remain popular to this day. The formulation of such a theory goes back to Darwin (1859), who in the The Origin of Species presented two fundamental principles: genetic variability caused by mutation, and natural selection. The first principle leads to diversity and the second one to the concept of survival of the fittest, where fitness is an inherited characteristic property of an individual and can basically be identified with its reproduction rate. Wright [530, 531] first recognized the importance of genetic drift in evolution in improving the evolutionary search capacity of the whole population. He viewed genetic drift merely as a process that could improve evolutionary search. About a decade later, Kimura proposed [317] that the majority of changes that are observed in evolution at the molecular level are the results of random drift of genotypes. The neutral theory of Kimura does not deny that selection plays a role, but claims that no appreciable fraction of observable molecular change can be caused by selective forces: mutations are either a disadvantage or, at best, neutral in present day organisms. Only negative selection plays a major role in the neutral evolution, in that deleterious mutants die out due to their lower fitness. Over the last few decades, there has been a shift of emphasis in the study of evolution. Instead of focusing on the differences in the selective value of mutants and on population genetics, interest has moved to evolution through natural selection as an abstract optimization problem. Given the tremendous opportunities that computer science and the physical sciences now have for contributing to the study of biological phenomena, it is fitting to study the evolutionary optimization problem in the present volume. In this chapter, we adopt the following framework: assuming that selection acts exclusively upon isolated phenotypes, we introduce the following compositum of mappings . . . Genotypes→ Phenotypes →Fitness . . . . We will refer to the first map as to the genotype-phenotype map and call the preimage of a given phenotype its neutral network. Clearly, the main ingredients here are the phenotypes and genotypes and their respective organization. In the following we will study various combinatorial properties of phenotypes and genotypes for RNA folding maps.

Phase Transitions for Quantum Search Algorithms

Phase transitions have long been studied empirically in various combinatorial searches and theoretically in simplified models [91, 264, 301, 490]. The analogy with statistical physics [397], explored throughout this volume, shows how the many local choices made during search relate to global properties such as the resulting search cost. These studies have led to a better understanding of typical search behaviors [514] and improved search methods [195, 247, 261, 432, 433]. Among the current research questions in this field are the range of algorithms exhibiting the transition behavior and the algorithm-independent problem properties associated with the difficult instances concentrated near the transition. Towards this end, the present chapter examines quantum computer [123, 126, 158, 486] algorithms for nondeterministic polynomial (NP) combinatorial search problems [191]. As with many conventional methods, they exhibit the easy-hard-easy pattern of computational cost as the degree of constraint in the problems varies. We describe how properties of the search space affect the algorithms and identify an additional structural property, the energy gap, motivated by one quantum algorithm but applicable to a variety of techniques, both quantum and classical. Thus, the study of quantum search algorithms not only extends the range of algorithms exhibiting phase transitions, but also helps identify underlying structural properties. Specifically, the next two sections describe a class of hard search problems and the form of quantum search algorithms proposed to date. The remainder of the chapter presents algorithm behaviors, relevant problem structure, arid an approximate asymptotic analysis of their cost scaling. The final section discusses various open issues in designing and evaluating quantum algorithms, and relating their behavior to problem structure. The k-satisfiability (k -SAT) problem, as discussed earlier in this volume, consists of n Boolean variables and m clauses. A clause is a logical OR of k variables, each of which may be negated. A solution is an assignment, that is, a value for each variable, TRUE or FALSE, satisfying all the clauses. An assignment is said to conflict with any clause it does not satisfy.

Ground States, Energy Landscape, and Low-Temperature Dynamics of ± J Spin Glasses

The previous three chapters have focused on the analysis of computational problems using methods from statistical physics. This chapter largely takes the reverse approach. We turn to a problem from the physics literature, the spin glass, and use the branch-and-bound method from combinatorial optimization to analyze its energy landscape. The spin glass model is a prototype that combines questions of computational complexity from the mathematical point of view and of glassy behavior from the physical one. In general, the problem of finding the ground state, or minimal energy configuration, of such model systems belongs to the class of NP-hard tasks. The spin glass is defined using the language of the Ising model, the fundamental description of magnetism at the level of statistical mechanics. The Ising model contains a set of n spins, or binary variables si, each of which can take on the value up (si = 1) or down (si= 1).

Threshold Phenomena and Influence: Perspectives from Mathematics, Computer Science, and Economics

Threshold phenomena refer to settings in which the probability for an event to occur changes rapidly as some underlying parameter varies. Threshold phenomena play an important role in probability theory and statistics, physics, and computer science, and are related to issues studied in economics and political science. Quite a few questions that come up naturally in those fields translate to proving that some event indeed exhibits a threshold phenomenon, and then finding the location of the transition and how rapid the change is. The notions of sharp thresholds and phase transitions originated in physics, and many of the mathematical ideas for their study came from mathematical physics. In this chapter, however, we will mainly discuss connections to other fields. A simple yet illuminating example that demonstrates the sharp threshold phenomenon is Condorcet's jury theorem, which can be described as follows. Say one is running an election process, where the results are determined by simple majority, between two candidates, Alice and Bob. If every voter votes for Alice with probability p > 1/2 and for Bob with probability 1 — p, and if the probabilities for each voter to vote either way are independent of the other votes, then as the number of voters tends to infinity the probability of Alice getting elected tends to 1. The probability of Alice getting elected is a monotone function of p, and when there are many voters it rapidly changes from being very close to 0 when p < 1/2 to being very close to 1 when p > 1/2. The reason usually given for the interest of Condorcet's jury theorem to economics and political science [535] is that it can be interpreted as saying that even if agents receive very poor (yet independent) signals, indicating which of two choices is correct, majority voting nevertheless results in the correct decision being taken with high probability, as long as there are enough agents, and the agents vote according to their signal. This is referred to in economics as asymptotically complete aggregation of information.

Constraint Satisfaction by Survey Propagation

Methods and analyses from statistical physics are of use not only in studying the performance of algorithms, but also in developing efficient algorithms. Here, we consider survey propagation (SP), a new approach for solving typical instances of random constraint satisfaction problems. SP has proven successful in solving random k-satisfiability (k -SAT) and random graph q-coloring (q-COL) in the “hard SAT” region of parameter space [79, 395, 397, 412], relatively close to the SAT/UNSAT phase transition discussed in the previous chapter. In this chapter we discuss the SP equations, and suggest a theoretical framework for the method [429] that applies to a wide class of discrete constraint satisfaction problems. We propose a way of deriving the equations that sheds light on the capabilities of the algorithm, and illustrates the differences with other well-known iterative probabilistic methods. Our approach takes into account the clustered structure of the solution space described in chapter 3, and involves adding an additional “joker” value that variables can be assigned. Within clusters, a variable can be frozen to some value, meaning that the variable always takes the same value for all solutions (satisfying assignments) within the cluster. Alternatively, it can be unfrozen, meaning that it fluctuates from solution to solution within the cluster. As we will discuss, the SP equations manage to describe the fluctuations by assigning joker values to unfrozen variables. The overall algorithmic strategy is iterative and decomposable in two elementary steps. The first step is to evaluate the marginal probabilities of frozen variables using the SP message-passing procedure. The second step, or decimation step, is to use this information to fix the values of some variables and simplify the problem. The notion of message passing will be illustrated throughout the chapter by comparing it with a simpler procedure known as belief propagation (mentioned in ch. 3 in the context of error correcting codes) in which no assumptions are made about the structure of the solution space. The chapter is organized as follows. In section 2 we provide the general formalism, defining constraint satisfaction problems as well as the key concepts of factor graphs and cavities, using the concrete examples of satisfiability and graph coloring.

Towards a Predictive Computational Complexity Theory for Periodically Specified Problems: A Survey

The preceding chapters in this volume have documented the substantial recent progress towards understanding the complexity of randomly specified combinatorial problems. This improved understanding has been obtained by combining concepts and ideas from theoretical computer science and discrete mathematics with those developed in statistical mechanics. Techniques such as the cavity method and the replica method, primarily developed by the statistical mechanics community to understand physical phenomena, have yielded important insights into the intrinsic difficulty of solving combinatorial problems when instances are chosen randomly. These insights have ultimately led to the development of efficient algorithms for some of the problems. A potential weakness of these results is their reliance on random instances. Although the typical probability distributions used on the set of instances make the mathematical results tractable, such instances do not, in general, capture the realistic instances that arise in practice. This is because practical applications of graph theory and combinatorial optimization in CAD systems, mechanical engineering, VLSI design, transportation networks, and software engineering involve processing large but regular objects constructed in a systematic manner from smaller and more manageable components. Consequently, the resulting graphs or logical formulas have a regular structure, and are defined systematically in terms of smaller graphs or formulas. It is not unusual for computer scientists and physicists interested in worst-case complexity to study problem instances with regular structure, such as lattice-like or tree-like instances. Motivated by this, we discuss periodic specifications as a method for specifying regular instances. Extensions of the basic formalism that give rise to locally random but globally structured instances are also discussed. These instances provide one method of producing random instances that might capture the structured aspect of practical instances. The specifications also yield methods for constructing hard instances of satisfiability and various graph theoretic problems, important for testing the computational efficiency of algorithms that solve such problems. Periodic specifications are a mechanism for succinctly specifying combinatorial objects with highly regular repetitive substructure. In the past, researchers have also used the term dynamic to refer to such objects specified using periodic specifications (see, for example, Orlin [419], Cohen and Megiddo [103], Kosaraju and Sullivan [347], and Hoppe and Tardos [260]).

Scalability, Random Surfaces, and Synchronized Computing Networks

In most cases, it is impossible to describe and understand complex system dynamics via analytical methods. The density of problems that are rigorously solvable with analytic tools is vanishingly small in the set of all problems, and often the only way one can reliably obtain a system-level understanding of such problems is through direct simulation. This chapter broadens the discussion on the relationship between complexity and statistical physics by exploring how the computational scalability of parallelized simulation can be analyzed using a physical model of surface growth. Specifically, the systems considered here are made up of a large number of interacting individual elements with a finite number of attributes, or local state variables, each assuming a countable number (typically finite) of values. The dynamics of the local state variables are discrete events occurring in continuous time. Between two consecutive updates, the local variables stay unchanged. Another important assumption we make is that the interactions in the underlying system to be simulated have finite range. Examples of such systems include: magnetic systems (spin states and spin flip dynamics); surface growth via molecular beam epitaxy (height of the surface, molecular deposition, and diffusion dynamics); epidemiology (health of an individual, the dynamics of infection and recovery); financial markets (wealth state, buy/sell dynamics); and wireless communications or queueing systems (number of jobs, job arrival dynamics). Often—as in the case we study here—the dynamics of such systems are inherently stochastic and asynchronous. The simulation of such systems is nontrivial, and in most cases the complexity of the problem requires simulations on distributed architectures, defining the field of parallel discrete-event simulations (PDES) [186, 367, 416]. Conceptually, the computational task is divided among n processing elements (PEs), where each processor evolves the dynamics of the allocated piece. Due to the interactions among the individual elements of the simulated system (spins, atoms, packets, calls, etc.) the PEs must coordinate with a subset of other PEs during the simulation. For example, the state of a spin can only be updated if the state of the neighbors is known. However, some neighbors might belong to the computational domain of another PE, thus, message passing will be required in order to preserve causality.

The Phase Transition in the Random HornSAT Problem

This chapter presents a study of the satisfiability of random Horn formulas and a search for a phase transition. In the past decade, phase transitions or sharp thresholds, have been studied intensively in combinatorial problems. Although the idea of thresholds in a combinatorial context was introduced as early as 1960 [147], in recent years it has been a major subject of research in the communities of theoretical computer science, artificial intelligence, and statistical physics. As is apparent throughout this volume, phase transitions have been observed in numerous combinatorial problems, both for the probability that an instance of a problem has a solution and for the computational cost of solving an instance. In a few cases (2-SAT, 3-XORSAT, 1-in-k SAT) the existence and location of these phase transitions have also been formally proven [7, 94, 101, 131, 156, 202]. The problem at the center of this research is that of 3-satisfiability (3-SAT). An instance of 3-SAT consists of a conjunction of clauses, where each clause is a disjunction of three literals. The goal is to find a truth assignment that satisfies all clauses. The density of a 3-S AT instance is the ratio of the number of clauses to the number of Boolean variables. We call the number of variables the size of the instance. Experimental studies [110, 395, 397, 466, 469] have shown that there is a shift in the probability of satisfiability of random 3-S AT instances, from 1 to 0, located at around density 4.27 (this is also called the crossover point}. So far, in spite of much progress in obtaining rigorous bounds on the threshold location, highlighted in the previous chapters, there is no mathematical proof of a phase transition taking place at that density [1, 132, 177]. Experimental studies also show a peak of the computational complexity around the crossover point. In Kirkpatrick and Selman [319], finite-size scaling techniques were used to suggest a phase transition at the crossover point. Later, in Coafra et al. [96], experiments showed that a phase transition of the running time from polynomial in the instance size to exponential is solver-dependent, and for several different solvers this transition occurs at a density lower than the crossover point.

Proving Conditional Randomness using the Principle of Deferred Decisions

In order to prove that a certain property holds asymptotically for a restricted class of objects such as formulas or graphs, one may apply a heuristic on a random element of the class, and then prove by probabilistic analysis that the heuristic succeeds with high probability. This method has been used to establish lower bounds on thresholds for desirable properties such as satisfiability and colorability: lower bounds for the 3-SAT threshold were discussed briefly in the previous chapter. The probabilistic analysis depends on analyzing the mean trajectory of the heuristic—as we have seen in chapter 3—and in parallel, showing that in the asymptotic limit the trajectory’s properties are strongly concentrated about their mean. However, the mean trajectory analysis requires that certain random characteristics of the heuristic’s starting sample are retained throughout the trajectory. We propose a methodology in this chapter to determine the conditional that should be imposed on a random object, such as a conjunctive normal form (CNF) formula or a graph, so that conditional randomness is retained when we run a given algorithm. The methodology is based on the principle of deferred decisions. The essential idea is to consider information about the object as being stored in “small pieces,” in separate registers. The contents of the registers pertaining to the conditional are exposed, while the rest remain unexposed. Having separate registers for different types of information prevents exposing information unnecessarily. We use this methodology to prove various randomness invariance results, one of which answers a question posed by Molloy [402].

The Satisfiability Threshold Conjecture: Techniques Behind Upper Bound Improvements

One of the most challenging problems in probability and complexity theory is to establish and determine the satisfiability threshold, or phase transition, for random 3-SAT instances: Boolean formulas consisting of clauses with exactly k literals. As the previous part of the volume has explored, empirical observations suggest that there exists a critical ratio of the number of clauses to the number of variables, such that almost all randomly generated formulas with a higher ratio are unsatisfiable while almost all randomly generated formulas with a lower ratio are satisfiable. The statement that such a crossover point really exists is called the satisfiability threshold conjecture. Experiments hint at such a direction, but as far as theoretical work is concerned, progress has been difficult. In an important advance, Friedgut [177] showed that the phase transition is a sharp one, though without proving that it takes place at a “fixed” ratio for large formulas. Otherwise, rigorous proofs have focused on providing successively better upper and lower bounds for the value of the (conjectured) threshold. In this chapter, our goal is to review the series of improvements of upper bounds for 3-SAT and the techniques leading to these. We give only a passing reference to the improvements of the lower bounds as they rely on significantly different techniques, one of which is discussed in the next chapter. Let ϕ be a random k-SAT formula constructed by selecting, uniformly and with replacement, ra clauses from the set of all possible clauses with k literals (no variable repetitions allowed within a clause) over n variables. It has been experimentally observed that as the numbers m, n of variables and clauses tend to infinity while the ratio or clause density m/n is fixed to a constant a, the property of satisfiability exhibits a phase transition. For the case of 3-SAT, when a is greater than a number that has been experimentally determined to be approximately α < 4.27, then almost all random 3-SAT formulas are unsatisfiable; that is, the fraction of unsatisfiable formulas tends to 1.