Cheating with Models

Beliefs and decisions are often based on confronting models with data. What is the largest “fake” correlation that a misspecified model can generate, even when it passes an elementary misspecification test? We study an “analyst” who fits a model, represented by a directed acyclic graph, to an objective (multivariate) Gaussian distribution. We characterize the maximal estimated pairwise correlation for generic Gaussian objective distributions, subject to the constraint that the estimated model preserves the marginal distribution of any individual variable. As the number of model variables grows, the estimated correlation can become arbitrarily close to one regardless of the objective correlation. (JEL D83, C13, C46, C51)

Gradient-Aware Model-Based Policy Search

Traditional model-based reinforcement learning approaches learn a model of the environment dynamics without explicitly considering how it will be used by the agent. In the presence of misspecified model classes, this can lead to poor estimates, as some relevant available information is ignored. In this paper, we introduce a novel model-based policy search approach that exploits the knowledge of the current agent policy to learn an approximate transition model, focusing on the portions of the environment that are most relevant for policy improvement. We leverage a weighting scheme, derived from the minimization of the error on the model-based policy gradient estimator, in order to define a suitable objective function that is optimized for learning the approximate transition model. Then, we integrate this procedure into a batch policy improvement algorithm, named Gradient-Aware Model-based Policy Search (GAMPS), which iteratively learns a transition model and uses it, together with the collected trajectories, to compute the new policy parameters. Finally, we empirically validate GAMPS on benchmark domains analyzing and discussing its properties.

Should Robots be Obedient?

Intuitively, obedience -- following the order that a human gives -- seems like a good property for a robot to have. But, we humans are not perfect and we may give orders that are not best aligned to our preferences. We show that when a human is not perfectly rational then a robot that tries to infer and act according to the human's underlying preferences can always perform better than a robot that simply follows the human's literal order. Thus, there is a tradeoff between the obedience of a robot and the value it can attain for its owner. We investigate how this tradeoff is impacted by the way the robot infers the human's preferences, showing that some methods err more on the side of obedience than others. We then analyze how performance degrades when the robot has a misspecified model of the features that the human cares about or the level of rationality of the human. Finally, we study how robots can start detecting such model misspecification. Overall, our work suggests that there might be a middle ground in which robots intelligently decide when to obey human orders, but err on the side of obedience.

Grain prices, oil prices, and multiple smooth breaks in a VAR

Ignored structural breaks in a VAR result in a misspecified model such that Granger causality tests are improperly sized; there is a bias towards a rejection of the null hypothesis of non-causality even when the null is correct. Instead of modeling structural breaks as being sharp, changes in the relationship between the maize and petroleum markets are likely to have occurred gradually. We show the flexible Fourier form has good size and power properties in testing for smooth structural change in a VAR. When applied to a VAR including maize and oil prices, we uncover important linkages between the two markets.