This chapter begins with a puzzle: how is it that reliable prediction is ever possible, in physics? The reason that this is puzzling is that, even if the systems we are making predictions about are governed by deterministic laws that are known to us, the information available to us is a minuscule fraction of what might in principle be required to make a prediction. The answer to the puzzle lies in the phenomenon of statistical regularity, first identified in the social sciences. In a sufficiently large population, reliable predictions can be made about the total number of events that, taken individually, are unpredictable. Aggregate order arises out of individual disorder. This means that, as James Clerk Maxwell perceived already in the nineteenth century, all observed regularities are statistical regularities. To understand these requires the use of probabilistic concepts. This means that probabilistic reasoning is required even in our most certain predictions. Probability permeates physics, and we are going to have to make sense of it.