The collected in situ temperature-resolved synchrotron powder data revealed that the transformation of the recently reported three-dimensional τ-Zr(HPO4)2 to cubic ZrP2O7 goes through two intermediate phases. The first intermediate phase, ρ-Zr(HPO4)2, is formed in a reversible phase transition at 598 K, which involves both rearrangement and disordering of the hydrogen phosphate groups of τ-Zr(HPO4)2. At 688 K condensation of the hydrogen phosphate groups leads to the formation of the second intermediate, a new polymorph of zirconium pyrophosphate (β-ZrP2O7). Heating above 973 K results in the gradual transformation of β-ZrP2O7 to cubic zirconium pyrophosphate (α-ZrP2O7). The crystal structures of the two intermediate phases were solved from the in situ powder diffraction data using direct methods and refined using the Rietveld method. Both phases are orthorhombic, space group Pnnm and Z = 2. The lattice parameters for the two phases are: ρ-Zr(HPO4)2: a = 8.1935 (2), b = 7.7090 (2), c = 5.4080 (1) Å; β-ZrP2O7: a = 8.3127 (5), b = 6.6389 (4), c = 5.3407 (3) Å. The formation mechanism for the new zirconium pyrophosphate polymorph, β-ZrP2O7, is discussed in relation to structurally restricted soft chemistry.