We discuss ideas and problems regarding classical and quantum gravity, gauge theory of gravity, and space–time transformations between accelerated frames. Both Einstein's theory of gravity and Yang–Mills theory are gauge invariant. The invariance principles are at the very heart of our understanding of the physical world. This paper attempts to survey the development and to reveal problems and limitations of various formulations to gravitational and Yang–Mills fields, and to space–time transformations of accelerated frames. Gravitational force and accelerated frames are two ingredients in Einstein's thought in the period around 1907. Accelerated frames are difficult to define and are not well developed. However, one cannot claim to have a complete understanding of the physical world, if one understands flat space–time physics only from the viewpoint of the special class of inertial frames and ignores the vast class of noninertial frames. The paper highlights three aspects: (1) ideas of gravity as a Yang–Mills field, first discussed by Utiyama; (2) problems of quantum gravity, discussed by Feynman, Dyson and others; (3) space–time properties and the physics of fields and particles in accelerated frames of reference. These unfulfilled aspects of Einstein and Yang–Mills' profound thoughts present a challenge to physicists and mathematicians in the 21st century.