We calculate Berry’s phase when the driving field, to which a spin-[Formula: see text] is coupled adiabatically, rather than the familiar classical magnetic field, is a quantum vector operator, of noncommuting, in general, components, e.g. the angular momentum of another particle, or another spin. The geometric phase of the entire system, spin plus “quantum driving field”, is first computed, and is then subdivided into the two subsystems, using the Schmidt decomposition of the total wave function — the resulting expression shows a marked, purely quantum effect, involving the commutator of the field components. We also compute the corresponding mean “classical” phase, involving a precessing magnetic field in the presence of noise, up to terms quadratic in the noise amplitude — the results are shown to be in excellent agreement with numerical simulations in the literature. Subtleties in the relation between the quantum and classical case are pointed out, while three concrete examples illustrate the scope and internal consistency of our treatment.