Terahertz modulators with capability of both intensity and phase are essential for THz imaging and communication systems. The low-voltage driven THz modulation technique is crucial for integrating the modulators with electronics components. There is still a lack of broadband devices able to achieve both amplitude and phase modulation with low voltage, due to the underlying physics behind existing approaches. Here, we demonstrate a graphene-loaded metal wire grating THz modulator in the total internal reflection geometry to achieve intensity modulation of 80% and phase modulation of 70 degree within 3 volts gate voltage. Quite different from using the strategy of metamaterials based on the electromagnetic resonance effects, our design has performed a broadband modulation for over 1 THz bandwidth.