In this study, a giant magnetostrictive ultrasonic transducer was designed with Terfenol-D as the active material. The influence of different Terfenol-D rod structures on the performance of the giant magnetostrictive transducer was investigated. Finite element analysis was used to analyze the dynamics of the theoretical giant magnetostrictive transducer and to validate the design. Eddy current losses were simulated for Terfenol-D rods of various structures using finite element analysis in ANSYS Maxwell. Two different Terfenol-D rods were then fabricated with either slices along the axial direction or slit along the radial direction, then the impedance and amplitude of the magnetostrictive ultrasonic transducer composed of the two Terfenol-D rods were measured. The results suggest that sliced Terfenol-D rods can suppress eddy currents more effectively, resulting in higher energy conversion efficiency; therefore, the Terfenol-D rods of small giant magnetostrictive transducers should be sliced along the axial direction to suppress eddy currents, whereas radial slits should be adopted for the rods of large giant magnetostrictive transducers to ensure integrity of the Terfenol-D rod and avoid adverse effects on the vibration performance of the giant magnetostrictive transducer that are caused by cutting the rods.