In the Al alloy A2024-T3 extruded material, a rod-like structure is generated parallel to the extrusion direction. In this study, the effects of rod-like structures on fatigue crack initiation and growth behavior were comprehensively investigated. Two types of specimens were used in a fatigue experiment, in which the direction of the load stress amplitude was parallel (specimen P) and perpendicular (specimen V) to the rod-like structure. Based on the experimental and analytical results, the following findings were obtained regarding the fatigue life, location of crack initiation, and fatigue crack growth behavior. Because the fatigue life of specimen P was longer than that of specimen V, it is inferred that the rod-like structure significantly affects the fatigue life. In specimen P, fatigue cracks were generated from the grain boundaries of the Al matrix. By contrast, in specimen V, cracks were generated from the Cu–Mg-based intermetallic compound in the Al matrix. In specimen P, fatigue cracks were more likely to propagate across the rod-like structure, which decreased the fatigue crack growth rate. In specimen V, fatigue cracks did not propagate across the rod-like structure; instead, they propagated through the Al matrix. Therefore, the fatigue crack growth resistance of specimen V was lower than that of specimen P. The relationship between the fatigue crack growth rate and the modified linear elastic fracture mechanics parameter could be used to predict the S–N curve (stress amplitude vs. fatigue life) and fatigue crack growth behavior. The predicted results agreed well with the experimental results.