In recent years, the boom of the quadcopter industry resulted in a broad range of real-world applications which highlighted the urgent need to improve quadcopter control quality. Typically, external disturbances, such as wind, parameter uncertainties caused by payload variations, or the ground effect, can severely degrade the quadcopter’s altitude control performance. Meanwhile, widely used controllers like the proportional-integral-derivative control cannot guarantee control performance when the system is critically affected by factors that exhibit a high degree of variability with time. In this paper, an adaptive control algorithm is proposed to improve quadcopter altitude tracking performance in the presence of both the ground effect and a time-varying payload. First, we derive an adaptive altitude control algorithm using the sliding mode control technique to account for these uncertainties in the quadcopter dynamics model. Second, we apply Lyapunov theory to analyze the stability of the closed-loop system. Finally, we conduct several numerical simulations and experiments to validate the effectiveness of the proposed method.