Assembly Line Balancing (ALB) aims at optimally assigning the work elements required to assemble a product to an ordered sequence of workstations, while satisfying precedence constraints. Notwithstanding the advances and developments in ALB over the years, recent and thorough surveys on this field reveal that only a small percentage of companies employ ALB procedures to configure their assembly lines. This paradox may be attributed, to some extent, to the fact that ALB is addressed mostly under ideal conditions. Despite the time variability inherent in manufacturing tasks, there is a strong research trend towards designing and implementing algorithms that consider ALB on a deterministic basis and focus on the optimality of the proposed task assignments according to existing ALB performance measures. In this paper, the need to assess the performance of the proposed solutions of various algorithms in the literature is corroborated through simulation experiments on a benchmark ALB problem under more realistic conditions. A novel ALB index, namely the Effective Cycle Time, ECT, is proposed to assess the quality of alternative assembly line configurations in paced assembly lines operating under task times variations.