False positive and false negative error rates are studied for competency testing where examinees are permitted to retake the test if they fail to pass. Formulae are provided for the beta-binomial and Rasch models, and estimates based on these two models are compared for several typical situations. Although Rasch estimates are expected to be more accurate than beta-binomial estimates, differences among them are found not to be substantial in a number of practical situations. Under relatively general conditions and when test retaking is permitted, the probability of making a false negative error is zero. Under the same situation, and given that an examinee is a true nonmaster, the conditional probability of making a false positive error for this examinee is one.