Most traditional olive-producing countries possess a diversified genetic patrimony in Olea europaea L. Since the emergence of modern olive growing system, the identification, classification, and conservation of autochthonous olive cultivars is a priority for these countries. In this work, a total of 84 accessions belonging to the “Boughrara”-Sfax olive germplasm collection located in Tunisia have been screened using a powerful set of eight simple sequence repeat markers (SSRs). The study revealed a high genetic variability among the collection and detected a total of 64 alleles. For better management of the mentioned germplasm bank, an improved classification of the entries, including new denominations, has been proposed. In addition, several cases of mislabeling, synonymy, and homonymy have been clarified. Genetic relationships among cultivars have been analyzed showing four major clusters. Finally, a correspondence factor analysis demonstrated that cultivars tend to cluster depending on their main use as oil or table olives. No clear clustering tendencies were observed when the geographical origin of cultivars was used as the criteria for the analysis. All results obtained by SSR screening and classification were in accordance with classification based on morphological traits of fruit endocarps.