Gabon is an African country located very close to the homeland of Bantu languages (Cameroon). Starting about 5,000 years ago, Bantu-speaking populations diffused into almost all sub-Saharan Africa. By processing with computational linguistic methods (Levenshtein distance) two independently collected lexical data sets recording the pronunciation of 88 and 158 words in more than 50 linguistic varieties spoken in Gabon, we obtained a numerical classification of the major linguistic groups. We compared this classification to those available based on historical linguistics methods (cognate-sharing defined by experts), and found them to overlap, which indicates that the two methods capture the same signal of linguistic difference (and relatedness). To focus on the historical relatedness between major linguistic clusters, we controlled for the linguistic similarity related to contact, proportional to geographic vicinity, and suggested that the first Bantu-speaking groups to people Gabon where those speaking KOTA-KELE (B20) languages. The other varieties concern five different immigration waves (B10, B30, B40, B50-B60-B70—Guthrie nomenclature) that penetrated Gabon later in history. To conclude, we suggest a peopling scenario that incorporates available paleoclimatic, archaeological, and population genetic evidence.