AbstractIt is well known that Cayley graphs of groups are automatically vertex-transitive. A pioneer result of Kelarev and Praeger implies that Cayley graphs of semigroups can be regarded as a source of possibly new vertex-transitive graphs. In this note, we consider the following problem: Is every vertex-transitive Cayley graph of a semigroup isomorphic to a Cayley graph of a group? With the help of the results of Kelarev and Praeger, we show that the vertex-transitive, connected and undirected finite Cayley graphs of semigroups are isomorphic to Cayley graphs of groups, and all finite vertex-transitive Cayley graphs of inverse semigroups are isomorphic to Cayley graphs of groups. Furthermore, some related problems are proposed.