Different disciplines have shown interest in the auditory evaluation of song and speech production. Existing voice assessments have specific purposes and are used mainly by voice professionals to evaluate voices in clinical, linguistic, and pedagogical contexts. However, the voice in the context of art has been increasingly of interest to researchers in disciplines such as psychology or musicology, particularly those who focus on reactions and sensations that reflect the impact singing voices have on our everyday lives. To facilitate such research with participants from a larger population, a tool is needed that untrained listeners can use to generate comprehensive vocal profiles representing the particularities of different voices. To achieve this goal, in an interview (N = 20) and a group study (N = 48), free voice descriptions by untrained listeners of 23 primarily popular music singing voices were compared with terms used by voice professionals, revealing a set of nine bipolar items indicating sound quality, pitch changes, mode of phonation (song or speech), articulation, and overall expression. For validation, these items were used for the evaluation of six popular song voices by trained and untrained listeners in a German (N = 216) and an English (N = 50) sample. Providing proof of concept, a discriminant analysis revealed that the tool could be used to satisfactorily distinguish between these voices. As neither expertise nor language had an effect on the evaluation, this short tool can be used in future research whenever evaluations of singing voices in English or German are needed.