A ring with involution ∗ is called ∗-clean if each of its elements is the sum of a unit and a projection (∗-invariant idempotent). In this paper, we consider the group algebras of the dihedral groups [Formula: see text], and the generalized quaternion groups [Formula: see text] with standard involution ∗. For the non-semisimple group algebra case, we characterize the ∗-cleanness of [Formula: see text] with a prime [Formula: see text], and [Formula: see text] with [Formula: see text], where [Formula: see text] is a commutative local ring. For the semisimple group algebra case, we investigate when [Formula: see text] is ∗-clean, where [Formula: see text] is the field of rational numbers [Formula: see text] or a finite field [Formula: see text] and [Formula: see text] or [Formula: see text].