Targeting cleaning of the artificial gypsum layer on white marble was studied. It was conducted by means of the specific depletion of the calcium and sulfate ions by the barium carbonate scavenger, which led to the continuous dissolution and clearance of gypsum layer. The cleaning effect was evaluated by scanning electron microscopy with energy dispersive X-ray spectroscopy (SEM/EDX), X-ray diffraction (XRD), capillary suction, and color difference measurement. By this method, only the gypsum layer was cleared away and the carbonate substrate of marble was left intact at the same time. This method will be highly useful for the conservation of marble relics from surface weathering.