A thermal radiative component is likely to accompany the first stages of the prompt emission of gamma-ray bursts (GRBs) and X-ray flashes. We analyse the effect of such a component on the observable spectrum, assuming that the observable effects are due to a dissipation process occurring below or near the thermal photosphere. For comparable energy densities in the thermal and leptonic components, the dominant emission mechanism is Compton scattering. This leads to a nearly flat energy spectrum (
νF
ν
∝
ν
0
) above the thermal peak at approximately 10–100 keV and below 10–100 MeV, for a wide range of optical depths 0.03≲
τ
≲100, regardless of the details of the dissipation mechanism or the strength of the magnetic field. For higher values of the optical depth, a Wien peak is formed at 100 keV to 1 MeV. In particular, these results are applicable to the internal shock model of GRBs, as well as to slow dissipation models, e.g. as might be expected from reconnection, if the dissipation occurs at a sub-photospheric radii. We conclude that dissipation near the thermal photosphere can naturally explain (i) clustering of the peak energy at sub-MeV energies at early times, (ii) steep slopes observed at low energies, and (iii) a flat spectrum above 10 keV at late times. Our model thus provides an alternative scenario to the optically thin synchrotron–synchrotron self-Compton model.