This paper argues that the early lyrics of Samuel Taylor Coleridge explore the ethical work of collective guilt, a feeling with enormous Romantic and contemporary significance. Coleridge's lyrics formally model collective guilt while making a cautious case for its social value. By reading ‘Fears in Solitude’ and The Rime of the Ancient Mariner through recent work in social psychology and the philosophy of ethics, I show how Coleridge creates causalities of feeling, affirming meaningful relationships of responsibility that go beyond personal guilt. I conclude that Romantic lyric offers an ideal form not only for illustrating how collective guilt works as a ‘structure of feeling’, but also for examining the emotion's potential to create positive social change.