In 1840, Church Vaughan’s dying father Scipio advised his family to return to Africa, the continent of their ancestors. Although Scipio had spent most of his life as a slave, his ten children and their mother were life-long free people of color in Kershaw County, South Carolina. Through hard work and the careful cultivation of white patrons, they made an independent living and owned their own land. As this chapter shows through the experiences of this family and their neighbors, however, inland South Carolina became increasingly restrictive and dangerous for free people of color in the first half of the nineteenth century. By the time Church Vaughan’s free Anglo-Catawba mother and African American father married in the 1810s, so many slaves worked on Kershaw County cotton plantations that planters had good reason to fear rebellion, such as one that was brutally suppressed in 1816. Over the following decades, plantation slavery expanded over land previously controlled by Native Americans. Though Scipio Vaughan was gradually manumitted, even free people of color faced increasing legal restrictions, social exclusion, and violence. This chapter illustrates their limited pathways to freedom as well as the mounting pressures on free people of color that made emigration attractive.