The interface between tourism and built heritage is complicated because much built heritage is located in the middle of living communities. Questions arise about how to achieve a balance between the expectations of tourists and the community. To study this question, this paper reports on tourists’ and residents’ impressions of an international heritage tourism site, the Kampong Taman Sari in Indonesia. Using a linear-numeric semantic differential as the measuring instrument and nine consensus photographs of the site as stimuli, the study investigated similarities and differences in impressions between three groups: tourists (international and domestic) and residents. Three principal dimensions were found to underlie impressions of the site: Attractiveness, Organisation, and Novelty. Significant differences were found among all three groups in their impressions of Attractiveness. In terms of impressions of the Organisation of the site, international and domestic tourists have similar impressions but these differ significantly from the impressions of residents. On the other hand, domestic tourists and residents have similar impressions of the Novelty of the site, which is evaluated differently by international tourists.