Background The number of dialysis patients in Japan is rising, with an increasing number requiring cardiovascular surgery. Methods We investigated the short- and long-term outcomes in 70 dialysis patients among a total of 1124 who underwent cardiovascular surgery in our hospital between 2004 and 2016. We investigated outcomes following open surgery and identified factors that affected the prognosis. We also compared the long-term survival rate with the survival rate of the Japanese dialysis population. Results The long-term survival rate was 70.6%, 51.1%, and 19.2% after 3, 5, and 10 years, respectively. The causes of long-term death were heart disease in 8 patients, cerebrovascular disease in 7, cachexia in 3, infection in 2, and other causes in 3. The freedom from cardiac death was 88.7%, 77.9%, and 54.9% after 3, 5, and 10 years, respectively. Multivariate analysis using Cox’s proportional hazard model showed that a history of atherosclerosis obliterans (hazard ratio 5.4, p = 0.05) and mediastinitis (hazard ratio 10.2, p = 0.03) were risk factors for death in long-term follow-up, and a history of atherosclerosis obliterans was an independent risk factor for cardiac death in long-term follow-up (hazard ratio 5.3, p = 0.01). Five-year survival of the study subjects was comparable to that of the Japanese dialysis population. Conclusions The prognosis for dialysis patients after open surgery was equivalent to that of Japanese dialysis patients in general. A high proportion of late postoperative deaths were due to heart disease. Patients with atherosclerosis obliterans had a poor prognosis.