Objective To explore the clinical efficiency of vancomycin-loaded calcium phosphate cement (CPC) in the treatment of chronic osteomyelitis (CO). Methods From December 1st 2014 to December 1st 2015, 98 patients with CO were randomly divided into the research group and the control group, with 49 in each group. The patients in the research group were primarily implanted with vancomycin-loaded CPC after debridement, while the ones in the control group were placed with irrigation and drainage device to take continous irrigation with antibiotics after debridement. The treatment effect and the recurrence rate in the two groups were observed. Results The patients in the two groups were followed up for 12 months. In the research group, 30 patients were cured, 16 were improved, and 3 were not improved with the total effective rate of 93.9%; no systemic adverse reactions and recurrence took place after the operation; X-ray results showed well CPC tamponade and partially degenerated osteogenesis. In the control group, 16 patients were cured, 20 were improved and 13 were not improved, with the total effective rate of 73.5%; 11 had recurrent inflammation in 2–6 months after operation and were reoperated again. Conclusion The primary implantation of vancomycin-loaded CPC in CO lesions can fill the dead space, resist infection persistently, induce osteogenesis in bone defect area, and reduce the recurrence of CO, which is an effective method for the treatment of CO.