Objectives To establish a course evaluation model for overseas medical students in West China Medical School of Sichuan University, to identify problems in teaching and to solve problems based on evidence so as to futher improve the quality of teaching. Methods We conducted a preliminary course evaluation to determine the limitations of our evaluation methods and to convey our intention to related stakeholders. Firstly, we identified problems in teaching according to the feedback from our students using a questionnaire. Secondly, we proposed an initial list of possible solutions to these problems based on evidence from literature searching and discussion within the Department of Teaching Affairs. We submitted the initial list to the administrative departments, teaching departments (teachers) and students to identify applicable solutions through two rounds of formal consensus. Their attitudes to this procedure of evaluation and decision-making were collected. Finally, incentives were given out by administrative departments and teaching departments to facilitate the implementation of applicable solutions. Results Teachers’ English ability and the didactic teaching methods were the most concerned problems. In addition, some semesters and courses were not well arranged and there was not enough practice time. An initial list of 14 items was submitted to teaching departments and students. They all agreed with the procedure of evaluation and decision-making. We also found that some aspects of the evaluation methods and styles need to be improved. An additional paper would report further results. Conclusion This preliminary evaluation was helpful for improving teaching and formal evaluation in the future. We need to strengthen the English language skills of younger faculty and gradually adopt a model of student-centered and enquiry-based teaching. This process of evaluation and improvement should be applied as a long-term policy and an evidence-based research group should be established to work together with our quality assurance unit.