Focusing on research quality is a crucial aspect of modern evidence-based medical practice, providing substantial evidence to underpin clinical decision-making. The increase in real-world studies in recent years has presented challenges, with varying quality stemming from issues such as data integrity and researchers’ expertise levels. Although systematic reviews and meta-analyses are essential references for clinical decisions, their reliability is contingent upon the quality of the primary studies. Making clinical decisions based on inadequate research poses inherent risks. With the lack of a specialized tool for evaluating the quality of real-world studies within systematic reviews and meta-analyses, the Gebrye team has introduced a new assessment tool - QATSM-RWS. Comprising 5 modules and 14 items, this tool aims to improve real-world research evaluation. This article aims to elaborate on the tool’s development process and content, taking using this tool to evaluate a published real-world study as an example and providing valuable guidance for domestic researchers utilizing this innovative tool.