Ten essential steps for performing a systematic review: A quick tutorial
Liang-Tseng Kuo1, Shih-Chieh Shao2, Ching-Chi Chi3
1 Division of Sports Medicine, Department of Orthopedic Surgery, Chang Gung Memorial Hospital, Chiayi; School of Medicine, College of Medicine, Chang Gung University, Taoyuan, Taiwan
2 Department of Pharmacy, Chang Gung Memorial Hospital, Keelung, Taiwan
3 School of Medicine, College of Medicine, Chang Gung University; Department of Dermatology, Chang Gung Memorial Hospital, Linkou, Taoyuan, Taiwan
Correspondence Address:
Prof. Ching-Chi Chi
Department of Dermatology, Chang Gung Memorial Hospital, Linkou, No. 5, Fuxing St., Guishan Dist., Taoyuan
Taiwan
Source of Support: None, Conflict of Interest: None
DOI: 10.4103/1027-8117.362992
In recent decades, the volume of health publications has increased. A well-conducted systematic review (SR), with or without a meta-analysis, is considered the best reference for health care professionals to catch up with the best evidence. With the rapid increase in the number of published SRs, their study quality varies despite well established reporting guidelines and references. Understanding the process of producing SRs is critical to the execution and is not trivial due to the obstacles researchers may encounter. The aim of this article was to provide a step-by-step approach to help researchers (mostly novices) keep their SR up to standard. The steps presented here are a quick tutorial with reference to known and accepted international guidances and our expertise.
Keywords: Evidence-based medicine, meta-analysis, systematic review
With the increased attention to evidence-based health care,[1] the number of published systematic reviews (SRs) has been rapidly growing in recent years.[2] The quality of SRs varies despite the existence of reporting guidelines and references, including the Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting guideline[3] and the Cochrane Handbook for SRs of interventions.[4]
Herein we summarize the essential steps in performing an up-to-standard SR. Relevant examples of SRs are cited.
Identify and Formulate your Review Question (PICOS)The review question should be formulated in a clear and structured format at the beginning stage of a SR. The review question should include patients or problem (P), intervention (I), comparison intervention (C), and outcomes (O). In addition, the type of study design (S) for the review question should also be indicated. For example, the review question on treatment effects usually relies on the summarized evidence of randomized controlled trials (RCTs). Furthermore, this step should also prespecify the inclusion and exclusion criteria for selecting studies.[5],[6],[7]
Register the ProtocolTo increase research transparency, registering the study protocol is highly suggested before performing a SR. The PROSPERO is the most widely used platform for registering SRs.[8]
Evidence SearchesAfter devising a review question, the authors should develop a search strategy along with experienced information specialists, such as medical librarians, to identify potential studies from the following resources:
Electronic databases: Searching of at least three databases (e.g., PubMed or MEDLINE, Embase, and Cochrane Central Register of Controlled Trials [CENTRAL]) are suggested.[9] Furthermore, some speciality databases should be searched for specific topics (e.g., PsycINFO for psychiatry-related review questions). Restrictions on language or geographic regions should be avoided.[10]Snowballing: Scanning the references lists of included studies for potentially eligible studies is suggested.[11],[12]Handsearching: Scanning the conference abstracts of top specialty meetings may help to find the latest relevant studies.[13]Unpublished or ongoing studies: Consulting the clinical experts and searching trial registers (e.g., clinicaltrial.gov) are suggested.[14],[15] Study SelectionAfter merging the search results from various data sources, review authors identify eligible studies based on the inclusion and exclusion criteria step by step:[16]
Removal of duplicate records.Scanning the titles and abstracts of the records and identifying potentially eligible studies.Obtaining and reading the full texts of potential studies, and confirming the included studies.Two review authors should independently perform study selection, and a third review author should be consulted if the disagreement cannot be resolved by discussion.[17]
Data extraction from Included StudiesIn this step, the review authors can use a prespecified sheet to extract the data from the included studies.[18] Usually, the essential items include details of the study (e.g., first author name and publication year), patient characteristics (e.g., mean age, proportion of sex, and important comorbidity), interventions (e.g., dose and frequency), comparisons, and outcomes of the included studies.[19]
Critical Appraisal of the Included Studies (Risk-of-Bias Assessment)Two review authors should evaluate the validity of included studies using appropriate tools. For example, the Cochrane Risk of Bias 2 tool[20] is the preferred tool for RCTs, and risk of bias in non-randomised studies - of interventions (ROBINS-I)[21] is the preferred tool to be used in Cochrane reviews for nonrandomized studies of the intervention. Any disagreements should be resolved by discussion,[22] or a third review author should be consulted.[23]
Data Synthesis (Meta-Analysis)Before performing this step, review authors should judge if synthesizing the outcomes of included studies is appropriate. For example, if the clinical diversity (heterogeneity) of included studies is too large to be accepted, a meta-analysis (MA) should not be performed.[13]
If the review authors decide not to do a MA, they should summarize the outcomes of included studies. If the review authors choose to synthesize the results of the included studies, two statistical models are available (fixed-effect and random-effects models). The choice of statistical model usually depends on the clinical heterogeneity (e.g., the similarity of PICOS details in the included studies) rather than statistical heterogeneity (e.g., the similarity of point estimates with 95% confidence intervals [CIs] in the included studies).[4]
Present and Interpret the DataThe forest plot is usually used for presenting the synthesized outcomes.[24] The following items should be reported:
Point estimate and 95% CI for the pooled effects.Degree of statistical heterogeneity (e.g. Cochran's Q test, I2 statistics) among the included studies.[25] Investigate the HeterogeneityFor considerable heterogeneity, additional analyses may be needed. For example, review authors could conduct subgroup or sensitivity analyses to investigate the potential sources of heterogeneity.[26],[27],[28] Meta-regression is another statistical approach to explore potential heterogeneity.[29] However, the heterogeneity is seldom confirmed by meta-regression unless a sufficient number of studies are available.
Judge the Certainty of EvidenceTwo review authors are suggested to follow the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) methodology to independently assess the certainty of evidence for each pooled outcome as “very low,” “low,” “moderate,” or “high.”[30],[31] This method judges the overall certainty of the evidence for each outcome by considering the risk in overall risk of bias,[32] imprecision,[33] inconsistency,[34] indirectness,[35] and publication bias.[36] Disagreements will be solved by discussion. If arguments still exist after discussion, a third author should arbitrate.[17]
In summary, following the above 10 essential steps would ensure an SR meets the general requirements. Further tailoring based on the journal's instructions for authors is still needed before submitting a manuscript.
Financial support and sponsorship
Nil.
Conflicts of interest
Prof. Ching-Chi Chi, the Editor-in-Chief of Dermatologica Sinica, had no role in the peer review process of or decision to publish this article. The other authors declared no conflicts of interest in writing this paper.
References
留言 (0)