We thank Bryant et al1 for their reply and appreciate their intent to study an inexpensive drug with potential to be repurposed during this pandemic. We recognize their self-published protocol and attempt to register with the Cochrane Collaboration.2 Inclusion of a preferred reporting items for systematic reviews and meta-analyses checklist would have obviated mention of that item. The authors seem to be mistaken about media reports within our commentary.1 Our citations regarding off-label ivermectin use only included the New England Journal of Medicine, the Center for Disease Control Health Advisory Network, the National Institutes of Health, and the Food and Drug Administration.3–5 These acknowledgments aside, Bryant's letter serves as an opportunity to highlight important and potentially controversial considerations when performing meta-analyses.1 As we demonstrate, decisions regarding these issues can dramatically influence the results of systematic reviews and meta-analyses.

INDIVIDUAL STUDY CONDUCT AND QUALITY Individual study evaluation

Critical evaluation of individual studies is an important step in assessing validity of a meta-analysis. This includes identifying whether studies used the appropriate reporting guidelines (eg, CONSORT/STROBE).6 Guideline use is associated with increased transparency and improved methodological and reporting quality and has been described as “central to application of findings from research to clinical practice.” 7–11 Only 2 of 14 mortality randomized controlled trials (RCTs) within Bryant's meta-analysis used the reporting guidelines subjecting these studies to potential biases related to allocation, randomization, confounding, and blinding issues described below.6,7,9,12–16

Ideally, poor quality articles are identified during risk of bias (RoB) assessment. Two studies (ie, Elgazzar and Niaee) with data irregularities and statistical errors in the initial meta-analysis by Bryant et al were not identified as having a high RoB within any domain.12,17,18 RoB quality assessment can be subjective as evidenced by the 4 overlapping studies within Bryant's and Popp's Cochrane mortality meta-analyses having poor interrater agreement among RoB domains (raw agreement 40%, kappa 0.02, and 95% confidence interval −0.23 to 0.26).12,19 Others have noted similar RoB appraisal inconsistencies within Cochrane systematic reviews with only poor-to-fair interrater reliability (kappa 0.13–0.34) for all domains except allocation concealment and sequence generation.20,21 It is not clear that RoB assessment alone can adequately detect poor quality articles.

Several studies in Bryant's meta-analysis had deficiencies not identified during RoB quality assessment.13 Previously, we noted that none were constructed to detect mortality differences, 8 failed to describe basic comorbidity differences between groups, 7 had inadequate random sequence generation or allocation concealment, and many contained inappropriate statistics.3 Although these study-level issues did not always affect RoB quality assessment, they indicate that many studies had an inadequate design, conduct, or methodology. As noted below, many of these deficiencies could be prevented using a strict population, intervention, comparison, outcome, and study design (PICOS) format.

Active controls

We disagree with Bryant's opinion that active controls might “under-stat(e) ivermectin's efficacy” and not skew results in favor of ivermectin.1 Three studies comparing hydroxychloroquine/chloroquine (HCQ/CQ) as active controls with ivermectin were included in their repeat meta-analysis.13,18,22,23 Those studies contained 65% (42/65) of control deaths despite comprising only 23% (212/938) of control cases.13 We have identified 17 meta-analyses concluding HCQ/CQ increased mortality in COVID-19, potentially worsening control outcomes, and overstating ivermectin's efficacy.24–40 Only 3 meta-analyses indicated reduced mortality with HCQ/CQ, potentially favoring control outcomes.41–43 Importantly, the Food and Drug Administration and European Medicines Agency recommend using active controls that are standard of care and already approved for the studied indication which is not the case for HCQ/CQ.44–46

Blinding

Cochrane Collaboration authors and Bryant et al suggest blinding is less important in trials analyzing mortality.1,47 However, inadequate blinding can affect mortality rates with treatment effects significantly larger in studies that are unblinded or have an unclear or high RoB related to blinding.48–51 An exaggerated mortality benefit, favoring treatment, has been noted across Cochrane intervention systematic reviews and in studies analyzing treatment for acute pulmonary emboli, acute myocardial infarction, and critical illness when blinding is inadequate.48–51 Ten studies in Bryant's meta-analysis had an unclear or high blinding RoB potentially favoring ivermectin.12 Removing the 2 studies with a high RoB (Bryant, Figure 2) within blinding domains renders mortality differences between ivermectin and controls insignificant, risk ratio (RR) 0.5 (0.22–1.16).12,13

Children

Bryant et al1 noted that “around” 0.4% of study subjects were children. Children comprised 13% of the patients in one study and an unknown number in another study potentially suppressing mortality rates in these studies.52,53 The preferred reporting items for systematic reviews and meta-analyses for children (PRISMA-C) guidelines recommend separately reporting pediatric events with consideration of unique developmental aspects, metabolism, pharmacology & safety, result applicability to pediatric age groups, and performance of pediatric subgroup analyses, none of which was performed in this meta-analysis.12,13,54–56 Bryant et al combined adult and pediatric outcomes into the same summary findings without mentioning different ages within results, discussion, or conclusion. This might lead readers to interpret their meta-analysis as supporting a treatment in children, despite children comprising a small minority of total subjects.12,13

META-ANALYSIS ISSUES Article selection

Article selection and framing of a study question (PICOS format) is an important determinant of the magnitude and direction of meta-analysis results. In their meta-analysis, Popp et al19 used more restrictive PICOS criteria: P (no COVID-19 negative patients and no children), I (no coadministered medicines), C (no active controls), O (28-day mortality vs. variable time limit), and S (no quasi-RCTs). Because they used a more selective PICOS framework to select articles and address ivermectin's effect on mortality, it is not surprising that Popp's meta-analysis came to a different conclusion than Bryant's meta-analysis.12,13,19

Small studies

Although Bryant et al believed that small studies do not constitute a bias in meta-analyses, others have described small-study effects as a “major threat” to their validity.1,57 Small studies have more exaggerated effects on results and are more likely to be selected for publication if statistically significant and more frequently exhibit methodological flaws.57–61 The funnel plot constructed to analyze publication bias was described as symmetric in Bryant's meta-analysis.12 Visual funnel plot inspection is subjective with trained individuals incorrectly interpreting 47.5% of all funnel plots.62 Contrary to Bryant's interpretation, smaller studies seem clustered to the left in their funnel plot (Figure 1A, and Bryant Figure 7) indicating unpublished small studies showing no benefit to ivermectin, spurious inflated effects due to poor methodology, data irregularities, chance, or true heterogeneity.12,60,63 Egger regression of their initial meta-analysis suggested potential small study bias (Figure 1A, B). After deleting studies by Elgazzar and Niaee, this bias persisted (Figure 2A). Trim and fill analysis excluding studies by Elgazzar and Niaee indicated 3 missing small studies with a mortality adjusted RR (0.84, 95% confidence interval 0.48–1.46) (Figure 2B). Although the true RR may differ from the adjusted result, this calculation serves as a sensitivity analysis indicating publication bias may be substantial.64

FIGURE 1.: (A) Funnel plot of original data, ivermectin versus control all-cause mortality, with no exclusion of studies, using log risk ratio (logRR) for x axis instead of RR. Note that no continuity correction for zero cell events made in the original figure (Figure 7) and studies and subsets with zero events in both study arms appear to be excluded leaving 11 studies within this figure and original figure.12,52,71–75 (B) Funnel plot with trim and fill for original data set excluding no studies. Four imputed missing studies to the right (•). Adjusted RR 0.59 (0.30–1.16). Egger regression intercept −1.4 (−2.97 to 0.17), P = 0.074 (2 tailed). P60,76–79FIGURE 2.: (A) Funnel plot of original data excluding studies by elgazzar and niaee.12 (B) Funnel plot with trim and fill excluding studies by Elgazzar and Niaee.12 Three imputed missing studies to the right (•). Adjusted RR 0.84 (0.48–1.46). Egger regression intercept −1.07 (−2.3 to 0.15), P = 0.076 (2 tailed). No continuity correction for zero cell events was made. If a continuity correction adding 0.5 to each zero event cell for studies with zero events in each arm is made, 3 missing studies would still be imputed and Egger regression intercept would be −0.55 (−1.14 to 0.04), P = 0.067 (2 tailed). P60,76–79Independent patient data

Bryant et al1 make an important point that independent patient data (IPD) rely on responsiveness of individual authors and can take more time. Despite these drawbacks, IPD meta-analyses are the “gold standard” of meta-analysis, “improve the quality of data,” and “produce more reliable results.”65 An IPD meta-analysis could have detected “apparent scientific fraud” and avoided publication delays while reworking their meta-analysis.17,66 Another study with impossible data points and miscalculated statistical tests might also have been identified.3,18 A third study described patients with illness duration subsets adding up to more patients than enrolled in the entire study, whereas a fourth study miscounted/miscalculated patient's sex, ethnicity, covariates, and treatments.23,67 These discrepancies indicate potential for more serious issues identifiable using IPD. Recently, Hill et al68 acknowledged “potential fraud” in 2 studies and re-evaluated others with a high RoB from their initial meta-analysis. Their reworked meta-analysis found no effect of ivermectin on mortality after deleting problematic studies.68 Because of these issues, Hill et al concluded “it is essential that access to patient-level databases is provided.” 68 Others recommend reviewing IPD in ivermectin meta-analyses “by default” after finding studies “contain(ed) impossible numbers,” “unexplainable (data) mismatches,” “timelines not consistent with veracity of data,” and “duplication of blocks of patient records.”69

FUTURE DIRECTION

An important question regarding meta-analyses remains: how can researchers ensure the quality and trustworthiness of these types of studies in the future? The Cochrane Collaboration has provided detailed instructions on the performance of meta-analyses and review of composite articles.70 Further steps are needed to obtain reliable results from systematic reviews and meta-analyses for interventions. Ideally, included studies should follow the reporting guidelines. Double blinding should be mandatory for RCTs evaluating mortality, active controls should only comprise drugs approved by regulatory agencies for the studied disorder, adult and pediatric cases should be evaluated separately, stricter PICOS format is needed, and a statistical evaluation for small study effects should be performed. To improve transparency, included studies should make deidentified data available for independent review and if not provided “should be considered at high RoB or excluded.”69 This would allow for investigation of data irregularities and performance of IPD meta-analyses.

SUMMARY

The 20 meta-analyses highlighted in our prior commentary contained studies with multiple defects previously outlined.3 Data irregularities have led to retraction of several of these studies.68,69 Without access to data contained within any study, it is impossible to know whether that data are reliable. Removing the retracted Elgazzar article and the flawed article by Niaee et al from the meta-analysis by Bryant et al and several other meta-analyses alters their results with ivermectin no longer showing a mortality benefit in COVID-19.3 We believe this should be a wake-up call to more rigorously evaluate and select individual studies, insist authors allow more in-depth access to data contained within individual studies, and promote IPD analyses as the gold standard for meta-analyses of interventions in medicine.

REFERENCES 1. Bryant A, Lawrie TA, Fordham EJ, et al. Re: “Commentary” by Rothrock et al. “Meta-analyses did not establish improved mortality with ivermectin use in COVID-19.” Am J Ther. 2022;29. epub ahead of print. doi: 10.1097/MJT.0000000000001465. 2. Bryant A, Lawrie T, Dowsell T, et al. Ivermectin for prevention and treatment of covid-19 (protocol). The evidence-based Medical Consultancy Ltd; 2021. Available at: https://tinyurl.com/cx7pnaxa. Accessed November 27, 2021. 3. Rothrock SG, Weber KD, Giordano PA, et al. Meta-analyses do not establish improved mortality with ivermectin use in COVID-19. Am J Ther. 2021;29:e87–e94. 4. CDC Health Alert Network. Rapid increase in ivermectin prescriptions and reports of severe illness associated with use of products containing ivermectin to prevent or treat COVID-19. Emergency preparedness and response. Centers for Disease Control and Prevention. Available at: https://emergency.cdc.gov/han/2021/han00449.asp. Accessed November 1, 2021. 5. Temple C, Hoang R, Hendrickson RG. Toxic effects from ivermectin use associated with prevention and treatment of COVID-19. N Engl J Med. 2021;385:2197–2198. 6. EQUATOR Network. Reporting Guidelines for Main Study Types. Available at: https://www.equator-network.org/. Accessed December 27, 2021. 7. da Costa BR, Cevallos M, Altman DG, et al. Uses and misuses of the STROBE statement: bibliographic statement. BMJ Open. 2011;1:e000048. 8. Devereaux PJ, Manns BJ, Ghali WA, et al. The reporting of methodological factors in randomized controlled tirals and the association with a journal policy to promote adherence to the Consolidated Standards of Reporting Trials (CONSORT) checklist. Control Clin Trials. 2002;23:380–388. 9. Huwiler-Muntener K, Juni P, Junker C, et al. Quality of reporting of randomized trials as a measure of methodologic quality. JAMA. 2002;287:2701–2804. 10. Plint AC, Moher D, Morrison A, et al. Does the CONSORT checklist improve the quality of reports of randomized controlled trials? A systematic review. Med J Aust. 2006;185:263–267. 11. Kane RL, Wang J, Garrard J. Reporting in randomized clinical trials improved after adoption of the CONSORT statement. J Clin Epidemiol. 2007;60:241–249. 12. Bryant A, Lawrie TA, Dowswell T, et al. Ivermectin for prevention and treatment of COVID-19 infection: a systematic review, meta-analysis, and trial sequential analysis to inform clinical guidelines. Am J Ther. 2021;28:e434–460. 13. Bryant A, Lawrie TA, Fordham EJ. Ivermectin for prevention and treatment of COVID-19 infection: a systematic review, meta-analysis, and trial sequential analysis to inform clinical guidelines. Am J Ther. 2021;28:e434e573–e460576. 14. Moher D, Jones J, Lepage L, CONSORT group. Use of CONSORT statement and quality of reports of randomized trials. JAMA. 2001;285:1992–1995. 15. Pouwels KB, Widyakusuma NN, Groenwold RH, et al. Quality of reporting of confounding remained suboptimal after the STROBE guideline. J Clin Epidemiol. 2016;69:217–224. 16. Turner L, Shamseer L, Altman DG, et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev. 2012;11:MR000030. Issue Art. No.: MR000030. 17. Elgazzar A, Eltaweel A, Youssef SA, et al. Efficacy and safety of ivermectin for treatment and prophylaxis of covid-19 pandemic. Res Square. 2020. doi: 10.21203/rs.3.rs-100956/v2. Preprint 18. Niaee MS, Namdar P, Allami A, et al. Ivermectin as an adjunct treatment for hospitalized adult COVID-19 patients: a randomized multi-center clinical trial. Asian Pac J Trop Med. 2021;14:266–273. 19. Popp M, Stegemann M, Metzendorf MI, et al. Ivermectin for preventing and treating COVID-19. Cochrane Database Syst Rev. 2021;7:CD015017. 20. Hartling L, Ospina M, Liang Y, et al. Risk of bias versus quality assessment of randomized controlled trials: cross sectional study. BMJ. 2009;339:b4012. 21. Hartling L, Hamm MP, Milne A, et al. Testing the Risk of Bias tool showed reliability between individual reviewers and across consensus assessments of reviewer pairs. J Clin Epidemiol. 2013;66:973–981. 22. Beltran-Gonzalez JL, Gamez MG, Enciso EAM, et al. Efficacy and Safety of Ivermectin and Hydroxychloroquine in patients with severe covid-19. a randomized controlled trial. medRxiv. doi: 10.1101/2021.02.18.21252037. Available at: https://www.medrxiv.org/content/10.1101/2021.02.18.21252037v1. Preprint. 23. Galan LEB, Santos NMD, Asato MS, et al. Phase 2 randomize study on chloroquine, hydroxychloroquine or ivermectin in hospitalized patients with severe manifestations of SARS-CoV-2 infection. Pathog Glob Health. 2021;115:235–242. 24. Abdelrahman Z, Liu Q, Jiang S, et al. Evaluation of the current therapeutic approaches for COVID-19. Front Pharmacol. 2021;12:607408. 25. Axfors C, Schmitt AM, Janiaud P, et al. Mortality outcomes with hydroxychloroquine and chloroquine in COVID-19 from an international collaborative meta-analysis of randomized trials. Nat Comm. 2021;13:2349. 26. Chandrasekar VT, Venkatesalu B, Patel HK, et al. Systematic review and meta-analysis of effectiveness of treatment options against SARS-CoV-2 infection. J Med Virol. 2021;93:7750785. 27. Das RR, Jaiswal N, Dev N, et al. Efficacy and safety of anti-malarial drugs (chloroquine and hydroxychloroquine) in treatment of COVID-19 infections: a systematic review and meta-analysis. Front Med (Lausanne). 2020;7:482. 28. Fiolet T, Guihur A, Rebeaud ME, et al. Effect of hydroxychloroquine with or without azithromycin on the mortality of coronavirus disease 2019 (COVID-19) patients: a systematic review and meta-analysis. Clin Microbiol Infect. 2020;27:19–27. 29. Ghazy RM, Almaghraby A, Shaaban R, et al. A systematic review and meta-analysis on chloroquine and hydroxychloroquine as monotherapy or combined with azithromycin in COVID-19 treatment. Sci Rep. 2020;10:22139. 30. Hussain N, Chung E, Heyl JJ, et al. A meta-analysis on the effects of hydroxychloroquine on COVID-19. Cureus. 2020;12:e10005. 31. Kim MS, An MH, Kim WJ, et al. Comparative efficacy and safety of pharmacological interventions for the treatment of COVID-19: a systematic review and network meta-analysis. PLoS Med. 2020;17:e1003501. 32. Kumar J, Jain S, Meena J, et al. Efficacy and safety of hydroxychloroquine/chloroquine against SARS-CoV-2 infection: a systematic review and meta-analysis. J Infect Chemother. 2021;27:882–889. 33. Misra S, Nath M, Hadda V, et al. Efficacy of various treatment modalities for nCOV-2019: a systematic review and meta-analysis. Eur J Clin Invest. 2020;50:e13383. 34. Patel TK, Barvaliya M, Kevadiya BD, et al. Does adding of hydroxychloroquine to the standard care provide any benefit in reducing the mortality among COVID-19 patients? A systematic review. J Neuroimmune Pharmacol. 2020;15:350–358. 35. Singh AK, Singh A, Singh R, et al. Hydroxychloroquine in patients with COVID-19: a systematic review and meta-analysis. Diatebetes Metab Synd. 2020;14:589–596. 36. Ullah W, M Abdullah H, Roomi S, et al. Safety and efficacy of hydroxychloroquine in COVID-19: a systematic review and meta-analysis. J Clin Med Res. 2020;12:483–491. 37. Wu SS, Zhou QX, Zeng XY, et al. Comparative effectiveness and safety of 32 pharmacological interventions recommended by guidelines for coronavirus disease 2019: a systematic review and network meta-analysis combinging 66 trials. Chin Med J (Engl). 2021;134:1920–1929. 38. Yang TH, Chou CY, Yang YF, et al. Systematic review and meta-analysis of the effectiveness and safety of hydroxychloroquine in treating COVID-19 patients. J Chin Med Assoc. 2021;84:233–241. 39. Zang Y, Han X, He M, et al. Hydroxychlorquine use and progression or prognosis of COVID-19: a systematic review and meta-analysis. Naunyn Schmiedebergs Arch Pharmacol. 2021;394:775–782. 40. Zhang C, Jin H, Wen YF, et al. Efficacy of COVID-19 treatments: a Bayesian network meta-analysis of randomized controlled trials. Front Public Health. 2021;9:729559. 41. Di Castelnuovo A, Costanzo S, Cassone A, et al. Hydroxychloroquine and mortality in COVID-19 patients: a systematic review and a meta-analysis of observational studies and randomized controlled trials. Pathog Glob Health. 2021;115:456–466. 42. Million M, Gautret P, Colson P, et al. Clinical efficacy of chloroquine derivatives in COVID-19 infection: comparative meta-analysis between the big data and the real world. New Microbes New Infect. 2020;38:100709. 43. Talaie H, Hosseini SM, Nazari M, et al. Is there any potential management against COVID-19? A systematic review and meta-analysis. Daru. 2020;28:765–777. 44. U.S. Food and Drug Administration. Active Control. Available at: https://www.accessdata.fda.gov/cder/cmc/topic7/topic7/da_01_07_0010.htm. Accessed November 28, 2021. 45. U.S. Food and Drug Administration. Center for Drug Evaluation and ResearchGuidance document. Demonstrating substantial evidence of effectiveness for human drug and biological products. Docket Number: FDA-2019-D-4964. Available at: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/demonstrating-substantial-evidence-effectiveness-human-drug-and-biological-products. 46. Committee for Medicinal Products for Human Use. European Medicines Agency. Need for active control in therapeutic areas where use of placebo is deemed ethical and one or more established medicines are available. Available at: https://www.ema.europa.eu/en/documents/scientific-guideline/reflection-paper-need-active-control-therapeutic-areas-where-use-placebo-deemed-ethical-one-more_en.pdf. Accessed December 27, 2021. 47. Higgins JPT, Savovic J, Page MJ, et al. Assessing risk of bias in a randomized trial. In: Higgins JPT, Thomas J, eds. Cochrane Handbook for Systematic Reviews of Interventions Version 6.2; 2021. Available at: www.training.cochrane.org/handbook. Accessed December 27, 2021. 48. Chalmers TC, Celano P, Sacks HS, et al. Bias in treatment assignment in controlled clinical trials. N Engl J Med. 1983;309:1358–1361. 49. Hao Q, Dong BR, Yue J, et al. Thrombolytic therapy for pulmonary embolism. Cochrane Database Syst Rev. 2018;12:CD004437. 50. Martin GL, Trioux T, Gaudry S, et al. Association between lack of blinding and mortality results in critical care randomized controlled trials: a meta-epidemiological study. Crit Care Med. 2021;49:1800–1811. 51. Savovic Savovic J, Turner RM, Mawdsley D, et al. Association between risk of bias assessments and results of randomized trials in Cochrane reviews. The ROBES meta-epidemiologic study. Am J Epidemiol. 2018;187:113–1122. 52. Hashim HA, Maulood MF, Ali CL, et al. Controlled randomized clinical trial on using ivermectin with doxycycline for treating COVID-19 patients in Baghdad, Iraq. Iraqi JMS. 2021;19:107–115. 53. Shahbaznejad L, Davoudi A, Eslami G, et al. Effects of ivermectin in patients with COVID-19: a multicenter, double-blind, randomized, controlled clinical trial. Clin Ther. 2021;43:1007–1019. 54. Farid-Kapadia M, Joachim KC, Balasingham C, et al. Are child-centric aspects in newborn and child health systematic review and meta-analysis protocols and reports adequately reported? Two systematic reviews. Syst Rev. 2017;6:31. 55. Kapadia MZ, Askie L, Hartling L, et al. PRISMA-Children (C) and PRISMA-Protocol (P-C) extension: a study protocol for the development of guidelines for the conduct and reporting of systematic reviews and meta-analyses of newborn and child health research. BMJ Open. 2016;6:e010270. 56. PRISMA-C Preferred Reporting Items for Systematic Reviews and Meta-Analysis–Reporting for Children. EQUATOR Network. Available at: https://www.equator-network.org/library/reporting-guidelines-under-development/reporting-guidelines-under-development-for-systematic-reviews/. Accessed November 27, 2021. 57. Lin L, Shi L, Chu H, et al. The magnitude of small-study effects in the Cochrane Database of Systematic Reviews: an empirical study of nearly 30 000 meta-analyses. BMJ Evid Based Med. 2020;25:27–32. 58. Boutron I, Page MJ, Higgins JPT, et al. Considering bias and conflicts of interest among the included studies. In: Higgins JPT, Thomas J, Chandler J, eds, et al. Cochrane Handbook for Systematic Reviews of Interventions Version 6.2; 2021. Available at: www.training.cochrane.org/handbook. Accessed December 27, 2021. 59. Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. 60. Sterne JAC, Sutton AJ, Ioannidis JPA, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomized controlled trials. BMJ. 2011;343:d4002. 61. Turner RM, Bird SM, Higgins JP. The impact of study size on meta-analyses: examination of underpowered studies in Cochrane Reviews. PLoS One. 2013;8:e59202. 62. Terrin N, Schmid CH, Lau J. In an empirical evaluation of the funnel plot, researches could not visually identify publication bias. J Clin Epidemiol. 2005;58:894–901. 63. Page MJ, Higgins JPT, Sterne JAC. Chapter 13: assessing risk of bias due to missing results in a synthesis. Table 13.3.b Possible sources of asymmetry in funnel plots. In: Higgins JPT, Thomas J, Chandler J, eds, et al. Cochrane Handbook for Systematic Reviews of Interventions Version 6.2. 2021. Available at: www.training.cochrane.org/handbook. Accessed November 28, 2021. 64. Borenstein M, Hedges LV, Higgins FPT, eds, et al. Publication bias. In: Introduction to Meta-Analysis. 2nd ed, Kindle Edition. West Sussex, United Kingdom: John Wiley & Sons, Ltd; 2021:313–331. 65. About IPD meta-analyses. Cochrane Methods. Available at: https://methods.cochrane.org/ipdma/about-ipd-meta-analyses. Accessed November 15, 2021. 66. Davey M. Huge Study Supporting Ivermectin as COVID Treatment Withdrawn over Ethical Concerns. The Guardian. Available at: https://www.theguardian.com/science/2021/jul/16/huge-study-supporting-ivermectin-as-covid-treatment-withdrawn-over-ethical-concerns. Accessed November 18, 2021. 67. Mahmud R, Rahman MM, Alam I, et al. Ivermectin in combination with doxycycline for treating COVID-19 symptoms: a randomized trial. J Int Med Res. 2021;49:3000605211013550. 68. Hill A, Mirchandani M, Pilkington V. Ivermectin for COVID-19: addressing potential bias and medical fraud. Res Square. 2022;9:ofab645. 69. Lawrence JM, Meyerowitz-Katz G, Heathers JAJ, et al. The lesson of ivermectin: meta-analyses based on summary data alone are inherently unreliable. Nat Med. 2021;27:1853–1854. 70. Higgins JPT, Thomas J, Chandler J, eds, et al. Cochrane Handbook for Systematic Reviews of Interventions Version 6; 2021. Available at: www.training.cochrane.org/handbook. Accessed December 27, 2021. 71. Ahmed S, Karim MM, Ross AG, et al. A five day course of ivermectin for the treatment of COVID-19 may reduce the duration of illness. Int J Infect Dis. 2020;103:214–216. 72. Babalola OE, Bode CO, Ajayi AA, et al. Ivermectin shows clinical benefits in mild to moderate COVID19: a randomized controlled double-blind, dose response trial in Lagos. QJM. 2022;114:780–788. 73. Chaccour Chaccour C, Casellas A, Blanco-Di Matteo A, et al. The effect of early treatment with ivermectin on viral load, symptoms and humoral response in patients with non-severe COVID-19: a pilot, double-blind, placebo-controlled, randomized clinical trial. EClinicalMedicine. 2021;32:100720. 74. Mohan A, Tiwari P, Suri T, et al. Ivermectin in mild and moderate COVID-19 (RIVET-COV): a randomized, placebo-controlled trial. J Infect Chemother. 2021;27:1743–1749. 75. Petkov Petkov S. Multicenter, randomized, double-blind, placebo-controlled study investigating efficacy, safety and tolerability of ivermectin HUVE-19 in patients with proven SARS-CoV-2 infection (Covid-19) and manifested clinical symptoms; 2021. Available at: https://www.clinicaltrialsregister.eu/ctr-search/trial/2020-002091-12/BG. Accessed October 20, 2021. 76. Egger M, Smith GD, Schneider M, et al. Bias in meta-analysis detected by a simple graphical test. BMJ. 1997;315:629629. 77. Lin L, chu H, Murad MH, et al. Empirical comparison of publication bias tests in meta-analysis. J Gen Intern Med. 2018;33:1260–1267. 78. Shi L, Lin L. The trim and fill method for publication bias: practical guidelines and recommendations based on a large database of meta-analyses. Medicine (Baltimore). 2019;98:e15987. 79. Song F, Khan KS, Dinnes J, et al. Asymmetric funnel plots and publication bias in meta-analyes of diagnostic accuracy. Int J Epidemiol. 2002;31:88–95.