It is extremely challenging to keep up to date with medical literature due to the high publication rate and data overload. More than 4.6 million papers are available on PubMed for different medical conditions in children (birth - 18 years). The rate of publications is increasing exponentially, starting with low numbers of publications for many decades and reaching more than 130000 publications in 2022 (Fig. 1). Electronic scoping searches of PubMed were performed on October 17, 2023 (Supplementary material). This trend presents in many medical branches1,2 implies a great challenge for healthcare professionals to keep updated with the progress in their specific fields. Consequently, one relies on reviews (both narrative and systematic) to get an overview of the specific topic.

The search of PubMed was performed on October 17, 2023.

In this paper, we focus on preclinical systematic reviews (SRs) which pertain to pediatric medicine. We start with an overview of reviews including clinical SRs, unique preclinical SRs features, and those that apply especially to pediatric medicine. Emphasis on the quality of SRs is reiterated.

SRs and meta-analyses systematically scrutinize available literature on the topic and evaluate the study limitations of the reported data, following a detailed protocol. SRs are based on robust methodology, starting with a well-defined review question, which steers the criteria for a comprehensive literature search and the precise inclusion and exclusion criteria.3 Thus, the reader gets the opportunity to reproduce the search and understand the selection of the papers. In addition, two people independently conduct the screening, extract relevant outcome data, and evaluate the risk of bias and certainty of the evidence, thus minimizing the risk of introducing an error in the process. The meta-analysis, where possible, allows to calculate the effect size for each outcome. Furthermore, the SR points out the possible presence of publication bias. SRs even report the characteristics of ongoing studies on the specific topic. By doing so, it is easier for readers to follow up on the latest developments in the field. With this premise in mind, SRs and meta-analyses are valuable tools, providing a systematic assessment of the specific questions, highlighting the knowledge gaps, and addressing independently the quality of published science, thereby raising awareness of research waste caused by studies of mediocre quality.

Despite this exponential growth of SRs in clinical medicine, they are less common in preclinical medicine (in vitro studies, animal studies, and ex-vivo studies).4 The first SR of animal studies was published by Omarini et al. in 1992.5 Their SR regarded the placental perfusion in seven different animal species either in situ or in vitro. Freedman et al. published the first meta-analysis of animal studies on the effects of dietary fat consumption on mammary tumor development in 1994.6 Since then, approximately 3000 SRs in animal studies have been published, and approximately one-third of those included a meta-analysis.4 Hunniford et al. reported in their epidemiological study conducted in 2015–2018 that approximately 54% of all preclinical SRs focused on pharmacological interventions and 46% on non-pharmacological interventions, mainly cell therapies, and surgery.7

Similar to SRs in the clinical field, preclinical SRs address a very specific research area and describe the current knowledge, possible gaps, and flaws of the study design, conduct, and reporting of each included study.8,9 By analyzing available data, SRs may prevent the duplication of experiments and thereby reduce research waste and unethical use of animals. Since launching the methodology for preclinical SRs, Radboud University in the Netherlands could reduce the use of research animals by 35% at their institution, and by 15% in the whole country.10,11 Additionally, they raised awareness of possible methodological flaws and biases, ideally resulting in improved study design, conduct, and report. Menon et al. demonstrated in their mixed case study that the conduct of preclinical SRs changed the mentality of the surveyed scientists on the quality of animal research, resulting in higher quality and transparency of the following work of the same preclinical researchers. It led to a desire to diffuse this knowledge within their research teams and advocate for the broader education of the scientists.2

Given the heterogeneity of preclinical research and multiple animal species used to model different health-related conditions, SRs may be extremely valuable in choosing the most appropriate animal model12,13 and outcome measures. SRs anticipate the information if the obtained evidence is sufficient to move the research question into the clinic or illuminate the gaps thereof justifying the need for new studies.10,14,15,16 One could argue that preclinical SRs may act as the bridge between the preclinical and clinical scientific world.

SRs are often valuable evidence sources for clinical guidelines, drug regulation processes, and decision-making tasks for physicians and policymakers, which require high quality.9,17 This is why SRs must follow rigorous and detailed guidelines for the summarized evidence to be reproducible and trustworthy. In the clinical field of healthcare, two international organizations Cochrane (formerly Cochrane Collaboration; https://www.cochrane.org) and JBI (formerly Joanna Briggs Institute, https://jbi.global/) provide the criteria and methodological standards for assessment of current evidence, periodically updating their methods based on the reflection of the new information and ever-changing needs. Importantly, they use the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system to assess the certainty of the presented evidence.18 The GRADE working group, which consists of various healthcare professionals, methodologists, guidelines developers, healthcare researchers, and economists, developed and implemented “a common, transparent and sensible approach to grade the quality of evidence and strengths of recommendations in healthcare”.18 Well-defined protocols and checklists have to be followed, involving a multi-step, peer-review process.19,20

Despite being the largest database for SRs in clinical medicine, the Cochrane Database of Systematic Reviews accounts only for 7% of published SRs.21 To date, the World Health Organization demands Cochrane standards to summarize the evidence for the development of their clinical practice guidelines.22 It has been reported that the quality of Cochrane reviews is superior to non-Cochrane reviews.9,23,24 For example, Kolaski et al. assessed the quality of SRs of interventions for children with cerebral palsy using the Measurement Tool to Assess Systematic Reviews-2 (AMSTAR-2).24,25 Eighty-three SRs were included in their analysis, four of these were Cochrane reviews. The only reviews that were approved by the AMSTAR-2 tool25 were published within Cochrane, the remaining SRs were deficient for critical and non-critical domains of AMSTAR-2 evaluation. This implies that recommendations on the treatment of children with cerebral palsy are based on critically low quality of evidence.24 One of the critical items in the AMSTAR-2 quality assessment tool is the publication of the SR protocol before conducting SR.25 Protocol registration increases the transparency and quality of the research and diminishes the risk of duplication, and potential misconduct.26 It thus appears to result in higher quality methodology.27,28,29 Protocol registration is mandatory for Cochrane reviews3 but rarely for non-Cochrane SRs, depending on journal requirements. To overcome this problem the Prospective Register of Systematic Reviews (PROSPERO, https://www.crd.york.ac.uk/prospero/) was launched in 2011.29,30 It is a free open database for the registration of protocols for SRs associated with health care. Differently to clinical trials on humans where the registration of protocol is obligatory,31 there is no such requirement for SRs. Indeed, according to a recent study by van der Braak et al., only 38% of SRs on interventions published between January 2020 and January 2021 had a preregistered/ published protocol.29 This percentage is increasing compared to 5.6% in 2013 (no protocols for SRs were found before 2013), and 31.6% in 2018.32

A tool to assess the risk of bias within SRs is the Risk of Bias in Systematic Reviews (ROBIS).33 Differently from the AMSTAR-2 that is applied for the intervention SRs,34 ROBIS may be applied for intervention, diagnostic, etiology, and prognostic SRs.33 The two tools are related and have several overlapping domains, however, they are not interchangeable. Both tools pinpoint the methodological quality (the prevention of systematic errors by study design, conduction, analysis, interpretation, and publication) and risk of bias (whether the results of the study are affected by the drawbacks in design, conduction, and analysis).35 Both AMSTAR-2 and ROBIS demonstrated good inter-rater reliability,24,25,33,35,36 being not superior to each other.36,37 Indeed, in the overview of SRs on complementary and alternative medicine therapies for infantile colic, inter-rater reliability was 0.6 for AMSTAR-2 and 0.63 for ROBIS.36 It is pivotal though to train the authors for AMSTAR-2 and ROBIS to understand the differences between these two methods, and to make the conscious choice of which one to use to follow methodological rigor.

While AMSTAR-2 and ROBIS are crucial tools to assess SR conduct, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) is the guideline for the reporting of SR.38,39 The updated version of the PRISMA guideline includes 27 items within seven sections (title, abstract, introduction, methods, results, discussion, and other information).38 The acknowledgment of PRISMA guidelines is beneficial already in the planning phase of a SR to ensure that all required items are covered and the appropriate methodological choices are made.9,38 Following the PRISMA guidelines allows the authors to generate a complete and transparent reporting of their SR.

Importantly, the PRISMA checklist determines how completely each of the seven sections of SR is reported, but does not ascertain the quality of conduct and performance. Likewise, AMSTAR-2 and ROBIS are tools to assess the conduct of SR but they do not replace the methodological guidance. It has been shown that adherence to the PRISMA checklist does not guarantee achieving AMSTAR-2 and/or ROBIS standards.40,41,42 In a quality assessment study on the timing of complementary feeding for early childhood allergy prevention, it has been demonstrated that only two SRs out of 12 fulfilled all PRISMA 2009 checklist items.39 However, both these SRs were assessed to have low and critical low quality assessed by the AMSTAR-2 tool; one of them had a low risk of bias, and the other one high risk of bias assessed by the ROBIS tool.42 Therefore, the implementation of AMSTAR-2 and ROBIS for the evaluation of SR conduct and PRISMA for the comprehensiveness of reporting is recommended (Table 1).

Preclinical studies aim to understand the pathophysiological processes of the diseases, explore and discover potential treatment strategies, and test the safety and efficacy of new drugs before the initiation of clinical trials.14,43 However, the attention to the methodological quality of primary animal studies is still unsatisfactory. Thus, preclinical SRs often have their focus on possible areas in improvement of study design, conduct, and report.

Within pediatrics, the impact of the findings of the SRs on antenatal steroids provides an excellent example. High certainty evidence shows that the administration of antenatal steroids in case of risk of preterm delivery reduces neonatal mortality.44 It is useful to look at similarities and discrepancies between the findings of preclinical45 and clinical46 SRs of antenatal steroids on long-term outcomes. The preclinical SR on antenatal steroids included 64 studies performed mainly in rodents.45 The number of primary studies included in this preclinical SR45 is twice as high as in clinical SR on antenatal steroids.46 However, it is not possible to calculate the total number of animals in this preclinical SR, due to unclear reporting of the sample size in the primary studies (personal communication with Dr. van der Merwe). This is not the case for the clinical SR where the total number of children is reported (1.25 million).46 The authors of the two SRs could not perform the subgroup analysis based on sex due to a lack of data in the primary studies.45,46 The mortality rate was also underreported in the primary studies included in the preclinical SR (personal communication with Dr. van der Merwe). The lack of information on sex, mortality rate, and how many animals were used at the entry in the primary study raises several ethical questions regarding the completeness of the reporting. The outcomes were measured on term-born animals: animals had mature organ systems and physiology, leading to a further relevant question as a translation of the data into the clinical field. In the clinical study setting betamethasone was the most used antenatal steroid (in 77% of the included studies),46 whereas in animal studies dexamethasone was used in 81% of the included studies.45 Moreover, only 28% of the animal studies used clinically equivalent doses of steroids.45 Two-thirds of studies in animals used multiple courses of steroids45 while in clinical studies 1/3 of included studies reported a single dose of antenatal steroids.46 Such divergence in the different steroids (betamethasone or dexamethasone), dosage, and administration regimen used between preclinical and clinical studies is problematic. Of note, the authors of the preclinical SR did not perform a meta-analysis of outcome data due to differences in outcome definition, animal model, the dosage of steroids, single/ multiple courses, age of the animal at assessment, and methods of outcome measurement.45

To improve the reporting of primary preclinical studies the ARRIVE (Animal Research: Reporting of In Vivo Experiments) guidelines were developed in 2010. The purpose of the ARRIVE guidelines is to increase the quality, reporting, transparency, and reproducibility of primary animal studies.47 Endorsement of these guidelines was applied by several journals. Yet, no marked progress was noted by the ARRIVE working group in 2020: randomization was reported by 30–40% of publications, blinding only by 20% of publications, sample size calculation, and basic animal characteristics below 10% of publications.47 The authors of the guidelines address two possible reasons for the limited adherence to the guidelines: scarce awareness of the weight of incomplete reporting, and to which extent the journal staff is committed to fulfilling the guidelines.47 To defeat the issue of compliance with the ARRIVE guidelines, the ARRIVE working group revised, updated, and reorganized the first version introducing a more user-friendly adaptation of ARRIVE 2.0 guidelines.47 They consist of two sets: the “ARRIVE Essential 10” and the “ARRIVE Recommended Set”. The former provides the fundamental requirements for the reliability of the manuscript: study design, sample size, measures to reduce subjective bias, outcome measures, statistical methods, animals, experimental procedures, and results.47 The “ARRIVE Recommended Set” invites to provide detailed information on animal husbandry and care, protocol registration, ethical disclosure, and declaration of interests.47

One of the possible solutions to these problems is protocol registration, or preregistration.47,48,49 Although widely accepted and used in clinical trials, it is still extremely uncommon in preclinical research. ARRIVE 2.0 guidelines strongly recommend the registration of protocol.47 Preregistration of protocol results in reporting on detailed study design, randomization, blinding, primary outcome measure, and planned analysis, which reduces risks of questionable research practices like HARKing (Hypothesizing After the Results are Known50 or cherry picking (report of advantageous results with occulting the unfavorable results).51 Registration of the primary studies’ protocols is a simple and free procedure, which might be performed in registers such as https://preclinicaltrials.eu/ or https://www.animalstudyregistry.org/.

The consultation with ARRIVE 2.0 guidelines at the protocol stage of the study enhances the chances of higher quality and addresses the potential biases. If the primary outcomes are accurately pre-specified in a-priori published protocol, the obtained data, independently whether it is positive, negative, or neutral, is more reliable.15,16,47,48,49,52,53 Moreover, it can minimize the risk of outcome switching based on results, thus the research remains hypothesis-driven and not result-driven.49

Additionally, PREPARE (Planning Research and Experimental Procedures on Animals: Recommendations for Excellence) guidelines, available at https://norecopa.no/prepare, may be used for individual animal studies.54 This guideline consists of the following parts: formulation of the study, dialog between scientists and the animal facility, and quality control of the various components of the study