This study systematically and critically appraised the methodological quality of 31 CPG on PAH treatments published by four continents using AGREE II and AGREE REX tools. The results of these instruments correlate to the guidelines’ credibility and the validity of their recommendations. While adherence to clinical guidelines is paramount for healthcare providers and patients to improve the quality of care and achieve positive outcomes, periodic assessment of guidelines’ quality is also important, including to address situations where treatment acquisition is restricted and associated with high resource utilization and financial strain—as is often the case in PAH management (e.g., high cost of new pharmacological treatments, challenges with prior authorization, tiered use) [4, 5, 10]. Yet, as moderate-to-good agreement rates among reviewers were obtained when using the AGREE tools in our study, findings can be considered reliable and consistent in similar scenarios [17, 30, 31].

Overall, we found higher scores for AGREE II’s domains 1 (scope and purpose) and 4 (clarity of presentation) and for AGREE REX’s domain 1 (clinical applicability), which is similar to the findings reported by previous studies for other clinical conditions, including nutrition care, mental disorders, and medical oncology [21, 32,33,34,35,36]. These results mean a CPG document written in comprehensive language with a clear structure and presentation of goals regarding specific health issues and target populations. Guidelines written in concise, unambiguous language appear to be the most accessed by clinicians and are more likely to be implemented into clinical practice [33]. Conversely, several methodological issues were identified in domains 3 (rigor of development) and 5 (applicability) of AGREE II, and domains 2 (values and preferences) and 3 (implementability) of AGREE REX as also previously highlighted in the biomedical literature, including in areas of intensive care units, skin disorders, and lipid management for patients with coronary heart disease [21, 24, 32, 34, 35]. Insufficient scientific rigor for developing guidelines, lack of standard procedures for evidence synthesis and recommendations’ formulation, scarce information on the implementation process, and failure to consider patients’ and other target users’ opinions may negatively affect the findings of these documents. This is concerning, especially for rare diseases (such as PAH) with deficient evidence. Skewed guidelines towards researchers’ or health professionals’ values and preferences may lead to less meaningful patient guidelines and a low CPG implementation rate, contributing to knowledge waste in various clinical settings (results are inaccessible or unused) [11, 37].

Although the information on “stakeholder involvement” (domain 2 AGREE II) and “editorial independence” (domain 6 AGREE II) was fairly provided by most PAH guidelines, important pitfalls were detected on AGREE REX domains 2 and 3 (values and preferences and implementability). These findings are consistent with previous research in actinic keratosis, pressure injury, and uveal melanoma, highlighting the need to improve the proximity of clinical practice to patients’ values and characteristics, which should not be restricted to populations’ characteristics (e.g., sex, gender, race, genetic profile), and consider behavioral and social aspects and treatment costs. The involvement of other stakeholders, such as health policymakers and government representatives, is paramount for the continuous development and practical implementation of a guideline [24, 38, 39]. For this to occur, communication between CPG’s working parties, including a wide range of representatives (e.g., professional organizations, regional and local offices, and relevant national bodies), is essential. Nonetheless, there may be several barriers hampering involvement, including the identification and recruitment of appropriate stakeholders, the definition of roles and responsibilities, and disclaiming the potential conflicts of interest and resources, which should be clearly stated in the guideline (either on the main document or supplementary materials) for data transparency. Moreover, if applicability issues, including types of facilitators and barriers surrounding guideline implementation, are not adequately addressed, CPG may be ineffective in clinical practice and increase the waste of resources. O’Connor et al. [40] reported that less than 20% of the published guidelines on antenatal management of twin pregnancies were considered in implementing recommendations. The costs and budget concerns of applying the findings are equally poorly communicated by guidelines in PAH or other areas (including mental health, viral infections, and oncology), limiting the extent to which health systems can disseminate and use the evidence [41, 42].

Although there has been a significant increase in the number of CPG publications in the past years, including for PAH [43, 44], which is probably due to the approval of new treatments and the intention to improve the quality of care for patients while informing healthcare costs and reducing variability in clinical practice [43], their overall methodological quality (AGREE II scores) has not improved, reinforcing the need to strictly develop and report these documents to avoid suboptimal decision-making processes [30, 31, 44]. From a global perspective, although some countries have initiated national protocols regarding developing, appraising, and implementing high-quality CPG, standardized approaches for their production are still lacking [43]. This may partially justify the heterogeneity among PAH guidelines regarding the evidence-based methodological approaches used to build the reports. Although a panel of experts and GRADE collaboration tools are widely recommended strategies, some guidelines use alternative approaches, which may be the cause of low results due to poor external validity and reproducibility of findings. Conversely, we observed a slight improvement in the quality of recommendations reported by CPG published after 2018, which may be due to greater use of the AGREE and GRADE instruments as a form of critical assessment of evidence and more careful elaboration of clinical guidelines.

Like Seow et al. [45] analysis of insomnia guidelines, we reported that the most productive regions of PAH guidelines were North America, Asia, and Europe, following the global geographical distribution of researchers, available technologies, science funding, and international collaboration [46]. However, although documents from Europe and some from North America scored slightly better in the AGREE II and AGREE REX tools, no significant differences were found in the overall quality of these guidelines. Likewise, guidelines’ scores did not differ according to the type of developer (society or government), which is consistent with the findings by De Melo et al. [41], who reported higher quality documents from professional societies and government institutions compared to independent researchers/universities. This may occur due to the different financial and human resources available in these organizations, especially considering CPG development is a time-consuming and costly process, as well as their scope and purposes.

Regarding the apparent lack of relationship between recommendations on PAH target therapy and AGREE II and AGREE REX quality, it is important to consider that there is no conclusive evidence comparing monotherapy and combination therapy or classes and drugs within classes. Therefore, the expert’s opinion is decisive on this matter, and thus, the influence of one guideline on another is notable, regardless of its quality.

Overall, future studies should prioritize methodological rigor (e.g., by means of systematic evaluations of existing guidelines to highlight areas where evidence is scarce or of poor quality), stakeholders’ engagement (e.g., qualitative research with peers, patients, caregivers, healthcare providers, and policymakers on their values and preferences can provide valuable insights regarding the real-world needs and challenges in PAH management), and clinical applicability and implementability (e.g., quality improvement initiatives including assessing barriers and facilitators aimed at testing the cost-effectiveness, feasibility, and adherence of treatment recommendations in clinical settings) [47, 48]. Moreover, although creating and updating high-quality guidelines every 2–5 years, as recommended, may be a costly process, especially for a small national professional society or governmental body, further strategies, including international consortiums, gathering external funding, and adaptations of high-quality published documents, are considered efficient. The use of evidence-to-decision frameworks can also provide better documents with more credible and transparent recommendations [1, 48].

Our review has some limitations. We only included freely available guidelines published after 2012, the year of the last update of the PAH treatment algorithm, which addressed pharmacological treatments; thus, results and conclusions cannot be generalized to other conditions. Nonetheless, a systematic and replicable methodology was performed following international recommendations and using four databases. Given the high heterogeneity among guidelines (i.e., year of publication, countries, target populations, and different methods to assess the level/certainty of evidence and propose recommendations), comparisons between guideline content should be avoided [27]. AGREE tools rely on inherent subjectivity when scoring their domains and may not fully account for the level of evidence attributed to recommendations, which can be subject to debate (e.g., perceptions of the tools and their measurement properties). Nonetheless, these instruments provide important frameworks to constantly evaluate and critically assess clinical practice guidelines. Although overall agreement across reviewers was considered substantial in our study, the evaluation of some items was arbitrary, leading to low consensus rates; however, this may reflect the opinions of a multidisciplinary team with different clinical experiences.