4.1 Summary of Key Results

This analysis highlights statistically significant differences in the characteristics of the case series of DME and non-DME signals as they appeared in VigiBase up to the year of communication. These were: the numbers of reports and their average completeness and the counts of positive dechallenges and rechallenges. Furthermore, except for a statistically significant difference in the interval in years between the first indication of disproportionality and the year of communication, the timing of communication did not differ between the two groups of signals. Finally, we found statistically significant patterns shared between DME and non-DME signals, such as the strong association between TTC and both completeness score and level of evidence, and how the time after launch of medicinal products relative to communication increased over the last 15 years of the study period.

4.2 Statistical Significance and Practical Relevance

While the comparisons of DME and non-DME signals were statistically significant, they were accompanied by small effect sizes in Brunner–Munzel estimates for numbers of reports, dechallenges, rechallenges and average completeness scores. This was especially surprising in relation to the difference in numbers of reports, as one might expect a DME signal in some cases to be based on as few as one report (i.e. ‘between-the-eyes’ adverse reactions [13]). However, DMEs tended to be supported by a number of reports exceeding by several times the (canonical) minimum of three required for signal detection [14]. Less stringent criteria for signal detection when fatal events are involved have been previously advocated [15], and the same may be extended to DMEs. The EMA states that member states use the categorisation of a range of adverse events as DMEs to focus on reports of suspected adverse reactions that deserve special attention. However, we could not find evidence of such an effect in our analysis, and prioritization of such signals may require further attention. An important consideration is that the size of case series in VigiBase may have been larger than those on which the communicated signals were based. In view of this, it may be helpful for pharmacovigilance stakeholders to consult global databases of case reports when a signal of DME is detected to ensure more data are available for its clinical assessment.

4.3 Relationship Between Strength of Evidence and TTC

Irrespective of categorization into DME or non-DME signals, we found not only statistical significance but also larger effect sizes in the association between the TTC and the strength of evidence. Whether in the form of higher quality evidence (i.e. OCEBM 1–3) or high average completeness of the information in an underlying case series (i.e. ‘well documented’), the strength of evidence appeared to be linked to an up to fivefold shorter TTC (Fig. 2 and Table 2).

4.3.1 Relationship Between OCEBM Level and TTC

A possible contributor to the observed relationship between OCEBM level and TTC may be that evidence of higher quality (OCEBM 1–3) tends to be collected and appraised in pre-approval stages, as evidenced, in part, by the negative intervals between launch and communication (Fig. 3b). Conversely, evidence of lower quality (OCEBM 4) begins to accrue later, during post-marketing; in this phase, signals are detected mainly through reports of ADRs and are continuously prioritized as per good vigilance practices through analyses of patient exposure and estimates of frequencies of ADRs [16]. Limitations inherent to the systems for collecting reports of ADR, such as under-reporting or low completeness of the reports, may have further contributed to the relationships observed in Table 2. Nevertheless, the types and frequencies of ADRs detected through pre- and post-marketing are different, the latter phase being primarily concerned with rare ADRs.

4.3.2 Association Between Completeness of Information in a Case Series and TTC

Well-documented reports have been associated with ‘certain’, ‘probable’ or ‘possible’ outcomes of causality assessments or with reports flagged as serious by international standards [17, 18]. We should stress that completeness of reports is accounted for in methods for signal detection, such as vigiRank [19], which increase the rate of detected signals compared with disproportionality analysis [20], but does not necessarily have a bearing on the timeliness of signal detection from disproportionality analyses [21]. Rather, completeness of reports has been regarded as useful in performing clinical reviews [14], which also constitute the main type of evidence underpinning signals [5]. We reiterate that although the difference in median completeness score between DME and non-DME signals was statistically significant, the effect size was small and we did not record a difference in TTC for DME and non-DME signals supported by well-documented reports. These results may call for improved international collaboration between regulators and reporters, with the aim of increasing the completeness of information in reports of suspected ADRs, as means of facilitating clinical reviews and expediting the TTCs of both DME and non-DME signals. The matter of completeness becomes more relevant when one appreciates that the volume of reports in databases has increased substantially over the past 30 years [22] and may continue to do so as developing countries progress towards more mature pharmacovigilance systems [23, 24]. Since a high degree of completeness is not always achievable [25,26,27] and completeness may vary across settings or attitudes of health carers and patients towards reporting [28], any intervention geared towards increasing the completeness of reports would probably be a complex one [29].

4.4 Increase in TTC Over Time

Our observations support prior research showing a growing proportion of signals supported by medicinal products launched over 10 years before communication. Early reviews of signals of the Pharmacovigilance Risk Assessment Committee suggested that the concerned medicinal products were on the market for a median of 12 years and 42% of them for less than 10 years [30]. In 2010, the median time on market of medicinal products involved in regulatory actions in the USA was 11 years [31]. Thus, on one hand, the increase in TTC may reflect evolving pharmacovigilance systems, able to manage signals concerning medicinal products that have been on the market for several decades, as noted in [30], namely: improved monitoring, completion of long-term observational studies to evaluate suspected harms or changes in patterns of use of medicinal products. On the other hand, it is worth considering that some adverse effects may be only detected with enough length of exposure; indeed, medicinal products that require longer durations of exposure have been found to be associated with larger numbers of amendments to product information [32]. In addition, the amount of post-approval exposure data (rather than pre-approval) predicts changes to the sections of untoward effects, and warnings and precautions, in European summaries of products characteristics (SmPCs) [33]. Taking these insights together, the increase in TTC may be conditional on the time needed to accrue sufficient data in the postmarketing phase, a time that may have been longer for some classes of medicinal products.

4.5 Strengths and Limitations

We compared large sets of DME and non-DME signals, relying on systematically collected data covering roughly 30 years. We used a heteroskedasticity-robust statistical method to compare the two groups of signals, ensuring intuitive interpretability of the results, excluding signals that may have distorted calculation of the intervals we had postulated. We are not aware of similar published work. These findings may provide a way forward for regulators and researchers in prioritising and communicating signals of rare events that are typically associated with medicines.

As this study concerned any reported signals/SDRs, irrespective of regulatory requirements for action or verificatory studies, our findings are relevant to the communication of signals alone. In other words, any differences or lack thereof we have identified may not necessarily solely concern signals that have significant effects on public health [16].

We used the list of DMEs rather than the list of important medical events (IMEs), since the latter includes far more events that may cause a report to be marked as ‘serious’ by international standards [34]. Both lists, however, presuppose seriousness; we chose to use DMEs, as they are regarded as often drug related. Relatedly, we did not quantify proportions of serious reports in either group of signals, so we cannot conclude whether DME signals were supported mostly (or not) by reports marked as non-serious.

Findings about TTC should be considered carefully. Although we have manually verified dates of receipt of reports (in VigiBase or at the national centres level) that were discrepant with launch years, we could not control for potential data entry errors in VigiBase. In addition, data retrieval was based on the definition of the events in the original publications; in the case of composite events, we retrieved all relevant MedDRA preferred terms. Consequently, frequently reported events may have biased the TTC of some of the most recently communicated signals/SDRs.

In our search for launch years, we have encountered minor mistakes in the available data sources (and have reported them to the data holders). For medicinal products launched in countries that no longer exist (e.g. Eastern Germany since 1978), some dates may have been replaced by default values by regulatory agencies, but we did not encounter enough examples (four in all) to warrant concern. Furthermore, we did not systematically evaluate any discrepancies between the sources we used to obtain launch years and the published literature, so there may be instances in which some medicinal products may have been launched earlier than recorded. More important is the effect of censoring, which may not have allowed sufficient time for an ADR to be recognized by reporters for medicinal products launched in recent years.

The method used to compute the completeness of a case series measures technical completeness but not clinical utility. In other words, formally complete case reports may still not necessarily contain sufficient information to produce a clinically sound judgment on a possible causal relationship between a medicinal product and an adverse event. It may well be that signals that were communicated rapidly contained a higher degree of clinically relevant information, which we could not measure.