The DILIN was established by the National Institutes of Health (NIH) in 2003 to study drug-induced liver injury (DILI). The DILIN’s Prospective Study enrolls patients with suspected DILI within 6 months of onset. Patients eligible for enrollment meet predefined laboratory criteria of serum aspartate aminotransferase (AST) or alanine aminotransferase (ALT) > 5 times or serum alkaline phosphatase (ALP) concentration > 2 times the upper limit of normal (or baseline before exposure) on two consecutive occasions at least 24 h apart. Those with an unexplained total serum bilirubin of greater than 2.5 mg/dL or an INR above 1.5 after exposure are eligible as well. A more in-depth review of eligibility, evaluation, and enrollment in the DILIN Prospective Study has been described in a previous publication [5].

The determination of attribution in the DILIN is based on a structured causality assessment approach complemented by consensus expert opinion. Three causality committee members, with expertise in DILI and hepatology (the clinical site principal investigator and two others), receive key clinical, laboratory, and diagnostic data abstracted from the DILIN baseline visit and clinical narrative [5]. Cases are graded as definite (> 95% likelihood), highly likely (75–95%), probable (50–74%), possible (25–49%), or unlikely (< 25%), reflecting the likelihood of liver injury being attributed to DILI (Table 1). In cases where more than one agent is implicated, each medication or product is scored separately for the likelihood that it was responsible for the injury. When the DILIN causality scores of the three assigned reviewers do not agree, the case is discussed by the entire committee on a teleconference wherein a final consensus score is assigned [5].

Since its inception through December 2020, the DILIN enrolled 2203 patients, with 23% of cases with implicated in HDS [6]. Since 2015, whenever available, implicated products are collected from patients and submitted for chemical analysis to the National Center for Natural Products Research at the University of Mississippi (NCNPR), which uses high performance liquid chromatography with mass spectroscopy to chemically analyze and catalog their contents. Specifically, the NCNPR assays for the presence of known hepatotoxins, including Camellia sinensis (also known as green tea) leaf (catechin, epicatechin, epigallocatechin, gallocatechin, epicatechin gallate, epigallocatechin-3-gallate or EGCG, caffeine, theobromine, theophylline), Garcinia cambogia fruit (hydroxycitric acid or HCA, garcinialactone, citric acid), Polygonum multiflorum root (resveratrol, emodin, aloe-emodin, physcion, polydatin, tetrahydroxystilbene or TSG), Scutellaria baicalensis (also known as skullcap) aerial parts (baicalein, wogonin, wogonoside, baicalin), Mitragyna speciosa leaf or kratom (mitragynine, speciogynine, paynantheine, corynoxine, 7-hydroxy mitragynine, mitrafoline, corynantheidine, speciofoline, mitragynalinic acid), Withania somnifera (also known as ashwagandha) root (withaferin A, withanolide D, withanoside IV, withanone, withanolide A, withanolide B, withanoside V), Curcuma longa (also known as turmeric) rhizome (curcumin, demethoxycurcumin, bisdemethoxycurcumin), anthraquinones (sennoside A-B, emodin, aloe-emodin, rhein, chrysophanol, cascarin, catenarin, danthron, cascarosides A-F, 1,8-dihydroxyanthraquinone, 2-aminoanthraquinone, 1-amino-2,4-dibromoanthraquinone,emodin-8-O-β-D-glucopyranoside,chrysophanol-8-O-β-D glucopyranoside), and aloe-emodin-8-O-β-D-glucopyranoside and non‐botanical (anabolic steroids, pharmaceuticals) compounds. For anabolic steroids and pharmaceuticals, the analyzed supplements were screened using Agilent MassHunter Forensics and Toxicology (9203 compounds) Personal Compound Database (PCD) kit with accurate mass measurements. Searching the PCD library helps to identify the compounds found by matching their product ion mass spectra and fragment ions. During this screening process, if any of these compounds were detected for anabolic steroids or pharmaceuticals, they were further confirmed using the reference standards.

The HDS are categorized on the basis of their marketed purpose for use, such as weight loss, performance enhancement, bodybuilding, and general wellness. Products collected from patients enrolled in the DILIN are assayed to assess whether the contents found reflect the labeled ingredients, when an actual or online list of purported ingredients label exists.

For this study, cases were included in which HDS was suspected as the cause of liver injury and originally scored as definite (> 95%); highly likely (75–94%); probable (50–74%); possible (25–49%); and unlikely (< 25%) DILI and for which product was collected from the patient for chemical analysis. Reviewers were blinded to original causality scores performed when the chemical analysis data were not available. There were 54 cases enrolled between 2004 and December 2020 that met these criteria (Fig. 1).

Flowchart of DILIN cases included in analysis

The initial step in our approach was to repeat the standard DILIN structured causality assessment approach with expert opinion from a panel of DILIN investigators (Table 1). The purpose of repeating the original causality assessment process was to create contemporaneous baseline scores for cases reviewed by the same reviewers involved in the second step of our study. Three reviewers were assigned to each case. An overall score that reflected the likelihood that the case represented hepatotoxicity due to an agent or agents was assigned.

In the second step, the 54 cases underwent a modified DILIN structured causality assessment approach, in which chemical analysis data were provided during the causality assessment process. Three reviewers were again randomly assigned to the cases for the second step review. The DILIN Data Coordinating Center (DCC) at Duke University was responsible for the random assignment of cases to reviewers, the blinding of reviewers to the original causality assessment scores, and the presentation of chemical analysis data for each case. The chemical analysis data reflected (1) whether the analysis confirmed the labeled contents and (2) the presence of any potential hepatotoxin, as listed above, or pharmaceutical. As per the DILIN’s protocol, cases were circulated to reviewers for independent assessment and score assignment. Scores were then reconciled by email, or when necessary, by conference call and majority vote.

The original overall causality scores (step 1) were compared with the overall causality scores resulting from causality assessment that incorporated chemical analysis (step 2). The data report the number and percentage of cases with the likelihood of DILI moving higher, lower, or remaining the same. To compare the causality scores in the first review versus the causality scores in the second review, we examined the data in a 5 × 5 contingency table cross-classified by the five causality categories from the first and second reviews. We tested whether there was a statistically significant difference in the marginal distributions of this contingency table. Specifically, we test whether the five-category causality distribution in the first review is the same as the five-category causality distribution in the second review. A chi-squared test with four degrees of freedom based on a weighted least square approach was used to test the equality of the two marginal distributions. This test was carried out by using the SAS procedure CATMOD with the weighted least squares approach to account for correlation due to the same subjects were scored twice. All analyses were carried out in SAS version 9.4 and a p-value less than or equal to 0.05 is considered statistically significant.