Introduction

Periprosthetic joint infection (PJI) is one of the most devastating complications after total knee arthroplasty (TKA) or total hip arthroplasty (THA). Though a rare complication, PJI is associated with substantial morbidity, mortality, and high economic costs1. Published evidence suggests the incidence of PJI is approximately 1.55% within two years after TKA and 0.45% within two to ten years after TKA2; the incidence of PJI after THA surgery is about 1%3, 4. PJI is also a leading cause of revision after knee and hip arthroplasty5, 6. Although there exist many methods for the diagnosis of PJI, final diagnosis and evaluation of PJI in clinical practice is still demanding7, 8. On the one hand, the detection rate of culture, which is the primary mean to identify the pathogen involved, is relatively low. Li et al. demonstrated that sensitivity of periprosthetic tissue culture in blood culture bottles was 70%9. Gallo et al. showed that the positive results of joint fluid culture were only 44% PJI patients10. On the other hand, there is no one recognized approach that is able to replace the culture for identifying the pathogen. Therefore, finding a new effective detection method to improve the detection rate of pathogenic microorganisms is becoming urgently necessary.

With the continuous development and improvement of high throughput sequencing technology, an increasing number of scientists recognize its potential value for identifying the pathogen. Next-generation sequencing (NGS), a DNA sequencing technology that has revolutionized genomic research, has developed rapidly over the last decade11. Next-generation sequencing has shown good value in identifying pathogens and diagnosing many infectious diseases12, 13. In the last five years, there has been emerging evidence on the potential value of NGS in diagnosing PJI14-20. A few studies have explored the diagnostic value of NGS for PJI14-17, but the evidence is inconsistent and uncertain. In a recently published study16, NGS had limited value in the diagnosis of PJI, a finding which was inconsistent with previous studies14. Some studies have also suggested that NGS has excellent diagnostic value for patients with negative microbial culture14, 15, 17, which is not at par with that of Kildow et al.16. The most challenging issue in the diagnosis of PJI is patients with negative microbial culture21, 22. Whether NGS can accurately identify culture-negative PJI patients is uncertain. Besides, considering the difficulties in diagnosis for patients with antibiotic administration history, it is also worth exploring whether the NGS could play an important role in diagnosis.

In this context, we conducted a systematic review of the published literature to summarize: (i) the diagnostic accuracy of NGS for PJI; (ii) the detection rate of NGS for culture-negative PJI; and (iii) the detection rate of NGS for PJIs with antibiotic administration history.

Materials and Methods Data Sources and Search Strategy

We performed this review following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines23. The systematic literature search was performed by two independent authors, with a third author resolving disagreements where necessary. The electronic databases Medline (PubMed), Embase (OvidSP), and Cochrane Central Register of Controlled Trials were searched for relevant studies published from January 1990 to March 2021. The literature search strategies for these three databases followed medical subject headings combination with terms. The detailed literature search strategies are reported in the Table S1. Also, unpublished and gray literature was sought and retrieved in established journals of the orthopedic field (such as, The Journal of Bone and Joint Surgery, The Journal of Arthroplasty, Clinical Orthopaedics and Related Research, or International Orthopaedics) from January 1990 to March 2021.

Inclusion and Exclusion Criteria

Two authors independently assessed the search results for inclusion in this systematic review by initially scanning titles/abstracts and conducting full-text evaluation of potentially eligible studies. Any disagreements between the two authors were resolved by consensus or through discussion with a third author.

The studies which evaluated the diagnostic values of the NGS for identifying PJI in patients were included. The inclusion criteria were: participants: patients suspected of having PJI; interventions: not applicable; comparisons: not applicable; outcomes: diagnostic values of the NGS for identifying knee or hip PJI, and patients were categorized as infected or aseptic using the Musculoskeletal Infection Society (MSIS) criteria, Infectious Diseases Society of America (IDSA) PJI diagnostic criteria or International Consensus Meeting (ICM) criteria as the reference standard24-26; study design: cohort study, cross-sectional study, or case–control study (prospective or retrospective). The exclusion criteria are: case reports, commentaries, expert opinion, and narrative reviews; and non-English language publications. A PRISMA flow diagram of the literature screening process was constructed after study selection. The detailed results are shown in Fig. 1.

Flow diagram of study selection process. NGS, next-generation sequencing.

Data Extraction

We imported all the retrieved articles into EndNote (version X9, Clarivate Analytics, Philadelphia, PA, USA). After identifying and excluding duplicate records, ineligible articles, and those published before 1990, the two researchers independently conducted literature screening and extracted data based on the inclusion and exclusion criteria. A third investigator resolved any disagreements.

We extracted the following information: study information (author, year of publication, country, institution, journal, type of study, etc.); study population baseline information (age, gender, body mass index (BMI), etc.); the number of PJI and non-PJI patients diagnosed with MSIS in each study; the number of NGS-positive, NGS-negative, culture-positive or culture-negative patients in both PJI and non-PJI groups; the number of NGS-positive or NGS-negative in culture-negative PJI patients for each study; and the number of NGS-positive or NGS-negative PJI patients who had an antibiotic administration history for each study. Literature Quality Evaluation

The methodological quality of included studies was appraised using the QUADAS (Quality Assessment of Diagnostic Accuracy Studies)-2 tool27. The QUADAS-2 tool evaluates bias based on the following four domains: patient selection, index test, reference standard, and flow and timing. The risk of bias was assessed in each domain, and concerns about applicability were assessed in the first three domains with signaling questions. These questions were answered with “yes” for a low risk of bias/concerns, “no” for a high risk of bias/concerns, or “unclear” when the relevant information was not clearly provided. Two authors independently evaluated these studies. A third author resolved any controversy to achieve a final consensus.

Statistical Analyses

A standardized Microsoft Excel spreadsheet was used to track the extraction of quantitative data from each study. True positive (TP), false positive (FP), true negative (TN) and false negative (FN) results were collected and plotted in a two-by-two contingency table. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (+LR) and negative likelihood ratio (−LR) were calculated for each study. Simultaneously, the detection rate of NGS for culture-negative PJI patients, which was equal to NGS-positive / culture-negative PJI patients expressed as a proportion of the total number of culture-negative PJI patients, was also calculated. Besides, the detection rate of NGS for PJI patients with antibiotic administration history would be calculated, which was equal to NGS-positive/PJI patients with antibiotic administration history. The statistical analyses were performed by two researchers independently; a third investigator resolved any disagreements.

Results Study Selection

We retrieved a total of 87 potentially relevant citations from the four electronic databases. After excluding 29 duplicates and 38 irrelevant citations based on titles and abstracts, 20 citations remained for full-text evaluation. Following this, we further excluded 11 articles because: (i) study designs did not meet the inclusion criteria (two reviews, one case report, and one editorial)18, 28-30; (ii) one study was related to therapeutics31; (iii) the population or sample included in three studies was not relevant20, 32, 33; (iv) the index test in one study did not include NGS34; (v) one study did not use the MSIS criteria, IDSA criteria, or ICM criteria as a reference standard; and (vi) one study involved the establishment of a PJI diagnostic model in which the data relating to NGS diagnosis could not be obtained19, 35. Nine articles were finally eligible for the review14-17, 36-40. There was no disagreement between the reviewers regarding the inclusion of these studies.

Study Characteristics and Quality

The nine eligible studies, which evaluated the diagnostic value of the NGS, were published between 2018 and 2021. Four studies14, 15, 36, 39 were conducted in the same institution, and another two studies37, 38 were also completed at the same institution. All included studies recruited a total of 1007 patients (541 PJI patients and 466 non-PJI patients). Six studies enrolled patients who underwent revision arthroplasty14, 15, 17, 36, 39, 40; two studies included the patients who were PJI or aseptic implant failures37, 38; one study enrolled patients who underwent revision arthroplasty or primary arthroplasty16. The mean ages ranged from 62.5 to 67.8 years. The percentage of women ranged from 35.4% to 51.7%. Five studies reported on patients' mean BMI, which ranged from 27.2 to 33.05 kg/m2 16, 17, 36, 37, 40. Seven included studies were cohort studies14-17, 36, 39, 40, and two studies were case–control studies37, 38. Meanwhile, six studies were prospective research14, 15, 17, 36, 39, 40 and three studies were retrospective research16, 37, 38. MSIS criteria were selected as the reference standard in seven studies14-17, 36, 39, 40, while two studies employed IDSA as PJI diagnostic criteria37, 38. The nine studies employed different sampling methods. Only synovial fluid was sampled in five studies15, 16, 36, 37, 40; only periprosthetic tissues were sampled in one study14; only sonicate fluid was sampled in one study38; synovial fluid, deep-tissue specimens, and swabs were sampled in one study17; and synovial fluid, sonication fluid, or homogenized tissue were sampled in one study39. For the studies which used synovial fluid for the NGS test, the minimal volume of synovial fluid was 0.6 mL, while the maximum volume was greater than 4 mL. Table 1 presents details of study characteristics.

TABLE 1. Characteristics of studies included in systematic review Study Institution Population Study design Prospective or retrospective Reference standard Sample for NGS Volume of synovial fluid (mL) Patients (All [PJI/Non-PJI]) Age (year, Mean [SD]) Sex (Female [%]) BMI (kg/m2, Mean [SD])

Cai et al.14

2020

First Affiliated Hospital of Fujian Medical University, Fuzhou, Fujian, China Revision arthroplasty Cohort Study Prospective MSIS Intraoperative (periprosthetic tissues) \ 44 (22/22) 62.50 (9.40) 38.64 NR

Fang et al.15

2020

First Affiliated Hospital of Fujian Medical University, Fuzhou, Fujian, China Revision arthroplasty Cohort Study Prospective MSIS Intraoperative and Preoperative (synovial fluid) 1 38 (25/13) 62.64 (19.68) 50 NR

Huang et al.36

2020

First Affiliated Hospital of Fujian Medical University, Fuzhou, Fujian, China Revision arthroplasty Cohort Study Prospective MSIS Intraoperative (synovial fluid) 1 70 (49/21) 65.67 (13.26) 44.29 27.81 (4.78)

Ivy et al.37

2018

Mayo Clinic, Rochester, Minnesota, USA Aseptic failure and PJI Case–control study Retrospective IDSA Preoperative (synovial fluid) 1 168 (96/72) 67.03 (12.21) 41.67 33.05 (10.02)

Kildow et al.16

2021

Duke University, Durham, North Carolina, USA Revision arthroplasty and primary arthroplasty Cohort study Retrospective MSIS Preoperative (synovial fluid) ≥2 116 (48/68) 67.80 (13.85) 51.7 31.70 (7.78)

Tarabichi et al17

2018

The Rothman Institute at Thomas Jefferson University, Philadelphia, Pennsylvania, USA Revision arthroplasty Cohort study Prospective MSIS Intraoperative (synovial fluid, deep-tissue specimens and swabs) NR 65 (28/37) 64.10 (10.73) 35.38 31.18 (6.62)

Thoendel et al.38

2018

Mayo Clinic, Rochester, Minnesota, USA Aseptic failure and PJI Case–control study Retrospective IDSA Intraoperative (sonicate fluid) \ 408 (213/195) 65.31 (16.72) 50 NR

Wang et al.39

2020

First Affiliated Hospital of Fujian Medical University, Fuzhou, Fujian, China Revision arthroplasty Cohort study Prospective MSIS Preoperative (synovial fluid, sonication fluid or homogenized tissue) ≥4 63 (45/18) NR NR NR

Yin et al.40

2021

Liaocheng People's Hospital and Liaocheng Clinical School of Taishan Medical University, Liaocheng, Shandong, China Revision arthroplasty Cohort study Prospective MSIS Preoperative (synovial fluid) 0.6 35 (15/20) 67.80 (7.40) 40 27.20 (2.30) BMI, body mass index; IDSA, Infectious Diseases Society of America criteria; MSIS, Musculoskeletal Infection Society criteria; NR, not reported; PJI, Periprosthetic Joint Infection.

The risk of bias was assessed for all included studies using theQUADAS-2 tool27. All eligible studies were at high risk of bias. The most common reason for a high risk of bias was the reference standard, where MSIS criteria or IDSA criteria were used as the reference standard. Studies were at high risk for the “patient selection” domain because patients were not enrolled consecutively14-16, 37, 38, 40. The high risk of bias for the “index test” was because the results of the reference test were definitive before the index test37, 38. Also, the high risk of bias for “flowing and timing” was because not all patients were included in the analysis in three studies16, 37, 38; one study enrolled acute PJI patients, which led to an inappropriate interval between index test and reference standard16; there was a long interval between index test and reference standard in two studies37, 38. The results are shown in Fig. 2.

Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) scores for risk of bias and concerns regarding applicability of the included studies.

The Diagnostic Value of Next-Generation Sequencing for PJI

All included studies reported the diagnostic performance of NGS for PJI. The diagnostic sensitivities and specificities ranged from 63% to 96% and 73% to 100%, respectively. The positive and negative predictive values ranged from 71% to 100% and 74% to 95%, respectively. The positive and negative likelihood ratios ranged from 3.30 to 20.56 and 0.02 to 0.42. The detailed results are reported in Table 2.

TABLE 2. Sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio and negative likelihood ratio of next-generation sequencing for periprosthetic joint infection Study Sampling TP FP FN TN Sensitivity (%) Specificity (%) PPV (%) NPV (%) +LR −LR Cai et al. 202014 Intraoperative 21 2 1 20 95 91 91 95 10.50 0.05 Fang et al. 202015 Intraoperative 24 0 1 13 96 100 100 93 N.S 0.04 Preoperative 23 1 2 12 92 92 96 86 11.96 0.09 Huang et al. 202036 Intraoperative 47 1 2 20 96 95 98 91 20.56 0.02 Ivy et al. 201837 Preoperative 73 5 23 67 76 93 94 74 10.95 0.26 Kildow et al. 202116 Preoperative 30 7 18 61 63 90 81 77 6.07 0.42 Tarabichi et al. 201818 Intraoperative 25 10 3 27 89 73 71 90 3.30 0.15 Thoendel et al. 201838 Intraoperative 157 7 56 188 74 96 96 77 20.53 0.27 Wang et al. 202039 Preoperative 43 1 2