Current off-field concussion screening instruments have sub-optimal accuracy and additional testing domains may be necessary to detect the full spectrum of concussion presentations. This study aimed to determine if additional cognitive tests add utility to off-field screening for sport-related concussion.

Reproducibility and diagnostic accuracy cohort studies were performed in the 2017 and 2018 seasons of the Super Rugby competition, conducted in Argentina, Australia, Japan, New Zealand, and South Africa.

Abridged versions of Stroop (score, time), Spatial Memory (score, failed trials), and Trail Making Trial-B (time, errors) cognitive tests, modified for off-field use, were examined. Players performed baseline testing prior to each season. Cases undergoing off-field screening as part of the World Rugby Head Injury Assessment Process underwent evaluation with the same cognitive tests during competition matches. Agreement between repeated pre-season baseline measurements, and the diagnostic accuracy of off-field testing against a clinical reference standard of concussion, were evaluated.

Data were available for repeated preseason baseline testing in 644 players, and 100 cases undergoing off-field concussion screening. There was little individual agreement across pre-season baseline assessments for all tests (Lin’s correlation and Gwets AC1 coefficients ranging between 0.2 and 0.3). There was significantly worse performance for the time taken to complete the modified Stroop Test in concussed players undergoing off-field screening, compared to non-concussed players (median time 21.1 v 18.4 seconds, p<0.01; area under the receiver operating characteristic curve 0.7 (AUROC, 95%CI 0.5-0.8). Other cognitive measures did not discriminate between injured and un-injured players, with no-statistically significant differences in distribution medians (p=0.6-0.9) and AUROC values close to 0.5.

The time taken to perform a modified Stroop Test may provide additional diagnostic accuracy if added to current off-field concussion screening tools. Abridged Spatial Memory and Trail Making Trial-B tests did not discriminate between concussed and non-concussed players.