The objective of this study was to explore factors and signs associated with clinicians’ certainty of an Autistic Disorder diagnosis to gain insight into the implicit knowledge that influences a clinician’s interpretation of diagnostic criteria and clinical decision making in general.

Certainty and ADOS items

As expected from previous research [9,10,11,12], we found a modest correlation between diagnostic certainty and autism symptomatology, confirming that a substantial fraction of participants with a relatively low ADOS score were diagnosed with the highest certainty, whereas, some participants did not receive the highest certainty rating despite having relatively high ADOS scores. As mentioned previously, the modest correlation may be explained by different ADOS items having different associations with certainty such that some items contributing to the total score have little association with certainty. By investigating the associations between individual ADOS items and certainty, we confirmed that certain autism signs markedly increased the odds of being diagnosed with Autistic Disorder with the highest certainty, whereas some signs showed only minor associations with certainty, and others even showed a trend towards a negative association. This finding could suggest that particular signs have a stronger impact on how certain clinicians are in their diagnostic decision, likely reflecting that these characteristics are consistent with how the clinicians expect Autistic Disorder to appear. More items from the communication domain were significantly associated with certainty in ADOS modules corresponding to a higher level of language ability. This makes intuitive sense, as lower language ability in itself was found to be strongly associated with higher diagnostic certainty. In these individuals, the qualitative characteristics of language use are likely less important, whereas in individuals with more developed language abilities, qualitative atypicalities may have a larger influence on certainty.

The observation that some ADOS items are more associated with certainty than others may suggest that new ways of constructing assessment instruments could be investigated in the future to improve the specificity of the recognition-definition-investigation cycle [19]. Additionally, scores on instruments such as the ADOS are traditionally based on an equal weighting of all or some items [28] meaning that each included item equally contributes to the severity score. Some scoring algorithms (e.g., the calibrated severity score) that only include a select subset of all items have been found to identify autism with higher specificity [29]. However, given that different items may have different associations with recognizable manifestations of autism, it is also worth considering alternative algorithms with differential weighting of items. Furthermore, it is still an open question as to whether there are interactions between different signs which could improve discrimination; for example, the presence of two items together may have a higher weight than the sum of each item presented separately.

Such considerations may be particularly relevant in relation to the specificity of an instrument since individuals with other conditions may display a substantial number of signs that may also be associated with autism. For example, Havdahl and colleagues [30] found that the presence of behavioural or emotional problems, as well as low IQ, had a marked influence on the discriminatory threshold of many commonly used diagnostic tools such as the ADOS, suggesting issues with specificity in a complex clinical setting. It would be informative to further explore which items, individual or combined, may be solely associated with autism and which items are also commonly observed in individuals with other conditions such as ADHD or intellectual disability.

Another possible explanation for the modest correlation between the ADOS total score and diagnostic certainty is that some clinicians may score ADOS items as present based on a range of qualitative expressions of a given sign [3], whereas only some of these expressions are recognized as autistic with high certainty. The distinction between different qualitative presentations is likely learned with experience and future research might investigate the association between qualitative variations in signs and diagnostic certainty.

Correlations between certainty, head circumference, and IQ ratio

We found that individuals diagnosed with the highest certainty had a significantly larger normalized HC than those with lower certainty ratings for all three ADOS modules. Furthermore, 85% of individuals with the largest normalized HC, i.e., individuals within the top 2.5th percentile, were rated with the highest certainty versus 64% of individuals not meeting this criterion. This could indicate that either merely presenting with a larger head than commonly expected or having characteristics that are associated with having a larger HC in the autism population may influence the certainty of the clinician. Exploring associations between the normalized HC and other variables revealed small, but significant positive correlations with several items in the ADOS. Interestingly, certain items overlapped between modules; for example, Shared Enjoyment in Interaction across all three modules, as well as Imagination/Creativity and Reciprocal Social Communication in modules 2 and 3. Most of the significant correlations between HC and ADOS items were within the social interaction, play behavior, and communication domains. In addition, many of the ADOS items in modules 2 and 3 that correlated with HC were also associated with an increased likelihood of having the highest diagnostic certainty. For example in module 2, Shared Enjoyment in Interaction, Reciprocal Social Communication, Amount of Social Overtures, and Showing, which are all from the area of social communication and interaction, were significantly associated with certainty and were also found to be associated with HC. Although previous results on an association between autism symptom presentation and HC have been inconsistent, some studies indicate that particularly social symptoms may be associated with macrocephaly in autistic individuals [31, 32] while others link it to non-social atypicalities [33]. The largest correlation of HC was with the verbal to non-verbal ratio (r = -0.17), but only in module 2. Deutsch and Joseph [34] found a similar association between macrocephaly and verbal to nonverbal discrepancy in 2003 although with a larger correlation coefficient (r = -0.35). Interestingly, Joseph and colleagues [35] found that school age children with an IQ profile of higher non-verbal than verbal IQ had significantly higher autism symptomatology scores within the social interaction domain. Given the associations between diagnostic certainty, HC, social symptoms, and a low verbal/nonverbal IQ ratio, it would therefore be prudent to further explore whether these characteristics are part of a specific autism presentation that is recognized by clinicians with high certainty.

Associations between certainty, language level, and age

Diagnostic certainty was associated with the age at assessment, as well as language level (ADOS module), with a significant interaction. A higher percentage of autistic children received the highest certainty rating when assessed with ADOS module 1 than those evaluated with modules 2 and 3, but the difference decreased with age. For those assessed with module 1 (no phrase speech), the percentage of high certainty was high regardless of age. For those assessed with module 2 (phrase but not fluent speech), diagnostic certainty was lower for children evaluated around three and six years old compared to children in age equivalent groups who were assessed with module 1. Interestingly, the percentage appeared to gradually reach the same high level as for module 1 for the children that are assessed at older ages. This likely reflects the fact that the absence of fluent speech becomes increasingly abnormal with age and, thus, those who are assessed with module 2 at older ages will likely be highly atypical compared to their age equivalent peers. A similar pattern was observed for those assessed with module 3, although the level of certainty was consistently slightly lower than for module 2, reflecting that a young child with highly developed language may be considered less likely to have autistic disorder.

Association between certainty and other variables

We found several significant associations between certainty and IQ, as well as adaptive and externalizing behaviours, although not consistently across ADOS modules. Associations between diagnostic certainty and other variables have been explored in previous studies [8,9,10,11,12]. Negative associations between IQ and diagnostic certainty have been observed previously [9, 10, 12], consistent with our findings for those assessed with ADOS modules 2 and 3. Adaptive behaviour has been found to be negatively associated with certainty in some studies [10, 12] while others have found no association [8]. We found a negative association between certainty and externalizing behaviour among those assessed with ADOS module 3, while no association was found with internalizing behaviour. One previous study using data from the whole autism spectrum in the SSC found weak negative associations with both externalizing and internalizing behaviours [9], whereas another study found a positive association with internalizing behaviour and no association with externalizing behaviour [12]. Generally, the previous studies are difficult to directly compare to our results as they operationalized certainty differently. Some previous studies have considered the certainty of the clinician’s decision regardless of whether the decision was autism or no autism. Thus, those who clearly did not meet the criteria would have had a high certainty along with those who clearly did meet the criteria. Furthermore, previous studies investigated all children meeting the criteria for an autism spectrum diagnosis, whereas we limited our focus to the certainty of meeting the criteria for Autistic Disorder specifically. Certainty for a spectrum diagnosis may cover a broader range of signs, corresponding to the broad range of presentations that can fall within the autism spectrum, whereas certainty for an Autistic Disorder diagnosis may reflect recognition of a less variable presentation. As also mentioned by McDonnell and colleagues [9], sample characteristics may moderate associations between clinical factors and certainty. The fact that we stratified the sample based on language level (ADOS module), which the cited previous studies did not do, thus also makes direct comparison of the results more difficult.

Limitations

The study focused on those diagnosed with Autistic Disorder, hypothesising that these individuals may be part of a subgroup corresponding to a particular prototype. The findings of this study, thus, do not describe certainty in a broader autism spectrum diagnosis. However, even the sample diagnosed specifically with Autistic Disorder contained variation, e.g. in terms of IQ, age at diagnosis, language level, and total ADOS score, and so might display some heterogeneity in terms of the factors that led to a clinician diagnosing them with higher or lower certainty. At the same time, the variation in the certainty rating among those diagnosed with Autistic Disorder was relatively low, with most individuals having certainty ratings close to the maximum value. This may have made it more difficult to detect associations between certainty and other variables. The heterogeneity of the sample is relevant for the interpretation of our results and may also have affected the magnitude of the identified effects. For example, an observed effect could be driven primarily by a smaller part of the population, but be diluted by other parts of the population that may have different mechanistic underpinnings. As such, the findings might be relevant for a small and potentially unknown subgroup, but not for most of the cohort represented in the SSC. Our finding demonstrating the correlation between HC and the verbal/nonverbal IQ ratio was only present in ADOS module 2. This highlights that it may be relevant to consider whether individuals can be stratified based on common features, such as language level or age at autism diagnosis, when investigating a heterogeneous autism population, [18] thereby making it more likely that the individuals have something in common. Analysing the whole population may result in the identification of a very small effect that is difficult to interpret. Phenotypic a priori stratification may decrease noise and make it more likely to identify larger effects that are relevant to the given subpopulation.

The demographic composition of the SSC may indicate a problem with representativeness, which can affect the interpretation of our findings. There was a high percentage of probands from families with a college degree, showing that the population studied had a higher level of education than that generally found in the adult US population [36]. The percentage of non-white groups was also low, particularly in the part of the sample assessed with module 3 that comprised only 2% African Americans. Predisposing factors of autism associated with, for example, race or the level of education may explain some of these discrepancies. However, it could also reflect a selection bias with certain demographic groups having better access to assessment facilities, thus impacting the generalizability of the findings from the SSC.

Clinicians’ diagnostic certainty is a subjective rating, and so it is expected to be associated with some degree of variability. For example, some clinicians may be certain more often than others, and different clinicians may not have the same understanding of what autism looks like depending on their clinical expertise and exposure to autism. Two clinicians thus may not report the same certainty rating if they were both to assess a given individual. Such differences in how certainty is rated introduce noise and would tend to decrease the size of the observable correlations between the certainty variable and the characteristics of the autistic individuals. Thus, our results likely do not show a universal pattern of how certainty correlates with clinical factors for every clinician, but rather represent an averaged picture across the participating clinicians and indicate those factors that are most associated with certainty. Furthermore, the clinicians contributing to the SSC cohort may not be representative of all clinicians performing autism assessments.

Finally, the study is an exploratory investigation of certainty for an Autistic Disorder diagnosis, and the findings should therefore be sought replicated in future studies.