The data used in this study were collected by the Province of Ontario Neurodevelopmental Disorders (OBI-POND) network. POND participants are recruited across five sites in Ontario, Canada (Holland Bloorview Kids Rehabilitation Hospital, The Hospital for Sick Children, McMaster University, Lawson Health Research Institute, and Queen’s University). Participants at all sites receive the same phenotypic measures. The POND data export was obtained on July 23, 2021 and included a subset of participants 6–21 years old who had completed both the ABAS and VABS assessments between March 2012 and November 2019. VABS assessments were not collected in POND after 2019 and as such our dataset include the full set of participants for whom both ABAS and VABS data are available. The age range was chosen to align with the age range for the ABAS-II school-age form, while minimizing the overlap with the preschool form. Participants had a primary diagnosis of autism confirmed using the Autism Diagnostic Observation Schedule–2 (ADOS) (Lord et al. 2000) and the Autism Diagnostic Interview–Revised (ADI-R) (Lord et al. 1994), or were neurotypical (NT; no history of a neurodevelopmental, psychiatric, or neurological diagnosis, born after 35 weeks gestation). The data were included for participants who had both VABS and ABAS measures available.

The vineland adaptive behavior scales II (VABS) scores used in this study were obtained by a researcher using a semi-structured interview with a parent/primary caregiver (Sparrow et al., 2005). The VABS items are scored on a 3-point scale: 0 = behavior never performed, 1 = behavior sometimes or partially performed, and 2 = behavior usually or habitually performed. This instrument provides a composite score as well as domain scores for communication, daily living skills, socialization, and maladaptive behaviors (optional) (Sparrow et al., 2005). In this analysis, we examined the prediction of the VABS composite score (age-normed standard scores; mean 100 ± 15).

The adaptive behavior assessment system II (ABAS) was administered through the parent/primary caregiver forms for 5–21 years and completed by a parent or caregiver [11]. ABAS items are rated on a four-point scale ranging from 0 = is not able, 1 = never when needed, 2 = sometime when needed, 3 = always when needed. The ABAS measures adaptive function in 10 skill areas (communication, functional academics, self direction, community use, home living, health and safety, self care, leisure, social, work). The scaled skill area scores are aggregated into age-normed standard scores (mean 100 ± 15) for three domains (conceptual, practical, social) and a total composite score [11]. The work skill area was excluded from the present analyses given the participants’ age. For the prediction task, we used both the raw skill area scores (sum of questionnaire items and the scaled skill area scores (age-normed, mean 10 ± 3. Unlike the VABS, ABAS composite scores have a floor of 40 and an age-dependent ceiling: 160 for 0–5 years, 130 for 5–7 years, and 120 for 8–89 years [11].

To further characterize the sample, IQ was measured using measures appropriate for the child’s age and ability level (the Wechsler Abbreviated Scale of Intelligence, the Wechsler Intelligence Scale for Children, and the Stanford Binet Intelligence Scales). Autism-like traits were quantified using the Social Communication Questionnaire (SCQ)—Lifetime form [17]. ADHD-like traits were measured using the Strengths and Weaknesses of Attention-Deficit/Hyperactivity Disorder Symptoms and Normal Behavior Scale (SWAN) parent questionnaire which provides two subscale scores of inattentive and hyperactive/impulsive [20]. Emotional and behavioral symptoms were measured using the child behavior checklist (CBCL [1, 2]) parents/primary caregivers version, Internalizing and Externalizing scores, respectively. ADHD and emotional/behavioural symptoms were selected to characterize the sample given their relatively high prevalence in autism (Lai et al., 2019).

Statistical tests were conducted in R version 4.3.0 and prediction analyses were performed using scikit-learn version 1.0.2. In this analysis, we predicted VABS scores from ABAS scores given that ABAS is a parent-reported measure and may offer advantages in terms of scalability and cost. We used six regression approaches, namely, ordinary least squares (OLS) linear regression, ridge regression [4], ElasticNet [24], LASSO [22], AdaBoost [10], and random forest (Breiman, 2001). The choice of linear regression models (OLS, ridge, ElasticNet, and LASSO) was motivated by the previously reported linear association between VABS and ABAS scores [7]. Noting that demographic and phenotypic predictor variables are highly correlated in our datasets (e.g., sex, age, autism/ADHD features), we employed ridge regression, ElasticNet, and LASSO, which provide relative strengths in handling multicollinearity compared to OLS regression (Variance inflation factors between 1.1 and 12.8). AdaBoost and random forest regression were used as two ensemble regressors (collection of several regressors) which can improve the bias and variance of estimation through the use of multiple estimators.

In all models, the VABS composite scores were predicted as outcomes. We have chosen to go from ABAS to VABS given that the ABAS may be more scalable and cost effective in terms of administration as a parent-rated questionnaire (versus the VABS clinician interview), and as such may be more practical to use in a wider range of settings, especially in applications involving the creation of large data cohorts [7].

Four sets of predictor variables were examined. First, we established a minimum set of variables consisting of only the ABAS general composite score (set 1). The second set included the ABAS composite as well as age, sex, and diagnosis (set 2). Age was included in this set given previous findings of residual association with age in the VABS [7]. The third set included variables in set 2 in addition to ABAS raw scores (set 3). These were included to mitigate the impact of ABAS floor and ceiling. The fourth feature set included variables in set 3 as well as IQ, SCQ, SWAN, and CBCL internalizing and externalizing problems (set 4). The SCQ, SWAN, and CBCL variables were included in the prediction to examine the impact of core autism features and frequently co-occurring symptoms (inattention, hyperactivity, internalizing, externalizing) on the difference between VABS and ABAS scores. This was based on previous findings of residual association of VABS scores with these domains. For example, IQ was found to explain 46.6% of the variance in VABS composite scores, but only 36.4% in the ABAS composite scores [7].

We ran the six models with each feature set for participants who had complete scores for all assessments included in the analysis as well as the autism group only (6 models × 3 features × 2 groups). Additionally, sensitivity analysis was conducted by running models for feature sets 2 and 3 using the subset of participants included in feature set 4. We also examined the effect of family nesting, by comparing the performance of the linear regression model with and without accounting for family nesting by including family as a random effect in the model (set 2). Furthermore, we ran the models on a subset of the participants that included one, randomly chosen child from each family.

To evaluate model performance, tenfold stratified cross validation was used. Folds were stratified based on VABS composite scores (10 point bins were used for scores between 60 and 110), sex, and diagnosis. Median absolute error (MAE) was used as a measure of the residuals magnitude to quantify the difference between predicted and true VABS scores. To understand the sources of prediction error, we examined the association between the error from the above linear regression model and the covariates. Residuals were computed as the difference between the predictions of VABS scores and their true values. Predicted scores were obtained from using a linear regression model with coefficient obtained through 1000 bootstrap iterations on feature set 2.

The contribution of each variable to the model was quantified using the permutation feature importance approach which computes feature importance as the decrease in the model R-squared when a given feature is randomly shuffled (Permutation Feature Importance, n.d.).