Young people are using a variety of strategic behaviors on social media to counter oppression and enact new liberatory futures for themselves and their communities. Although quantitative research has utilized study-specific scales to measure similar concepts as sociopolitical action on social media, such as online political participation, there is a need for a scale developed with an anti-oppressive lens based on sociopolitical development theory. This study, which is composed of three sequential studies, developed and validated the Sociopolitical Action Scale for Social Media (SASSM) with a gender, racially and ethnically, and immigrant diverse sample of young people, as well as youth from adolescence (14–17) to emerging adulthood (18–25). Although young people around the world are engaging in sociopolitical action on social media and their actions to challenge oppression may be similar to those used by young people in the U.S., this study focused on U.S.-based youth for two reasons: the initial qualitative study upon which the scale was based was conducted with U.S. based digital activists, and most research using SPD theory has been conducted in the U.S.

Study I MethodParticipants and Procedure

Participants for Study I, a qualitative study, were youth (N = 20) ages 16–21 (Mage = 19) who were highly civically engaged on Twitter and resided in 10 U.S. states. Youth were recruited through the “followers” of social movement chapters on Twitter, including March for Our Lives, Sunrise Movement, and Black Lives Matter. Although participants were not asked to identify their political ideology, it is likely that most participants identified as liberal. We return to this issue in the discussion to further consider how the sampling shapes the interpretation of findings. Participants were contacted through Twitter direct message and invited for one-hour semi-structured phone interviews, which were conducted between March and September 2020. Fourteen (70%) participants were first or second generation immigrants, 10 (50%) identified as part of non-majority religions in the U.S., 45% identified as cis-gender men, 40% as cis-gender women, and 15% as gender nonbinary. Participants identified as Asian American (45%), Black (20%), Latinx (5%), Middle Eastern and North African (10%), Multiracial (10%), and White (10%).

Study I Findings and Item Development

Please refer to Wilf and Wray-Lake (2021) for a full accounting of the methodology and themes in Study I. Critical consciousness, a framework that is closely related to sociopolitical development theory (Diemer et al., 2021), was used as a sensitizing concept in analysis. Using inductive Constant Comparative Analysis (Fram, 2013), the authors identified three forms of sociopolitical action on social media (which were called online civic engagement) that formed the basis for items in this scale: Restorying, Building Community, and Taking Collective Action. Each of these forms of sociopolitical action were described by youth as ways to resist and heal from the psychological effects of oppression. Restorying, where youth reframed, challenged, and imagined new narratives, included four sub-themes:(1) personal storytelling, (2) challenging and reframing dominant narratives, (3) envisioning new futures, and (4) self-love. Building community included (1) emotional support and (2) allyship. Youth described emotional support as individual (self-care), relational (providing support to others), and collective (group healing). Taking collective action included (1) holding people accountable and (2) artivism. All youth viewed holding people accountable as an important way to make their voices heard in the face of structural barriers to political participation (such as the voting age). Fewer youth mentioned artivism, but those who did explained that it was a powerful tool for resistance and healing.

Based on how young people in Study I described their online civic engagement, this study conceptualized youth sociopolitical action on social media as behaviors that transcend one-time, discrete actions (like using a hashtag or signing a petition), an approach which allows for a range of actions and is more resistant to technological advances. Because youth are participating in so many distinct actions online to create sociopolitical change, a scale measuring youth sociopolitical action through these distinct types of actions could include hundreds, if not thousands, of items. For example, a young person might debate a sociopolitical topic by posting their own opinion, posting a news article, sharing someone else’s post, commenting on someone else’s post, creating a video, using a hashtag — the list goes on, and will continue to increase with the growth of new platforms and tools. Indeed, youth sociopolitical action on social media is constantly evolving with new technologies, which could render such a scale rapidly obsolete. To remedy this issue, the present study aimed to measure youth higher-level actions such as telling their personal story on social media to challenge stigmas and stereotypes, rather than discrete actions such as signing a petition. This approach aligns better with young people’s own descriptions of their online sociopolitical action (Wilf & Wray-Lake, 2021).

Items developed for the scale based on the abovementioned themes were identified in the interviews (see Table 1 below for how each theme and sub-theme translated into items). First, items were created based on each theme and sub-theme, without limiting the number of items. Next, items were reduced or merged to reduce overlap. For example, two items focused on holding people accountable (differentiated by individuals and institutions) were merged. Whenever possible, item wording reflected exact terms used by youth in the interviews. The initial item set included items focusing on specific actions (such as “participate in a digital strike or protest for a social or political issue”), but these were removed before EFA analysis due to the scale’s focus on higher-level sociopolitical actions. Given that building community was not interpreted in the same way by all youth, items that reflected the sub-themes of emotional support and allyship were included instead of an item on community building.

For the SASSM, respondents report the frequency of engagement online by answering, “How often do you do the following on social media?” Responses were a 5-point frequency scale based on a monthly calendar (Never, Once or twice a month, Once a week, Several times a week, and Daily) because almost all U.S.-based youth use social media on a daily basis (Vogels et al., 2022). The items were worded to be comprehensible to youth from a broad age range. Examples were included in parenthesis for some items to ensure they were interpreted in the same way by all participants.

Table 1 Study I SASSM Initial Item Development Study IIParticipants and Procedure

The Study II sample consisted of 809 participants recruited through Instagram advertising in July 2020 to support the validation of the SASSM. The advertisement redirected participants to an online survey hosted on Qualtrics with a consent form. The sample was majority cis-gender women (55.7%), with 35.5% cis-gender men and 8.8% non-cisgender participants, and age ranged from 14 to 25 years old (Mage=17) with most participants (68.6%) 14–17 years old. Participants identified as Asian (19.9%), Black / African American (11.7%), Hispanic or Latinx/a/o (11.3%), White (39.9%), and Multiracial (16%), as well as American Indian / Alaska Native, Native Hawaiian or Pacific Islander, and Middle Eastern / North African (1.2%). A majority of respondents (54.5%) identified as first or second generation immigrants. Participants resided in 40 different U.S. states.

Study II Methods

In Study II, an exploratory factor analysis (EFA) was conducted with the 16-item scale to test its factorial structure. To determine the best-fitting factor structure eigenvalues, scree plots, and parallel analysis were utilized (El-Den et al., 2020). Factor loadings were evaluated based on a cut-off point of 0.4 (Costello & Osborne, 2005). Model fit indices were used to evaluate model fit, including the Comparative Fit Index (CFI) of ≥ 0.95, the Root Mean Square Error of Approximation (RMSEA) of ≤ 0.06, and the Standardized Root Mean Square Residual (SRMR) of ≤ 0.06 (Little, 2013).

Study II Results

First, data adequacy tests were conducted in SPSS version 27 using Bartlett’s test of sphericity, which confirms a significant correlation between variables, and Kaiser-Meyer-Olkin (KMO), which examines the proportion of common variance between variables. Bartlett’s test of sphericity was 0.00, under the threshold of 0.05, and KMO was 0.96, indicating that data were satisfactory for factor analysis. Next, data normality was examined. Skewness ranged from − 1.00 to 2.13, and kurtosis ranged from − 1.41 to 4.47. These tests confirmed that data were not normal, with implications for the EFA (described below). Missing data ranged from 5.30 to 6.70% for each item, with an average of 6.10% missing on each item. Using Little’s MCAR test, data were confirmed to be missing completely at random (x2 = 666.890, df = 645, p = .267).

Next, an EFA was conducted in MPlus version 8.1. Because data were non-normal, maximum likelihood estimation with robust standard errors (MLR) was used. Geomin Oblique was utilized because the items were expected to correlate with each other as aspects of youth online sociopolitical action. Full information maximum likelihood (FIML) estimation was used to handle missing data. The first four eigenvalues of (1) 8.058, (2) 1.247, (3) 0.948, and (4) 0.773, indicated that a two-factor solution might be best fitting. Next, the item factor loadings were investigated. In the two-factor model, only two items (items #15 and 16 in Table 2 above) loaded above 0.30 on the second factor. These two items were determined to be encompassed in items 5 and 7 (challenging stigmas, and changing how people think), and removed from the scale.

Table 2 Study II and Study III EFA Factor Loadings and Communalities

A second EFA was conducted with the 14 remaining items. The first four eigenvalues were (1) 7.407, (2) 0.953, (3) 0.861, and (4) 0.770, indicating a one-factor solution. Parallel analysis was used to determine the number of factors to retain using randomly generated correlation matrices (Hayton et al., 2004). The first two eigenvalues randomly generated were (1) 1.268 and (2) 1.206. Because the second randomly generated eigenvalue was slightly larger than the second eigenvalue, a one-factor solution was determined. Over half of the items loaded above 0.7, demonstrating that these items were satisfactory in explaining variance in the factor (see Table 2). Model fit indices for the 14-item scale suggested that a one-factor model was a good fit: the RMSEA was 0.059, the CFI was 0.951, and the SRMR was 0.035 (Little, 2013).

Before conducting Study III, the first author made three additional changes to the scale based on consultations with the second author and feedback from participants (Boateng et al., 2018). Items 10 and 11 (encouraging others to take action online and in-person) were merged to increase parsimony and reduce confusion. An open-ended question in the survey asking “Did we miss anything? Are there other ways you engage on social media for social and political issues?” resulted in two additional items on personal storytelling and amplifying marginalized voices (see Table 2 below for changes from the Study II to the Study III survey). Lastly, most examples in parentheses were removed from the items to facilitate comprehension, with the exception of artivism, as the term “art” may not be understood by all youth as inclusive of poetry, paintings, infographics, and other media.

Study IIIParticipants and Procedure

The Study III sample consisted of 820 participants, recruited through Instagram advertising in October 2020 to validate the SASSM. Participants identified as cis-gender women (45.9%), cis-gender men (43.5%), gender nonbinary/ nonconforming (7.3%), and transgender (7.9%). Participants’ ages ranged from 14 to 25 years old (Mage=17) with most participants (63.9%) ages 14–17. Participants identified as Asian (18.2%), Black / African American (8.1%), Hispanic or Latinx/a/o (11.7%), White (46.1%), and Multiracial (13.7%), as well as a small number of participants who identified as American Indian / Alaska Native, Native Hawaiian or Pacific Islander, and Middle Eastern / North African (2.2%). Participants who selected more than one racial and ethnic identity were coded as multiracial. Participants who did not select a category but wrote in a racial/ ethnic identity were categorized by the first author based on U.S. Census descriptions (U.S. Census Bureau, 2021). For measurement invariance tests by racial and ethnic identity, participants who identified as American Indian/Alaska Native, Native Hawaiian or Pacific Islander, and Middle Eastern/North African were removed, as low sample size precluded model estimation. A high percentage of participants (47.9%) identified as first or second generation immigrants. Respondents from 45 U.S. states were represented.

Study III Methods

In Study III, an EFA and CFA were conducted with the modified 15-item scale to confirm the unidimensional factor structure and to examine model consistency. An EFA and CFA are typically conducted on two independent samples (Little, 2013). Because of changes made to the SASSM between Study II and Study III, the second sample of 820 youth was randomly split into two independent samples to conduct an EFA (n = 250) and a CFA (n = 570). This approach has been used in previous psychological scale validation research (e.g., Diemer et al., 2017), and aligns with guidelines that sample sizes of 100–200 participants, or at least 10 participants per item, are sufficient for using structural equation modeling (Kline, 2015).

In Study III, an EFA was first conducted with the randomly selected sample of 250 participants to confirm the factor structure from Study II. Then, the rest of the Study III sample (n = 570) was utilized to conduct a CFA to evaluate model fit indices, following guidelines on goodness-of-fit cut-offs described above (Little, 2013). The same CFA sample (n = 570) was then used to conduct convergent validity tests with two theoretically related constructs (political efficacy and critical reflection). Finally, due to low sample sizes for certain racial and ethnic groups, the entire Study III sample (n = 820) was used to conduct configural, metric and scalar measurement invariance tests by age, gender, race and ethnicity, and immigrant identity. Measurement invariance determines whether items hold similar meaning for participants from different groups using a series of nested models that test for equivalent factor loadings (metric invariance) and item intercepts (scalar invariance) (Little, 2013). The configural model with no imposed constraints is compared to the metric model with factor loadings constrained to be equal across groups, and the metric model is compared to the scalar model that constrains both the factor loadings and the item intercepts to equality across groups. Theory suggests that little-to-no changes in the CFI (△CFI > 0.01), RMSEA (△RMSEA > 0.01), and SRMR (△SRMR > 0.025) demonstrate no significant decreases in model fit (Little, 2013). If any of these three fit indices were insufficient, partial invariance testing would be pursued.

Study III Measures

In addition to the SASSM, Study III utilized two scales to test convergent validity: critical reflection and political efficacy. A positive correlation between the SASSM scale and the two latent variables of political efficacy and critical reflection was hypothesized, supported by prior empirical research linking the three concepts (Hope et al., 2016; Diemer et al., 2017).

Critical Reflection

Critical reflection was measured by an 8-item sub-scale adapted from Diemer et al. 2017 (α = 0.90). The full scale has three components (Critical Reflection: Perceived Inequality, Critical Action: Sociopolitical Participation, and Critical Reflection: Egalitarianism). For this study, only the first sub-scale (Critical Reflection: Perceived Inequality) was utilized. The items assess perceived societal inequalities due to class, race, and gender, such as “Certain racial or ethnic groups have fewer chances to get a good high school education” and “Poor people have fewer chances to get good jobs.” The scale was measured on a five-point Likert scale from strongly agree to strongly disagree.

Political Efficacy

The Political Efficacy scale is a 4-item scale adapted from Hope and Jagers (2014), (α = 0.57) and Hope (2016), (α = 0.75). The scale measures youth perceptions of their own efficacy in improving society and creating change. The scale is coded so that higher numbers indicate greater efficacy. The items are on a five-point likert scale from strongly agree to strongly disagree, and include items such as “I believe that by participating in politics I can make a difference,” and “I have the skills and knowledge necessary to participate in politics.”

Study III Results Exploratory Factor Analysis

A random sample of 250 youth was selected for the EFA from the Study III sample of 820 youth, leaving 570 youth for the CFA (described below). Data were not normally distributed, with most items having kurtosis less than − 1. Bartlett’s test of sphericity was significant at χ2(105) = 3,375.853, p < .001, and the Kaiser-Meyer-Olkin measure of sampling adequacy was 0.747, indicating that data were favorable for factor analysis. Missing data ranged from 4.9 to 7.2% on each item; thus FIML was utilized to use all available data. Little’s MCAR test confirmed that data were missing completely at random (x2 = 226.980, df = 213, p = .243).

Because data were not normally distributed, maximum likelihood estimation with robust standard errors (MLR) and Geomin Oblique rotation were utilized to conduct the EFA in Mplus version 8.1. Similar to Study II, the first three eigenvalues (9.484, 0.771, and 0.705 respectively) indicated that a one-factor solution might be the best fit to the data, which was further confirmed by parallel analysis using randomly generated correlation matrices (1.445, 1.343, 1.267, and 1.194). Only the first eigenvalue of 9.484 was higher than the randomly generated value of 1.445, suggesting a one-factor solution was best. The majority of the items (11 out of 15) loaded above 0.7, and all items loaded above the suggested cut-off of 0.4. All but two of the item loadings increased from the Study II to the Study III EFA, indicating that wording changes strengthened the scale.

Confirmatory Factor Analysis

The one-factor model in the Study III EFA was cross-validated by conducting a CFA with the random, independent sample of 570 participants to evaluate how well each item in the scale loaded onto the single factor (Kline, 2015). Maximum likelihood estimation with robust standard errors (MLR) was used because the data were not normally distributed (see Table 3). The fixed factor method was used to scale the latent construct. Model fit indices indicated the one-factor solution was a good fit to the data, well within the acceptable cut-off ranges: SRMR = 0.025, RMSEA = 0.045, and CFI = 0.975. The Chi-Square test was significant, indicating model misfit to the data, at x2 (90, N = 570) = 106.17, p = .00, yet prior research has found the Chi-Square test to be sensitive to large sample sizes and recommends prioritizing relative fit indices (Babyak & Green, 2010). Overall, the scale demonstrated a moderate to strong fit to the data, with the lowest factor loading 0.573 and the majority loading above 0.7, providing evidence that the scale aligned with this study’s conceptualization of youth sociopolitical action on social media.

Table 3 Final Item Study III Confirmatory Factor Loadings (N=570) Measurement Invariance Gender

Measurement invariance testing was conducted with three gender groups: cis-gender women, cis-gender men, and non-cisgender youth (including nonbinary, transgender, and youth with other gender identities combined due to low sample sizes). The configural and metric models fit well across groups (see Table 4), but changes in CFI from the metric to scalar model were higher than recommended (△CFI = 0.015). Based on modification indices, the intercepts for two items were freed for cis-gender men (Little, 2013), resulting in partial scalar invariance by gender. Intercepts were freed for Item #3: Discuss or debate a social or political issue on social media, where compared to cis-gender women (M = 2.54) and non-cisgender youth (M = 2.32) the intercepts for cis-gender men were higher (M = 2.99). Based on the item average of 1 standard deviation (SD = 1.38), these differences represented 0.33 and 0.49 SDs. Intercepts were also freed for Item #7: Promote a new way of thinking or a new narrative about a social or political issue that more people need to know about, where cis-gender women (M = 2.56) and non-cisgender youth (M = 2.32) had lower means compared to cis-gender men (M = 2.82), representing 0.19 and 0.37 SDs (average SD = 1.35). A Wald Test of parameter restraints confirmed no significant difference between the intercepts on either item for cis-gender women and non-cisgender youth.

Table 4 Goodness-of-Fit Indicators for Measurement Invariance by Subgroup Racial and Ethnic Background

Next, invariance testing was conducted for youth from five racial and ethnic groups: Asian, Black or African American, Hispanic or Latino/a/x, Multiracial, and White. Fit indices for the configural model confirmed that the one factor model was a good fit. Next model comparison tests were used to confirm metric and scalar invariance (see Table 5). Scalar invariance was achieved. However, the sample size for Black and Latinx participants was under 100 for each group, and therefore results for these two groups should be interpreted with caution.

Table 5 Correlations Among Latent Constructs, Study III Sample (n= 820) Age

Measurement invariance testing was conducted for two age groups as determined by literature documenting differences in youth civic engagement in high school (ages 14–17), and emerging adulthood (18–25). Fit indices confirmed that the configural model was a good fit (see Table 4). Next changes in fit indices were used to confirm metric and scalar invariance. Metric invariance was established, indicating that participants in high school and beyond interpreted the scale in similar ways. Scalar invariance was well established across both age groups.

Immigrant Identity

Measurement invariance testing was conducted with two groups comprised of youth of first or second generation immigrant origin (i.e., either the participant or one or both parents were born outside of the U.S.), and non-immigrant youth. Strong invariance was established across these two groups, with small changes between configural, metric, and scalar models. Thus, first and second generation immigrant and non-immigrant youth interpreted the scale similarly.

Convergent Validity

Next convergent validity was examined to evaluate whether the SASSM was correlated with conceptually related constructs of political efficacy and critical reflection. Using MPlus version 8.1, structural equation modeling estimated correlations among the latent variables of online sociopolitical action using the SASSM, political efficacy and critical reflection. Like the measurement invariance tests, the entire Study III sample of 820 participants was used to increase power to detect statistical significance.

As hypothesized, a significant positive correlation of 0.438 was found between the SASSM and political efficacy, higher than past studies that have reported correlations ranging from 0.21 to 0.40 (See Table 5; Hope et al., 2016). A significant positive correlation of 0.341 was found between the SASSM and critical reflection: perceived inequality, which is also slightly higher than previously documented associations with in-person actions that ranged from 0.18 to 0.29. Therefore, these findings are consistent with prior research demonstrating positive associations between youth political efficacy and critical reflection, and their online sociopolitical action. Indeed, these results show that the associations between these constructs and youth online sociopolitical action may be even greater than with their in-person sociopolitical actions.