Sex and gender play important roles in contributing to disease and health outcomes and represent essential, but often overlooked, measures in biomedical research. Here, sex is defined as the biological characteristics (e.g., sex chromosomes, sex steroid hormones, anatomy, etc.) that define humans as female, male, or intersex [1]. Gender is a social construct relating to the norms and roles associated with being a man, woman, or an identity beyond these categories, and relations between groups [2]. The biological differences between males and females impact one’s risk of disease, manifestation of illness, immune responses to treatment, and many other aspects of one’s health and well-being [1]. On the other hand, gender norms, roles, and relations impact access to important health resources, health behaviors, and health outcomes. Gender dimensions include: access to resources, such as knowledge, education, or financial resources; decision-making around who gets to seek care and when; practices that can increase risk of poor health, such as substance abuse, smoking, and violence; norms around what is appropriate for men and women and gender-minority individuals; and care-seeking behaviors [3]. Gender is, therefore, an important social determinant of health.

Within biomedical research, gender is often conceptualized as gender identity—whether a person identifies as being a woman, man, or gender minority individual. How this is captured in research also differs, for example, by directly asking whether someone is a woman, man, or gender minority individual. There are different ways to ask about a person’s gender identity due to the variations in genders that exist. The National Academies of Sciences, Engineering, and Medicine have issued guidance for researchers on how to measure sex, gender identity, and sexual orientation [4], which can be used to navigate these complexities. Gender norms, roles, and relations are different from, but related to, gender identity in that they are more systemic, simultaneously influencing how society is organized more broadly in terms of social norms, institutions, structures, resources, interpersonal relationships between individuals (particularly men and women), and a person’s individual gender identity. Asking about a person’s gender identity can help disaggregate data and explore differences between groups but will not get at the ways in which gender norms, roles, and relations impact a person’s access to important health resources, health behaviors, and health outcomes.

The context-specific, multifaceted, and relational nature of gender norms, roles, and relations (hereafter referred to as gender dimensions) makes its incorporation into biomedical research challenging. Many aspects of gender can also be difficult to quantify, making its incorporation into research methodologies, such as surveys or questionnaires, even more difficult. Gender scores—measures of gender dimensions—can help researchers incorporate gender into quantitative methodologies, which are commonly used in biomedical research. These measures enable researchers to calculate the presence of the gendered phenomena of interest using data collected from survey respondents.

While not exhaustive, the gender scores that have been utilized in biomedical research include: the Bem Sex Role Inventory (BSRI) scale [5,6,7], the Gender and Sex Determinants of Cardiovascular Disease: From Bench to Beyond-Premature Acute Coronary Syndrome questionnaire (GENESIS-PRAXY), which embeds the BSRI in its items [8, 9], the Conformity to Masculine Norms Inventory (CMNI) [10], the Gender Role Conflict Scale (GRCS) [11], the Personal Attributes Questionnaire (PAQ) [12], and the Gender Equitable Men’s Scale (GEMS) [13]. Many of these scores seek to assess participants’ identification with traditional masculine and feminine traits, which capture different gender dimensions as a proxy for gender. Specifically, trait measures recognize that an individual may possess both masculine and feminine characteristics and these may represent the extent of an individual’s adherence to cultural gender norms [14]. Other measures, such as the GEMS and CMNI, measure gender ideologies by assessing an individual’s endorsement of a culture’s ideological beliefs about gender roles. Select scores are further described in Table 1. These gender scores have been used across an array of medical fields, including obesity [5], cancer [11, 15], mental health [6, 16], injuries [17, 18], and heart disease [8, 19, 20], with mixed results.

For example, one study conducted in 2007 using the short form of the BSRI reported that men with higher femininity scores had a lower risk of coronary heart disease, yet the same relationship was not observed among women [21]. Another study using the GENESIS-PRAXY gender score found worse cardiovascular health and a higher prevalence of heart disease were associated with gender roles and personality traits typically ascribed to women, regardless of the individual’s sex, among Canadian and Austrian population samples [20]. In both populations, gender correlated more strongly with a higher risk of heart disease than sex. Another study using the BSRI examined the role of gender in treatment adherence among participants with bipolar disorder and found that males with high masculinity scores were nearly four times more likely to not adhere to medication when compared to males who did not have high masculinity characteristics [16]. No significant relationship between gender scores and adherence was found in females. While gender is closely tied to an individual’s health-seeking behaviors, access to healthcare, and gender-related health risks, the consideration of gender in biomedical studies remains limited [22].

To highlight the complexities of using gender scores within biomedical research, we explored the application of the BSRI, a commonly used gender score, to evaluate sex and gender differences in immunological response and adverse events to the influenza vaccine among older adults (75+) [23,24,25,26]. A description of the study, its methodology, and BSRI findings are provided below.

In this paper, we focus on the findings from our longitudinal gender score data, irrespective of adverse event data, to provide commentary on the reliability of gender scores, such as the BSRI, and the complexities of their application. The BSRI was originally developed in 1974 and used a 12-item scale of masculine traits (e.g., leadership abilities, strong personality, acts as a leader, dominant, makes decisions easily, and defends own beliefs) and feminine traits (e.g., warmth, gentleness, affection, sympathy, sensitivity to others’ needs, and tenderness), which were representative of the time to ascertain whether a person was feminine, masculine, androgynous, or undifferentiated [7, 23].

We applied the BSRI within a larger vaccination study, the Johns Hopkins Longitudinal Influenza Immunization Study of Aging (JH-LIISA), which recruited participants during the 2019-20, 2020-21, and 2021-22 influenza seasons in Baltimore, Maryland, United States. We used a modified short form of the 12-item BSRI to calculate femininity and masculinity scores and to assign participants to one of the four gender categories. A five-point Likert scale, ranging from 1 = never to 5 = always, was used to measure each trait and totaled, with gender categories assigned relative to the sample median feminine and masculine score as previously described [7, 23]. Androgyny combines masculine and feminine traits, and undifferentiated describes people whose scores on feminine and masculine traits were low [7]. The BSRI was chosen for this study as it has been used and validated in populations of older adults in multiple different cultures over the past decade [7, 23, 27,28,29]. Despite (or perhaps the result of) being developed over five decades ago [30], it is one of the most commonly used and validated measures of gender dimensions [7].

As previously published, the odds of reporting an adverse event following influenza vaccination did not depend on the gender category, but rather biological sex, among older adults (75+) [23]. Here, we explored whether participants’ gender scores remained the same or differed from year-to-year to better understand the reliability of gender scores over time. Of the 162 total study participants included within JH-LIISA, 69 were enrolled in all three consecutive seasons and 35 participants were enrolled in two consecutive seasons. 32% of the 69 participants (n = 22 total) enrolled in all three consecutive seasons scored within the same BSRI gender category from year-to-year, while 46% of the 35 participants (n = 16) enrolled in two consecutive seasons scored within the same BSRI gender category from in both years (Fig. 1). Most participants changed in how their gender was categorized by the BSRI (n = 47 of 69, 68% of those enrolled in all three seasons; n = 19 of 35, 54% of those enrolled in two consecutive seasons). Pearson correlation coefficients were calculated for the BSRI masculinity and femininity scores for consecutive years (2019–2020 and 2020–2021) to assess retest reliability, with r values ranging from 0.52 to 0.58 (all p < 0.001), suggesting moderate reliability of the BSRI from year-to-year. These changes demonstrate the nuances and fluidity of gender identity and how interpretations of BSRI data, when measured against outcome data, must be time and context-specific as results are unlikely to be replicated across years.

Graphic representation of gender categories based on participants’ scoring via the Bem-Sex Role Inventory (BSRI), connected and color-coded by gender category across two or three seasons, depending on participant enrollment, of a longitudinal influenza vaccine study. The chords show participant BSRI scores from year-to-year. Chords connecting from the same color between years represent participants with unchanging BSRI scores. Chords connecting different colors between years represent changes in gender scoring. Number of participants within each category is noted. (n = 162; all three seasons n = 69; two consecutive seasons n = 35)