This methodological study based on IRT used data collected with the Food Intake and Physical Activity of Schoolchildren (WebCAAFE) questionnaire to develop and validate the Meal and Snack Assessment (MESA) quality scale. The developed scale was tested on a representative sample of schoolchildren. A flowchart of the study design is presented in Fig. 1.

Fig. 1

Study design flowchart. The Meal and Snack Assessment (MESA) Quality Scale development was guided by recommendations for psychometric studies, a science that studies the measurement of non-directly observable phenomena (latent traits): Latent trait definition (1); Item generation (2); Dimensionality analysis (3); Estimation of item parameters (4); Linear transformation of parameters (5); Scale levels definition (6); Assessment of scale's validity (7), and reliability (8); Estimation of the latent trait of the individuals in the sample (9)

Step 1: Data sourceStudy population and sampling design

The description of the design and sampling procedure has been published previously [20]. In brief, data collection was conducted from August to October 2013, 2014, and 2015 in Florianópolis, Santa Catarina, Brazil. The target population comprised schoolchildren of 6–15-y-old attending from the 2nd to 5th grade of municipal elementary schools equipped with computer rooms and internet access. The primary sampling units were eligible classrooms randomly selected from a complete list of schools with computer rooms provided by the education authority (34 out of 37 schools with 6,227 students enrolled in 2013; 34 out of 36 schools with 6,500 students in 2014; 35 out of 36 schools with 7,104 students in 2015). School years were considered secondary sampling units, with four classes from each educational unit being randomly selected, one from each year. All students from the selected classes were invited to participate in the study. Surveys included a total sample of 7,053 schoolchildren (2013–2015). Of these, 654 children with implausible dietary data were excluded (222 for reporting consumption of fewer than four food items per day and 432 for reporting values three times greater than the standard deviation of the mean). The final sample for the scale development included 6,399 students (1,950 in 2013, 2,019 in 2014, and 2,430 in 2015). For the IRT analyses, samples of around 250–500 respondents with different levels of the latent trace are often recommended [21].

WebCAAFE

WebCAAFE is an online questionnaire developed for the periodic assessment of dietary intake, physical activity, and sedentary behavior in students from the 2nd to the 5th grade of public schools. It collects data on the previous day (24-h recall) and comprises three sections, namely general information, dietary intake, and physical activity/sedentary behaviors [22]. WebCAAFE was tested for both reproducibility and validity [23,24,25,26]. A demonstrative version is available at https://caafe.ufsc.br. The dietary intake section includes six eating occasions (breakfast, morning snack, lunch, afternoon snack, dinner, and evening snack), each illustrated with up to 32 food icons (rice, vegetables, greens leaves, vegetable soup, beans, cassava flour, corn/potatoes/mashed potatoes, pasta, instant pasta, French fries, beef/poultry, sausages, eggs, fish/seafood, fruits, bread, cheese bread, cream cookies, breakfast cereal, porridge, cheese, coffee with milk, milk, yogurt, chocolate milk, fruit juices, sodas, sweets, packaged snacks, pizza/hamburger/hot dog, nuggets, and plain cake). This section was used to develop the MESA. Schoolchildren selected the foods consumed at each meal of the previous day. The instrument does not allow identifying food amounts or portions and, therefore, does not provide information on total energy or nutrient intake. The objective is to investigate the consumption of healthy and unhealthy foods. Additionally, schoolchildren answer questions about school meals, including the frequency of school meal consumption (0–1, 2–3, or 4–5 times/week). The physical activity and sedentary behaviors were described in detail by Lobo et al. (2019) [20] and assessed by three periods of the day (morning, afternoon, and night). The subject's physical activity score (PAS) was the sum of all activity scores. The variable was categorized into tertiles. The daily frequency of screen-based sedentary activities (television, videogame, computer, tablet, cell phone) was determined for each child and categorized into tertiles. These data were used to assess the quality of schoolchildren's meals in terms of the population description. The instrument was applied once to each child, in computer rooms at schools, in the presence of trained researchers. The day on which the questionnaire was applied differed among children.

Anthropometric measurements

Weight and height were measured by trained anthropometrists using standardized procedures [27]. The schoolchildren were barefoot and wearing lightweight clothing. Weight was measured with a digital scale (Marte, model PP, 180 kg maximum capacity, 50 g precision, São Paulo, Brazil). Height was measured using a portable stadiometer (AlturExata®, 1 mm precision, Belo Horizonte, Brazil). The body mass index (BMI) was computed as weight (kg) divided by height squared (m2). Age- and sex-specific BMI Z-scores were calculated according to the World Health Organization [28]. Weight status was categorized as non-overweight (thinness and normal weight, BMI Z-score for age <  + 1) or overweight including obesity (BMI Z-score for age ≥  + 1). Anthropometric data were collected on the same day that the schoolchildren answered the WebCAAFE questionnaire.

Other variables

Information on sex, date of birth, and school shift was provided by the administrative sector of the schools. Age was calculated using the child's date of birth and the date of data collection and categorized as 7–9 and 10–12 years. Family income was estimated from the average income of the census tract of the school′s location available from the Brazilian Institute of Geography and Statistics (IBGE) [29]. The variable was categorized into tertiles.

Step 2: Development of the MESA scaleLatent trait

Although diet quality is a multidimensional concept that encapsulates, among others, nutritional quality (food diversity, dietary adequacy, nutrient density), sensory organoleptic quality, food safety, the social dimension of food [30]. For this study, meal quality was defined considering the current recommendations [18, 19] for choosing and combining foods to compose a healthy meal. Healthy meals are based on a great variety of unprocessed or minimally processed foods, balanced across food groups, while restricting ultra-processed foods [18, 19]. In a healthy diet, processed foods may be consumed in small quantities as ingredients in culinary preparations or as part of meals based on unprocessed/minimally processed foods [11].

Item generation

WebCAAFE foods were classified according to the NOVA [17] system into three groups, as follows: (i) unprocessed/minimally processed foods (MPF), including rice, green leaves, vegetables, vegetable soup, beans, cassava flour, corn/potatoes/mashed potatoes, pasta, beef/poultry, eggs, fish/seafood, fruit, porridge, coffee with milk, milk, plain cake; (ii) processed foods (PF), including bread and cheese; and (iii) ultra-processed foods (UPF), including instant pasta, French fries, sausages, cheese bread, cream cookies, breakfast cereal, yogurt, chocolate milk, fruit juices, sodas, sweets, chips, pizza/hamburger/hot dogs, nuggets. The group of processed culinary ingredients was not included because none of the WebCAAFE foods are classified as such. For the IRT analyses, three items (consumption of MPF, PF, and UPF) were proposed to represent the foods on each of the six eating occasions, totaling 18 items. Two response categories were defined: non-consumption and consumption. WebCAAFE consumption reports were used to determine the consumption or non-consumption of MPF, PF, and UPF by each schoolchild in each of the six daily meals. A child was classified as having consumed MPF, PF, or UPF in the meal if they had consumed at least one food from the WebCAAFE classified as MPF, PF, or UPF.

Dimensionality

Dimensionality was evaluated by full-information factor analysis with oblimin rotation. The set of items was considered unidimensional when a dominant factor explained more than 20% of the data variation [31]. Factor loadings (≥ 0.3) and commonality (≥ 0.2) were also considered.

Item parameters

Discrimination (α) and location (δ) parameters were estimated using the generalized graded unfolding model (GGUM) [32] of IRT, represented by the following equation:

$$}\left(}=}1\uptheta }_}}\right)=\frac}\left[}}_}}\left(}\left(_}}-_}}\right)-\sum\limits_}=0}^_}}\right)\right]+}\left[}}_}}\left(\left(}-}\right)\left(_}}-_}}\right)-\sum\limits_}=0}^}}_}}\right)\right]}}=0}^}}\left[}\left(}}_}}\left[}\left(_}}-_}}\right)-\sum\limits_}=0}^}}_}}\right]\right)+}\left(}}_}}\left[\left(}-}\right)\left(_}}-_}}\right)-\sum\limits_}=0}^}}_}}\right]\right)\right]}$$

where:

Zi = observable response to item i;

z = 0, 1, 2, 3, ..., H; with z = 0 representing the strongest level of disagreement and z = H representing the strongest level of agreement;

θj = parameter of location of individual j on the latent trait scale, also called individual score or individual latent trait;

αi = discrimination parameter of item i;

δi = location parameter of item i on the latent trait scale;

τik = threshold location parameter of subjective response category k on the latent trait scale relative to the position of item i;

H = number of observable response categories minus 1; and

M = 2H + 1.

The equation quantifies the probability of an item response as a function of latent trait and item parameters, represented by the item characteristic curve (ICC). Unfolding models are proximity models developed for attitude, behavior, and preference measures. Individuals and items are expressed in the same metric along the latent trait continuum. The probability of a positive response increases as the individual's value in the latent trait is close to the item's value. Individuals with a latent trait level close to the item will be more likely to agree with the item. IRT uses the individual's response to items and the psychometric property of items themselves to generate a score that represents the latent trait measure [32]. For computational convenience and based on the principle of invariance, all item parameters were initially represented on a (0,1) scale, with 0 representing the mean and 1 the standard deviation of the respondents [21]. The discrimination parameter (α) indicates the ability to discriminate individuals with different latent trait levels, serving as a measure of item quality. Items with α ≥ 0.7, on the (0,1) scale, provide better discrimination of the latent trait. The location parameter (δ) identifies the item's position on the latent trait continuum [33]. Values are expected to be in the range of –2 to + 2 [34]. Item parameters were estimated by the maximum marginal likelihood method and analyzed in terms of standard error (SE) and ICC. Estimates of individual parameters (scores) were obtained by the Bayesian expected a posteriori method [32]. Analyses were performed using the MIRT package in software RStudio v.1.2.5033 (RStudio Team, 2020).

Linear transformation of parameters

To facilitate the development and interpretation of the scale, avoiding negative or decimal numbers, we performed the linear transformation of parameters to mean 100 and standard deviation 10 (scale 100,10).

Definition of scale levels

The probability of consumption and non-consumption of each item in the six daily meals, calculated using ICC, is represented on the scale. The cut-off points for each level considered the gain in diet quality with food intake characteristics (MPF, UPF, or both). Three levels of meal quality were established, namely healthy, mixed, and unhealthy. This step was performed by an expert. Scale levels and their descriptions were reviewed by authors.

Step 3: Validity and reliability of the MESA scale

Evidence of construct validity was obtained by the analysis of the internal structure, estimation of IRT parameters, and differential item functioning (DIF) [35] for sex and age. DIF is present when an item behaves differently between two or more groups of individuals with the same level of a latent trait. A lack of DIF is indicative of good construct validity [21]. The assessment of scale reliability was performed using the IRT test information curve (TIC).

Step 4: MESA Application and interpretation

The scale was applied to a subsample of elementary schoolchildren from Florianópolis, Brazil (n = 6,372), to estimate the quality of their meals using the sampling plan created for data collection through the WebCAAFE surveys from 2013 to 2015. The sampling plan was stratified into two levels, the strata being the combination of year and school; level 1 was the class, and level 2 was the student within the class. For interpretation, items and individuals were simultaneously positioned on the latent trait continuum, with the individual's parameter being allocated according to their score, and items being allocated according to the values of their location parameters. Students with scores close to the item's position probably agree with the item. The 95% confidence interval (95% CI) was used to analyze differences between survey years and meal quality levels as well as between meal quality levels and the other variables. Analyses were performed using STATA 16.1 (StataCorp,2020). All analyses considered the effect of study design, incorporating sample weights through the “svy” command in STATA.