Despite growing interest in understanding how food environments shape dietary behaviors, European longitudinal evidence is scarce. We aimed to investigate the associations of 9-year average and change in exposure to local retail food environments with the diet quality of residents in Luxembourg. We used data from 566 adults enrolled in both waves of the nationwide ORISCAV-LUX study (2007–2017). Dietary quality was assessed by the Diet Quality Index-International (DQI-I). Exposure to “healthy” and “less healthy” food outlets was assessed by both absolute and relative GIS-based measurements. The results showed a 56.3% increase in less healthy food outlets over the period. In adjusted linear mixed models, high (vs. low) 9-year average exposure to less healthy food outlets was associated with lower DQI-I, when examining spatial access (β = −1.25, 95% CI: −2.29, −0.22) and proportions (β = −1.24, 95% CI: −2.15, −0.33). Stratified analyses showed these associations to be significant only among urban residents. There was no association between change in exposure to less healthy food outlets and DQI-I. Increased exposure to healthy outlets in rural areas, using absolute measurements, was associated with worsened DQI-I. Neighborhood socioeconomic status did not moderate the above associations. Findings suggest that the proliferation of less healthy food outlets may have contributed to the deterioration of the diet quality of urban residents, and support the use of relative measurements to fully capture the healthiness of food environments.