Air pollution is a known risk factor for several diseases, but the extent to which it influences COVID-19 compared to other respiratory diseases remains unclear. We performed a test-negative case-control study among people with COVID-19-compatible symptoms who were tested for SARS-CoV-2 infection, to assess whether their long- and short-term exposure to ambient air pollution (AAP) was associated with testing positive (vs. negative) for SARS-CoV-2. We used individual-level data for all adult residents in the Netherlands who were tested for SARS-CoV-2 between June and November 2020, when only symptomatic people were tested, and modeled ambient concentrations of PM10, PM2.5, NO2 and O3 at geocoded residential addresses. In long-term exposure analysis, we selected individuals who did not change residential address in 2017–2019 (1.7 million tests) and considered the average concentrations of PM10, PM2.5 and NO2 in that period, and different sources of PM (industry, livestock, other agricultural activities, road traffic, other Dutch sources, foreign sources). In short-term exposure analysis, individuals not changing residential address in the two weeks before testing day (2.7 million tests) were included in the analyses, thus considering 1- and 2-week average concentrations of PM10, PM2.5, NO2 and O3 before testing day as exposure. Mixed-effects logistic regression analysis with adjustment for several confounders, including municipality and testing week to account for spatiotemporal variation in viral circulation, was used. Overall, there was no statistically significant effect of long-term exposure to the studied pollutants on the odds of testing positive vs. negative for SARS-CoV-2. However, significant positive associations of long-term exposure to PM10 and PM2.5 from specifically foreign and livestock sources, and to PM10 from other agricultural sources, were observed. Short-term exposure to PM10 (adjusting for NO2) and PM2.5 were also positively associated with increased odds of testing positive for SARS-CoV-2. While these exposures seemed to increase COVID-19 risk relative to other respiratory diseases, the underlying biological mechanisms remain unclear. This study reinforces the need to continue to strive for better air quality to support public health.