This study aimed to explore the prospective associations between both alcohol and cannabis use (frequency and DSM-5 substance use disorder symptom counts) as they relate to academic performance over a period of ∼ four years. The study members were students enrolled in college in the Southeastern United States (N = 209, 62 % female, 90 % white). Baseline survey data were obtained in the first year of college while participants finished the follow-up assessments ∼ 1 and ∼ 4 years from baseline (80+ % retention rates). We aimed to tease apart antecedent from consequence in the associations between substance use and academic performance using a sophisticated modeling technique that accounts for between-person influences in evaluating within-person change over time. Results showed that greater alcohol use disorder symptoms in the first year of college were associated with a within-person decline in grades in the second year of college. This was not demonstrated for alcohol or cannabis frequency, alcohol quantity, or cannabis use disorder symptoms or in relation to the later developmental transition out of college. In addition to widely implemented prevention efforts, these results indicate that screening first-year college students for alcohol use disorder symptoms and connecting them with resources and treatment may be beneficial in attempts to promote academic performance.