Studies on associations of various polymorphism in xenobiotic metabolizing genes with different cancers including esophageal squamous cell carcinoma (ESCC) are mixed and inconclusive. To evaluate the association of CYP1A1*4, SULT1A1*2 and SULT1A2*2 genotypes with ESCC risk and their modifying effects on different risk factors of ESCC, we conducted a case–control study in Kashmir, India, an area with relative high incidence of ESCC. We recruited 404 histopathologically confirmed ESCC cases, and equal number of controls, individually matched for sex, age and district of residence to respective case. Information was obtained on various dietary, lifestyle and environmental factors in face-to-face interviews, using a structured questionnaire, from each subject. Genotypes were analyzed by polymerase chain reaction, restriction fragment length polymorphism and direct sequencing. Conditional logistic regression models were used to calculate odds ratios (ORs) and 95% confidence intervals (95% CIs). A higher risk was observed in the subjects who harbored variant genotype of CYP1A1*4 (OR = 2.06; 95% CI: 1.28–3.32); and the risk was further enhanced in ever smokers (OR = 3.47; 95% CI: 1.62–7.42), adobe dwellers (OR = 6.71; 95% CI: 3.02–14.89), and biomass fuel users (OR = 5.11; 95% CI: 1.34–19.50). We did not find any significant differences in the polymorphic variants of SULT1A1*2 and SULT1A2*2 between cases and controls. The study indicates that, unlike SULT1A1*2 and SULT1A2*2, the polymorphism of CYP1A1*4 is associated with ESCC risk. However, replicative studies with larger sample size are needed to substantiate our findings.