Testing rationality of decision-making and choice by evaluating the mathematical property of transitivity has a long tradition in biology, economics, psychology, and zoology. This paradigm is fraught with conceptual, mathematical, and statistical pitfalls. In this overview, we tackle five major obstacles. One challenge lies in spelling out what transitivity of latent preferences really says and what it actually implies about observable choice behavior. Most notably, this step is fraught with aggregation artifacts, in that aggregated behavior can be profoundly misleading about individual behavior. Another hurdle comes from hard mathematical problems associated with characterizing the properties of heterogeneous transitive populations. A third challenge is the prevalence of straw man hypotheses in this area of research. The fourth difficulty is associated with adopting appropriate statistical inference tools that correctly accommodate the idiosyncratic mathematical properties of order-constrained statistical hypotheses. The fifth hurdle arises with the role of scientific parsimony in rationality research. We walk readers through key concepts, mathematical models, and statistical techniques for testing rationality. Throughout, we provide examples using the methods and data of two prominent published papers on animal choice behavior as our case studies. We explain how these papers tackled the five hurdles to varying degrees of success.