Colorectal cancer (CRC) is the fourth most frequently diagnosed cancer worldwide. Bone marrow stromal cells (BMSCs) play an essential role in tumor development by secreting exosomes. Scavenger receptor class A member 5 (SCARA5) is a newly identified tumor suppressor. This study aimed to investigate the effects of BMSCs-derived exosomes (BMSCs-Exos) on CRC development and to explore their regulatory mechanisms. BMSCs-Exos showed an oval-shaped, bilayer membrane structure. BMSCs-Exos inhibited growth and motility of CRC cells, while BMSCs-Exos with SCARA5 knockdown significantly promoted cell proliferation and movement. Exosomal SCARA5 also effectively suppressed colorectal tumor growth in mouse xenografts. Further analysis revealed that exosomal SCARA5 inhibited the phosphorylation of protein kinase B and phosphoinositide 3-kinase in both CRC cells and tumors. In conclusion, SCARA5 in BMSCs-Exos inhibited CRC progression by inactivating PI3K/Akt, thus suggesting the potential clinical application of SCARA5-containing BMSCs-Exos for CRC treatment.