Osteoarthritis (OA) is among the most frequent diseases of the musculoskeletal system. Degradation of cartilage extracellular matrix (ECM) is a hallmark of OA. During the degradation process intact/full-length proteins as well as proteolytic fragments are released which then might induce different downstream responses via diverse receptors, therefore leading to different biological consequences. Collagen type II and the proteoglycan aggrecan are the most abundant components of the cartilage ECM. However, over the last decades, a large number of minor components have been identified and for some of those a role in the manifold processes associated with OA has already been demonstrated. To date, there is still no therapy able to halt or cure OA. A better understanding of the matrikine landscape occurring with or even preceding obvious degenerative changes in joint tissues is needed and might help to identify molecules that could serve as biomarkers, druggable targets, or even be blueprints for DMOADs. For this narrative review, we screened PubMed for relevant literature in English language and summarized the current knowledge regarding the function of selected ECM molecules and the derived matrikines in the context of cartilage and OA.