Accumulating evidence indicates that social stress in the juvenile period affects hypothalamic–pituitary–adrenal (HPA) axis activity in adulthood. The biological mechanisms underlying this phenomenon remain unclear. We aimed to elucidate them by comparing adult mice that had experienced social isolation from postnatal day 21–35 (juvenile social isolation (JSI) group) with those reared normally (control group). JSI group mice showed an attenuated HPA response to acute swim stress, while the control group had a normal response to this stress. Activity levels of the paraventricular nucleus in both groups were comparable, as shown by c-Fos immunoreactivities and mRNA expression of c-Fos, Corticotropin-releasing factor (CRF), Glucocorticoid receptor, and Mineralocorticoid receptor. We found greater vascular coverage by tanycytic endfeet in the median eminence of the JSI group mice than in that of the control group mice under basal condition and after acute swim stress. Moreover, CRF content after acute swim stress was greater in the median eminence of the JSI group mice than in that of the control group mice. The attenuated HPA response to acute swim stress was specific to JSI group mice, but not to control group mice. Although a direct link awaits further experiments, tanycyte morphological changes in the median eminence could be related to the HPA response.