In this large-scale sample of 19,768 service users with SMI, using novel text-mining methods, we found that an extremely high proportion (85%) of service users had experienced unemployment. Our findings are consistent with previous studies [2, 34]. Unemployment rates in the general population for the catchment area were estimated to be 4.3% in London by the Office for National Statistics at the time of the study [35], providing further context. Although these unemployment statistics for the local area capture unemployment at one particular timepoint, and in our study we looked at any recorded experiences of unemployment in the health record, our findings highlight the potentially deep inequalities which impact this group.

We also found key clinical differences between service users who had been unemployed compared with those who had no recorded unemployment. As hypothesized, service users who had experienced inpatient admissions, longer inpatient stays and compulsory admissions were more likely to have experienced unemployment. This can be considered in the context of previous work which has found that service users with SMI who have more severe symptoms and lower functioning scores are more likely to experience unemployment [36]. Our study provides a new perspective on this by investigating associations between inpatient stays and unemployment. An inpatient stay may suggest that the service user had more intensive contact with services and may have been more clinically unwell, which could have made it difficult to then initiate and maintain employment [36]. We also found that patients with comorbid substance use disorders had over twice the odds of experiencing unemployment. Substance use disorders were ascertained using ICD-10 codes; however, rates were lower than expected [37]. The presence of substance use disorders could therefore be an underestimate, due to under-recording in the structured field of the health record for some service users. In addition, service users with an earlier age of SMI onset and a non-affective SMI diagnosis were also more likely to experience unemployment in the sample—these could also be indicators of illness severity and functioning.

We found evidence to support our hypotheses for associations between sociodemographic characteristics and unemployment including age and relationship status, but not for sex. We found that service users who were aged between 50 and 59 were more likely to have experienced unemployment. These findings corroborate previous research: investigators have previously found that being of an older or middle age is associated with an increased likelihood of unemployment in people with SMI [13,14,15,16,17,18,19,20,21,22,23]. This may be a consequence of having more chances of being unemployed compared with younger service users. We found that service users who were not in a relationship were more likely to experience unemployment—this observation has been supported by work in other countries [38,39,40,41,42]. Although men had a higher likelihood of unemployment in the age adjusted model, this association was no longer evident in fully adjusted models, taking into account area deprivation, relationship status, ethnicity, and other clinical factors. Previous research on the relationship between sex and unemployment in SMI samples have also found no evidence of an association [17, 19,20,21, 25, 40, 42,43,44,45,46,47,48].

A key strength of the present study is that, using methodologies developed for large-scale textual analysis, we were able to assess a large sample of almost 20,000 service users. This electronic health record dataset provides ‘real-world’ data and insights [28] into occupation and unemployment for service users accessing mental health services in south London. As the study catchment area includes a high proportion of service users from Black African and Black Caribbean ethnic groups, we were able to undertake comparisons between minority ethnic subgroups, which has in general been limited and not usually possible to this extent. We hypothesized that there would be higher odds of unemployment for ethnic minority groups—this hypothesis was supported for some ethnic minority groups, but not others. Relative to White British service users, we found that Black Caribbean and Black African service users were more likely to have experienced unemployment, after adjusting for other variables, including area deprivation. Irish service users were also more likely to have experienced unemployment in age and sex adjusted models, although this was less apparent in fully adjusted models. We observed no differences between South Asian service users and White British service users in this sample. However, as Indian, Pakistani, and Bangladeshi ethnic groups were grouped as ‘South Asian’ due to smaller sample sizes, differences between these groups may have been masked, and it is a limitation that we could not disaggregate this group further. Overall, these findings suggest ethnic inequalities in employment outcomes for people with SMI are particularly apparent for Black Caribbean and Black African service users, but not Irish or South Asian service users.

Ethnic inequalities in unemployment are also evident in the general population in the UK: individuals from Black ethnic groups are currently twice as likely to be unemployed compared to White British individuals [49], and are more likely to be in precarious, temporary employment, which may exacerbate the risk of unemployment when unwell [50]. In addition, Black Caribbean, and Black African people in the general population have been shown to be more likely to receive lower earnings compared to White British people; these trends have also been noted in people of Pakistani and Bangladeshi origin [50]. A range of factors, including prior experiences of unemployment, younger age, lower education levels, and the impact of cumulative discrimination and racism over the life-course, have been implicated [50,51,52]. Our findings reflect these wider structural inequities, but with the additional focus on SMI impacting racially minoritized groups. Our findings are also consistent with other UK cohort studies: the investigators of the AESOP longitudinal study found that Black Caribbean people with first-episode psychosis were more likely to be unemployed compared with the White British reference group [24, 34].

A limitation of this study is that, by looking at mentions of ‘unemployment’, this may miss some service users who were unemployed where this was not mentioned in the health record. As unemployment was our primary outcome, we excluded patients who were missing occupation data from our sample. This recording and selection bias may impact some groups more than others. For example, older service users of retirement age may only be described as ‘retired’, despite experiencing unemployment prior to retirement age; therefore, unemployment may be disproportionately missed in this group. In a previous study, we found that people with more contacts with services, who may have a more severe course of illness, were more likely to have employment status recorded [33]. This may have partly influenced associations between unemployment and service use variables in the present study. However, our comparison group included service users who had no mentions of unemployment but did have other occupations, which could similarly be better recorded where service users had more contacts. Some other patient groups may be less likely to have occupation recorded and may therefore not be represented in the complete sample, for example those with communication difficulties, although this is likely to be a small number in the sample. Compared to a previous study with all secondary mental health service users, this group of patients with SMI had comparatively lower levels of missing employment data (20.6% in the present study, compared to 43.3% with all service users) [33].

We were unable to evaluate recall rates for the text-mining algorithms, as it was unfeasible to read through the service user’s whole care record when occupation is rarely mentioned in the context of the wider healthcare notes. However, our estimates of occupation recording after deploying the NLP application, which indicated that 79.4% of records had an occupation recorded (Fig. 1), approximated closely to the proportions previously reported in a national audit of SMI patients’ case notes [53]. Occupations should be well recorded for service users with SMI diagnoses, as employment history forms part of a holistic psychiatric assessment, with occupation support for people living with schizophrenia or bipolar disorders evidenced as a quality standard for delivering care [54]. As clinical notes tend to repeat information multiple times across the record [55], this increased opportunities for unemployment to be identified by the application. Furthermore, despite a risk that unemployment may have been under-recorded, our estimates of unemployment were still highly consistent with findings from other studies [1, 2, 56].

The present study was cross-sectional in design by extracting data held in the EHR in January 2020. It would have been desirable to look at patterns of unemployment longitudinally in service users with SMI—this was a key strength in the AESOP longitudinal study of outcomes for people with first-episode psychosis [56]. Limitations of the data and current methodology meant that we were unable to look at unemployment temporally in this study: we were unable to identify when a service user became unemployed, how long they were unemployed, or whether they were recently or currently unemployed. This further detail on unemployment would be of clinical and research interest. There were also other characteristics which could not be examined here—including service user’s educational attainment, and duration of untreated psychosis, which could have explained some of the variation in the analysis [13, 47, 57]. We were, however, able to measure other indicators of illness severity and functioning (inpatient admissions, length of inpatient stay, and compulsory admissions) and found that these indicators for more severe illness had strong and substantial associations with unemployment.

This study was conducted at a single secondary care mental health provider in an inner-city area in the UK, albeit with near-complete coverage of the catchment area of approximately 1.3 million people. The catchment area reflects an urban, ethnically diverse but highly deprived area in the UK [58]. The study catchment may be similar to other metropolitan/urban areas in the UK; however, the findings may be less generalizable to more rural catchment areas.

This study demonstrates that text-mining methodology can be useful to access and analyze the social determinants of mental health conditions in EHR data [59]. These approaches may be further developed in future to enhance understandings of inequalities in routinely collected health records data [60].