1 INTRODUCTION

Pay-for-performance (P4P) refers to the “transfer of money or material goods conditioned by the fulfilment of a measurable action or a predetermined performance goal”.1 It is used to improve quality2-4 and access to health services. Historically, it has been used at the hospital level5, 6 and in primary health care7 in several countries.4, 7-10 In Brazil, a few studies reported local experiences of P4P at municipal level,11, 12 but in the national context, P4P has been implemented through the Program for the Improvement of Access and Quality (PMAQ). An arm of PMAQ is the evaluation of Dental Specialities Centres (in Brazil referred to as Centro de Especialidades Odontológicas or CEO) that are part of the Brazilian National Health System (SUS). Each CEO's performance was evaluated based on three domains: (a) self-evaluation, (b) ability to meet goals and (c) local external evaluation of the centre structure, processes and patient satisfaction. This strategy adopted by managers aims to improve the performance of specialized services, improving the health of the population.13

Some studies describe several potential advantages of P4P programs in health services. For example, they have described increased motivation and production, reduced absenteeism, teamwork qualification, clinical data recording and work process monitoring.5, 12, 14 A systematic review of P4P found several studies using different outcomes and contradictory results.15 The existing evidence on the effectiveness of pay-for-performance programs in dental services is quite limited,16, 17 and reveal small to moderate increase in the production of dental procedures.

The rates of clinical procedures have been used as a proxy of use and coverage by services, and the factors associated with it have been described in earlier studies.18-25 In Brazil, prior evidence showed that cities with larger CEOs, a higher number of dentists, higher coverage of Oral Health Team (ESB) at primary health care, different population sizes and higher per capita health expenditure were associated with a higher per capita number of dental procedures.18, 22, 24-26 A large network of secondary dental care could be related to extensive coverage of ESB at primary health care because the former is expected to uptake patients by referrals only. Indeed, higher ESB coverage was more strongly associated with fewer tooth extractions in places with CEO than in those without it, likely due to the presence of the CEO referral service where options other than tooth extractions are available.27

However, previous studies found that smaller cities have higher ESB coverage, while larger cities have higher secondary care coverage with good performance.18, 25, 26 Finally, a recent multilevel study showed that private dental insurance coverage was associated with less frequent use of public dental care.28

The PMAQ-CEO has as its objective “to increase access and improve quality of CEOs”.13, 29 The program should give priority to evaluating the impact of incentives. In current arrangements, municipalities do not need to transfer the financial incentive to professionals; they can use it to provide technical assistance, supplies, infrastructure and other resources that may positively affect performance.30 To date, no study seems to have evaluated the effect of the program in the overall production of specialized dental procedures by the CEO. Therefore, this study's objective was to evaluate the impact of a pay-for-performance program on changes in the number of dental procedures performed by public secondary dental care services in Brazil.

2 METHODS

A longitudinal study was performed, having all public secondary dental care services in Brazil as the units of analysis. Two time periods were considered (a) July 2011 to June 2013, before the PMAQ-CEO was launched (baseline); and (b) July 2015 to June 2017, the period after the CEO were informed about their performance.

The data used were extracted from the Department of Informatics of the Brazilian National Health System (DATASUS) using the following information systems: Ambulatory Health Information System (SIA-SUS), National Register of Health Facilities (CNES) and Public Health Budget Information System (SIOPS). Data from the Brazilian Institute of Geography and Statistics (IBGE) and microdata from the external evaluation of the first cycle of PMAQ-CEO. Data extraction on dental procedures by specialized dental services was carried out using the statistical software R version 3.6.2 (microdatasus package).

The outcome was the increase of specialized dental procedures between 2011/2013 and 2015/2017 using registered information in the SIA-SUS system. We considered three specific groups (endodontics, periodontics and surgery) and the total number of dental procedures (sum of the three groups). The codes for each procedure used in the current study were those described in the Ordinance nº 1464, 18 July 2011 that set the production target by the CEOs type. Preliminary analysis showed that the outcome variable had considerable random variation and several outliers, not making linear models a good choice due to poor fit. Therefore, the outcome variable was categorized as 0 = ‘no increase or reduction’ or 1 = ‘increase’.

Secondary dental care services qualified as CEO receive a federal financial incentive for maintenance, and municipal resources secure any non-CEO service. As part of the PMAQ-CEO, all CEOs were eligible for a further financial incentive based on their performance according to Ordinance no 677, 18 June 2015. Such incentive means that a CEO could double their maintenance incentive. Among the 932 eligible CEOs, 802 participated in PMAQ-CEO, and 50% (n = 402) having the lowest scores were classified as average or below, 34% (n = 273) of them were rated above average, and 16% (n = 127) were classified as far above average. Therefore, the 802 CEOs received pay-for-performance incentives, while the 379 non-CEOs (local public polyclinics without any federal financial support) were identified in the CNES system based on their specialized oral health production in the period studied. There were 112 CEOs that chose not to participate in the PMAQ-CEO, and 18 were classified as having unsatisfactory performance (receiving no incentive). Consequently, our main exposure variable had five categories: 0—No incentive (neither CEO nor PMAQ), 1—No PMAQ incentive (CEOs not participating in PMAQ-CEO or having unsatisfactory rating), 2—PMAQ-CEO incentive of 20% over maintenance values (average rating or below), 3—PMAQ-CEO incentive of 60% over maintenance values (above average rating), 4—PMAQ-CEO incentive of 100% over maintenance values (rating well above average).

The following co-variables were selected for the analysis: (a) municipal coverage of Oral Health Teams (ESB) in Primary Healthcare in 2014, obtained as continuous and categorized into up to 60% coverage, 60%–80%, and more than 80%; (b) per capita health expenditures in public health, in Brazilian Reais in 2014, categorized by quartiles; (c) municipal population in 2014, categorized into up to 50 000, 50 001 to 100 000, 100 001 to 500 000 thousand and over 500 000 inhabitants; (d) increase in coverage of private dental insurance at the municipal level from 2011–2013 to 2015–2017, categorized in reduced, increased up to 1% point, or increased >1% point; (e) type of specialized dental service, categorized as non-CEO (other local public polyclinics without federal support) or CEO types I, II, or III; and (f) the equivalent number of part-time (20-h/week) specialist dentists (endodontics, periodontics and surgery) with the difference between baseline and follow-up calculated to generate the following categories, 0 = ‘remained the same number or decreased’ or 1 = ‘increased’.

2.1 Statistical analyses

The categorical variables were described as absolute and relative frequencies. Bivariate analyses were performed to evaluate the association between the increase in production and independent variables using chi-square tests. Crude and adjusted analyses were modelled using logistic regression. Interactions between the incentive variable and CEO type, population size and ESB coverage were tested.

A directed acyclic graph representing relations among selected variables was organized to depict important municipal-level variables described in the introduction and associated with dental care use (Figure 1). In that model, only city size is an exogenous variable, and all variables had previously been associated with the outcome; however, the evidence for their interrelation is weak and uncertain. Therefore, it is unknown which variables may be mediators or confounders, and all were included in an initial analytical regression model. The stepwise backward technique was applied to modelling, removing variables with the highest p-value >.20 one by one until a final model was obtained. Odds ratios, 95% CIs and p-values of the variables were presented. The goodness of fit of the final model was evaluated using the Hosmer-Lemeshow test (ResourceSelection package, hoslem.test command). The data were tabulated and analysed with the statistics software R version 3.6.2.

Directed acyclic graph representing relations among variables in analytical model [Colour figure can be viewed at wileyonlinelibrary.com] 3 RESULTS

The use of secondary care increased in 634 (48.4%) of the 1311 dental services evaluated. There was an increase in the average of part-time (20-h/week) specialist dentists in 54.1% of the services. Some 49.7% of the CEO increased production. Most of the services were located in municipalities with coverage of ESB on until 60%. The average annual total health expenditure per inhabitant in 2014 was R$ 646.8 (~US$ 275). Approximately half of the services, 54.7%, are located in municipalities with up to 100,000 inhabitants. In the bivariate analysis, the variables financial incentive, number of specialist dentists presented a significant association with the outcome (Table 1).

TABLE 1. Distribution of specialized healthcare services and changes in dental procedures in Brazil between 2015–2017 and 2011–2013 Variables n Changes in specialized dental procedures Remained or decreased Increased n % n % Financial incentivea,b, a,b No incentive (non-CEO) 379 28.9 210 55.4 169 44.6 Only CEO maintenance 130 9.9 62 47.7 68 52.3 20% over maintenance 402 30.7 216 53.7 186 46.3 60% over maintenance 273 20.8 137 50.2 136 49.8 100% over maintenance 127 9.7 52 40.9 75 59.1 Specialist dentists (all specialities)a Equal or decrease 602 45.9 380 63.1 222 36.9 Increase 709 54.1 297 41.9 412 58.1 Healthcare service type Non-CEO service 385 29.4 214 55.6 171 44.4 CEO Type I 346 26.4 170 49.1 176 50.9 CEO Type II 471 35.9 236 50.1 235 49.9 CEO Type III 109 8.3 57 52.3 52 47.7 Municipal coverage of ESB Up to 60% 901 68.7 469 52.1 432 47.9 60.1%–80% 151 11.5 86 57.0 65 43.0 80.1% or more 259 19.8 122 47.1 137 52.9 Total health expenditure per capita Up to R$ 450.3 328 25.0 176 53.7 152 46.3 R$ 450.4 to R$ 592.9 328 25.0 169 51.5 159 48.5 R$ 593 to R$ 732.7 327 24.9 171 52.3 156 47.7 R$ 732.8 to R$ 3711 328 25.0 161 49.1 167 50.9 City size (inhabitants) <50 thousand 473 36.1 242 51.2 231 48.8 50 to 100 thousand 244 18.6 122 50.0 122 50.0 >100 to 500 thousand 352 26.8 186 52.8 166 47.2 >500 thousand 242 18.5 127 52.5 115 47.5 Private dental insurance Reduced 178 13.6 83 46.6 95 53.4 Increased up to 1% point 404 30.8 209 51.7 195 48.3 Increased more than 1% point 729 55.6 385 52.8 344 47.2

In the crude analysis, the financial incentive was significantly associated (p = .04) with a higher probability of increment in the production of dental procedures (Table 2). Also, services receiving incentives (20%, 60% and 100%) were more likely to have increased their production than those non-CEOs. However, CEOs that did not receive the pay-for-performance incentive were also more likely to increase their production relative to polyclinics or clinics/speciality centres that are not CEOs. Associations between production and type of services were similar regardless of procedure (Periodontics, Surgery, Endodontics) and are shown and Table S1.

TABLE 2. Logistic regression analysis of associated variables with the increase in dental procedures performed in specialized Brazilian healthcare services between 2015–2017 and 2011–2013 Variables Crude analysis Adjusted analysis OR 95% CI p-Value OR 95% CI p-Value Financial incentivea No incentive (non-CEO) 1 .04 1 .12 Only CEO maintenance 1.36 0.91 2.03 1.22 0.81 1.84 20% over maintenance 1.07 0.81 1.42 0.98 0.73 1.31 60% over maintenance 1.23 0.90 1.69 1.10 0.80 1.51 100% over maintenance 1.79 1.19 2.69 1.65 1.09 2.51 Specialist dentists (all specialities) Equal or decrease 1 <.01 1 <.01 Increase 2.37 1.90 2.97 2.35 1.88 2.94 Healthcare service type Non-CEO service 1 .30 CEO Type I 1.30 0.97 1.73 CEO Type II 1.25 0.95 1.63 CEO Type III 1.14 0.75 1.75 Municipal coverage of ESB Every extra 10% point 1.02 0.99 1.05 .27 0.27 Total health expenditure per capita Every extra R$ 100 1.00 0.97 1.03 .79 0.79 City size Every 1000 inhabitants 1.00 0.99 1.01 .14 0.14 Private dental insurance Every increase in 1% point 0.98 0.94 1.02 .36 0.36 Note The chunk test to define the p-value of multinomial variables is chi-square.

Services that increased the number of specialist dentists more than doubled the probability of increasing their production than those that maintained the same number of professionals or reduced it. These findings were observed in the crude and adjusted models for all outcomes and were statistically significant (p < .01). The probability of an increase in the production of periodontal procedures almost tripled in the adjusted model (Table S1).

The increase in private dental care insurance coverage at the municipal level was on average 1.7% and 55.6% of municipalities had an increase higher than 1%. The higher the private dental insurance coverage, the lower the probability of an increase in production in the crude analysis (p = .36). In municipalities that reduced private dental insurance coverage, 53.4% of services increased their production, while in municipalities with up to 1% point increase and >1% point increase in private dental insurance there was an increase of 48.3% and 47.2%, respectively. In the crude model of total procedures, every increase of 1% in private dental insurance was associated with an OR = 0.98 (95%CI: 0.94–1.02, Table 2) of decreasing the production in public service.

In all fully adjusted models, the CEOs that received 100% of the financial incentive from the PMAQ-CEO were more likely to increase production than non-CEOs, although this was not statistically significant after the adjustment. However, CEOs only maintenance were also more likely to increase production relative to non-CEOs for periodontal (OR = 1.47, 95%CI: 0.97–2.24), endodontic (OR = 1.49, 95%CI: 0.98–2.26) and total procedures (OR = 1.22, 95%CI: 0.81–1.84; Tables 2 and S1). As for the tested interactions, all had p > .05. On the other hand, the Hosmer-Lemeshow test for model adjustment was significant, that is the saturated model is not the same as the model under study, revealing adjustment failures.

4 DISCUSSION

This study showed a gradient of increase in the production of dental procedures by CEOs who received pay-for-performance financial incentives. CEOs that received higher incentives were more likely to increase production than any of the control groups; however, caution is needed because the association was not strong and lost statistical significance after adjustment. Finally, our findings showed that services that increased the number of specialized dentists were more likely to increase their production of dental procedures.

Dental Specialities Centres that received an incentive of 100% were 1.65 times more likely to increase their production when compared with public polyclinics or services not accredited as CEOs. This finding confirms the perception of Curitiba's professionals, who considered that a similar P4P program, targeted at Primary Health Care, promoted a moderate or substantial increase in production.12 In Chile, P4P was associated with a 24% increase in the rate of dental treatments completed on 6-year-old children.31 The CEOs that received an incentive of 20% and 60% were less likely to increase production than those that received 100% incentive. One explanation may be that a lower or insufficient incentive is not able to influence production. The PMAQ-CEO incentive goes to the municipal health system, not the CEO staff (dentists and auxiliaries), then, managers can purchase equipment, supplies, or spend it on any other structural changes but also increase salaries. However, a smaller incentive may have little capacity to stimulate structural changes. The amount of the financial incentive has already been discussed in other studies, and a big incentive can bring about significant changes and increase the risk of unintended effects, such as misreporting or fraud.32 Considering the demands from managers to make public health services meeting goals and targets, in addition to the absence of audit/monitoring of registered procedures, then some over-notification to reach the goal can be an unintended side effect instead of the patients' overtreatment reported in the remuneration per procedure systems. A systematic review assessed the effects of P4P on health care in general (not including dental services) and stressed that more research is needed on the dose-response relationship of incentives.33 Importantly, CEOs with 100% incentive doubled their resources, creating inequities among CEO that can result in a larger performance gap.30 As such, the incentive may not have removed barriers that generated inequalities in production.

The strengths of this study include the use of two control groups and its longitudinal design. In addition, the external evaluation of the PMAQ-CEO was intended to include all CEOs, and we added possibly all non-CEO secondary dental care services in the country, making this a census and, therefore, the observed effect is unlikely due to sampling variation. Considering that specialized dental care is also provided at centres other than CEOs, having such a control group made results more robust. Having CEOs as a unit of analysis allows comparison to other studies.18, 24 Another positive point was the standardization of the workload of dentists. In a previous study, where managers were questioned about dentists' average workload, there was no standard.24 Moreover, all the information was obtained from official sources that local managers can also use. Among limitations, there may be notification errors in the information systems; also the outcome was dichotomized to minimize the effect of extreme values which may result in lost of information. Finally, residual confounding is still possible as few variables were good predictors, and the theoretical model is still exploratory due to a lack of evidence on contextual level variables. After a theoretical review, we did not identify any other known and available variables to be used.

In conclusion, the PMAQ-CEO performance incentive may be positive when fully deployed to improve services. Some municipalities have chosen, through municipal legislation, to compensate professionals with a percentage of the financial incentive.30 There is no consensus in the literature about the best option,33 but it is possible to define the most appropriate way collectively by management and professionals in partnership with users.32, 34 In this case, transparency and consistency in the use of resources would be monitored so that the incentive would be genuinely reinvested in secondary dental care. The duration of the effect is unknown, but some studies suggest that the payment should be close to the evaluation.30, 35 The PMAQ-CEO proposed to make an evaluation every 2 years. The institutionalization of a permanent performance committee5 at the national and municipal levels could monitor and improve the program. It could identify teams with greater difficulties and find ways to qualify them, reducing barriers that hinder growth and quality improvement. Also, it could monitor and evaluate performance indicators, qualifying information systems. Future research on unmet targets should include all actors (managers, users and professionals) to identify barriers to work and access and to monitor if such incentives decrease inequalities15 and give more access to those who need most. Finally, mechanisms explaining the reported association must be further investigated.

ACKNOWLEDGEMENTS

We receive no financial support for this work. RKC and FNH received a PQ2 fellowship from the National Research Council (CNPQ 311592/2019-8).

AUTHOR CONTRIBUTIONS

FSC participated in the conception of the study, analysis and interpretation of data, wrote the manuscript, critically reviewed the content and approved the final version. FNH participated in the conception of the study, critically reviewed the content and approved the final version. RKC participated in the conception of the study, in the analysis and interpretation of data, wrote the study, critically reviewed the content and approved the final version.

Data openly available in a public repository that does not issue DOIs. The data that support the findings of this study are openly available in http://aps.saude.gov.br/ape/pmaq.

Filename Description cdoe12717-sup-0001-TableS1.docxWord 2007 document , 10.9 KB Table S1

Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.

REFERENCES