Developing and evaluating a tool to measure general practice productivity: a multimethod study

Abstract

Systems for measuring the performance of general practices are extremely limited. The aim was to develop, pilot test and evaluate a measure of productivity that can be applied across all typical general practices in England, and that may result in improvements in practice, thereby leading to better patient outcomes. Stage 1 – the approach used was based on the Productivity Measurement and Enhancement System (ProMES). Through 16 workshops with 80 general practice staff and 72 patient representatives, the objectives of general practices were identified, as were indicators that could measure these objectives and systems to convert the indicators into an effectiveness score and a productivity index. This was followed by a consensus exercise involving a face-to-face meeting with 16 stakeholders and an online survey with 27 respondents. An online version of the tool [termed the General Practice Effectiveness Tool (GPET)] and detailed guidance were created. Stage 2 – 51 practices were trained to use the GPET for up to 6 months, entering data on each indicator monthly and getting automated feedback on changes in effectiveness over time. The feasibility and acceptability of the GPET were examined via 38 telephone interviews with practice representatives, an online survey of practice managers and two focus groups with patient representatives. The workshops resulted in 11 objectives across four performance areas: (1) clinical care, (2) practice management, (3) patient focus and (4) external focus. These were measured by 52 indicators, gathered from clinical information systems, practice records, checklists, a short patient questionnaire and a short staff questionnaire. The consensus exercise suggested that this model was appropriate, but that the tool would be of more benefit in tracking productivity within practices than in performance management. Thirty-eight out of 51 practices provided monthly data, but only 28 practices did so for the full period. Limited time and personnel changes made participation difficult for some. Over the pilot period, practice effectiveness increased significantly. Perceptions of the GPET were varied. Usefulness was given an average rating of 4.5 out of 10.0. Ease of use was more positive, scoring 5.6 out of 10.0. Five indicators were highlighted as problematic to gather, and 27% of practices had difficulties entering data. Feedback from interviews suggested difficulties using the online system and finding time to make use of feedback. Most practices could not provide sufficient monthly financial data to calculate a conventional productivity index. It was not possible to create a measure that provides comparability between all practices, and most practices could not provide sufficient financial data to create a productivity index, leaving an effectiveness measure instead. Having a relatively small number of practices, with no control group, limited this study, and there was a limited timescale for the testing and evaluation. The GPET has demonstrated some viability as a tool to aid practice improvement. The model devised could serve as a basis for measuring effectiveness in general practice more widely. Some additional research is needed to refine the GPET. Enhanced testing with a control sample would evaluate whether or not it is the use of the GPET that leads to improved performance. The National Institute for Health Research Health Services and Delivery Research programme.