In response: A benchmarking scoping review of research output from hospital pharmacy departments in Australia

To the Editor,

We read with interest the article by Penm et al.¹ describing a benchmarking scoping review of research output from principal referral hospital pharmacy departments in Australia. We support the notion that benchmarking research output may assist pharmacy departments to improve their research performance, which has the potential to benefit patients, staff, and the broader health system.^{1, 2} For this to be effective, it is important that benchmarking data be as accurate as possible and presented in a way that enables pharmacy departments to compare themselves with departments in similarly sized health services. Here we propose some ideas that may improve these aspects for future benchmarking studies.

A limitation of the benchmarking study noted by its authors was that the search strategy would not have captured research articles where authors used the name of their local health network instead of the principal hospital in their affiliation.¹ We are concerned that this may have led to a significant under-estimation of pharmacy research output. This is especially likely in Victoria, where each of the principal referral hospitals is part of a health network with one pharmacy department operating across all hospitals in the network. For example, Austin Health has one pharmacy department across its three hospitals, including the principal referral hospital, Austin Hospital. Pharmacy staff rotate between, or work across, the three sites. Research projects often span multiple sites. The affiliation that staff usually use is ‘Pharmacy Department, Austin Health’. Over the 2018–2020 benchmarking study period, the Austin Health Pharmacy Department produced over 50 peer-reviewed research articles that would have met the review's inclusion criteria.³

The affiliation search terms described in Appendix S1 of the benchmarking paper¹ are based on those used by the Australian Institute of Health and Welfare for principal referral hospitals, but they are not necessarily the affiliation terms used by pharmacy department–based authors. The search terms would have missed many articles from Victorian principal referral hospitals at Austin Health, Barwon Health, and Monash Health but would have captured articles from all hospitals within the Alfred, Royal Melbourne, and St Vincent's networks. Hence, the data cannot be used to accurately benchmark Victorian hospitals.

An approach that would ensure data are consistently captured for all principal referral hospitals is to contact each pharmacy department to verify the literature search results. Criteria for which articles are eligible from pharmacy departments that service secondary hospitals in addition to the principal hospital could be developed (e.g. the article must have been produced by one or more pharmacy department staff members who worked at the principal hospital).

Another option is to broaden the search to include local health networks that have a principal referral hospital and a single pharmacy department. This would be consistent with the benchmarking study authors' goal of focusing on health services that were expected to have the highest research output and avoid missing articles from some principal referral hospitals. The Society of Hospital Pharmacists of Australia (SHPA) directory could be used to determine which principal hospitals were part of a health network with one pharmacy department (i.e. has one director of pharmacy, for example Austin Health in Victoria and North Adelaide Local Health Network in South Australia).⁴

Some principal referral hospitals are larger than others, and this will impact on research capacity. Reporting the data in a way that adjusts for the size of the hospital or health network associated with the pharmacy department may make comparisons more valid. For example, research output could be reported as publications per 500 inpatient beds.

Some states have more principal referral hospitals than others (e.g. New South Wales has 11, while Australian Capital Territory, Northern Territory, and Tasmania each have one). Standardising the data in Table 3 of Penm et al.¹ according to the number of principal referral hospitals or beds in that state would also facilitate valid state comparisons.

We applaud the authors of the benchmarking paper for undertaking this work and hope that our comments can inform future benchmarking studies for principal referral hospitals. To encourage all pharmacy departments to publish their research and enable them to benchmark their output against similar hospitals, future studies might also include non-principal referral hospitals.

This letter does not contain any studies involving human participants performed by the authors.