London School of Hygiene & Tropical Medicine

These are the findings of new research from the London School of Hygiene & Tropical Medicine, published in the Lancet.

“Our study reveals that mortality rates do not reliably reflect surgeon performance when the number of procedures is low,” said lead author Dr Kate Walker, Lecturer of Medical Statistics at the School.

“For example, for cardiac surgery, surgeons perform around 100 procedures each year, whereas for bowel cancer surgery, the typical case load is around 10 per year. When you are talking about one, two or three deaths, then chance factors will overwhelm the impact of surgeon performance on the mortality rate. In other words, low volumes lead to false complacency.”

Using national mortality data for adult cardiac surgery and key operations in three other specialties (bowel cancer resection, oesophagectomy or gastrectomy, and hip fracture surgery), the research team calculated how many procedures would be necessary for reliable detection of poor performance and how many surgeons in English NHS hospitals actually do that number of operations.

They found that the numbers of operations needed for the statistical power necessary to detect truly poor performance exceed the annual number of procedures typically done by surgeons.

For example, they estimated that for bowel cancer surgeries, to achieve even 60% statistical power (meaning that of 10 surgeons that were truly performing poorly, on average six would be identified) the annual median number of surgeries would need to be 10 times higher than it currently is.

When looking at three years of data, about three-quarters of surgeons do sufficient numbers of hip fracture and cardiac operations every year to identify cases of poor performance. For bowel cancer resection and oesophagectomy or gastrectomy procedures the percentages were just 17% and 9% respectively, and rose to about a third of surgeons based on five years of data.

It was hoped that public disclosure of surgeon performance would provide transparency, help patients choose the best surgeons, and provide an incentive to improve the quality of care. But such reporting is only useful if it contains accurate information.

 “The reporting of results for individual surgeons should be based on outcomes that are fairly frequent, and fortunately, from the point of view of patients, mortality is not one of them,” said Dr Jenny Neuburger, one of the researchers who worked on the study.

“For specialties in which most surgeons do not perform sufficient numbers of operations to reliably assess their outcomes, reporting should be at the level of the surgical team or hospital, and not the surgeon.”

Publication:

• Kate Walker, Jenny Neuburger, Oliver Groene, David Cromwell, Jan ven der Meulen, Public reporting of surgeon outcomes: low numbers of procedures lead to false complacency, Lancet. DOI: 10.1016/S0140-6736(13)61491-9