Death data not good enough to spot poor surgeons

Posted 07/07/2013

Published death rates for individual surgeons will not spot poor performers in certain fields because too few operations have taken place. In June, the NHS published the first data including death rates for vascular surgeons with nine more specialist areas to follow. The paper says however that it would be better if some data was shown per hospital.NHS England said that the reason for the initiative was transparency and not to try to spot poorly performing doctors.The team who looked at the data from the London School of Hygiene and Tropical Medicine used it to look at death rates in four specialist areas. These were adult heart surgery, bowel cancer resection (removing part of the colon or rectum), oesophagectomy (removing the oesophagus) and gastrectomy (taking out all or part of the stomach) and hip fracture surgery.The study then assessed how many procedures would be necessary for reliable detection of poor performance and how many surgeons in English NHS hospitals actually do that number of operations. The researchers concluded that enough heart and hip operations were carried out to make death rate data statistically reliable but they suggested that the same was not true for the other categories studied. For example the median number of bowel cancer resections carried out by an individual surgeon was nine, meaning the number was too small and too easily affected by a range of other factors, such as one surgeon doing more complex operations than their peers on sicker patients. In addition, death rates after hip surgery are extremely low and so would not give a fair reflection of the procedures success.Other factors were that in many areas individual performance data would not be robust, they may give a false sense of complacency because an absence of evidence could be falsely interpreted as evidence of an acceptable performance.It is said that for specialities in which most surgeons do not perform sufficient numbers of operations to reliably assess their outcomes, reporting should be at the level of the surgical team or hospital and not the surgeon.Professor Jan Van Der Meulen who also worked on the paper, said there should be as much transparency as possible. He also said it is not just the surgeon who has an impact, it is the whole care package, the whole team and the pre and post-operative care. Professor Norman Williams, President of the Royal College of Surgeons, said: “The driving force behind the publication of surgeon level outcome data is to improve surgical performance and allow patients to find out more about the quality of care provided by hospitals and individual surgeons”. He said it was an extremely challenging project and particularly complex for cancer operations where there were low volumes of surgery and heralded the publication of data as “the beginning of a new approach to transparency and healthcare”.Professor Sir Bruce Keo, Medical Director at NHS England, said: “The authors have misunderstood the purpose of public disclosure of information. It is not a statistical exercise to identify poor performers, rather it is an exercise to provide information on activity and outcomes to focus minds on improving results and to reassure the public that the NHS provides high quality surgery. It does raise the question as to whether it is better to have your complex operation performed by a surgeon doing a lot of that operation or only a few.Julie Crossley a medical injury lawyer at Ashtons Legal, adds: “Obviously this data is not reliable at present and needs to be refined before it can be relied upon.”