“Implementation of the WHO Surgical Safety Checklist was associated with robust reduction in morbidity and length of in-hospital stay and some reduction in mortality,” says the conclusion of a paper in the May 2015 issue of Annals of Surgery.

Now I am not against checklists. When I was a surgical chairman, I implemented and used one in both the operating room and the SICU. They probably do not add costs and may be helpful.

However, there are some problems with the paper. The authors looked at 5,295 operations done in two Norwegian hospitals. The intervention was a 20-item checklist consisting of three critical steps–the sign in before anesthesia, the timeout before the operation began, and the sign out before the surgeon left the operating room. Using a stepped wedge cluster design, patients were randomized to control or the checklist.

Complications occurred in 19.9% of the control patients and 11.5% in those who got the checklist, a significant difference with p<0.001.

A look at Table 2 finds that of 27 complications or groups of complications, 14 occurred in significantly fewer patients in the checklist group.

Of the significant 14, a few, such as cardiac or mechanical implant complications, could possibly have been prevented by the implementation of the checklist.

For most of the others, the relationship between the use of a checklist and a post-operative complication is tenuous. How could a checklist possibly prevent technical complications like bleeding requiring transfusion, surgical wound dehiscence, and unintended punctures or lacerations?

Urinary tract infection, pneumonia, asthma, pleural effusion, dyspnea, and the nebulous categories of “complications after surgical and medical procedures” and “complications to surgery not classified” also occurred significantly more often in the control cohort. What item on a checklist could have prevented those complications?

Embolism, sepsis, and surgical site infection, three complications one would expect a checklist to impact because of reminders to give prophylactic anticoagulation and antibiotics, did not occur at significantly lower rates in the checklist group.

Even the cardiac complication category is open to question because none of the five subcategories (cardiac arrest, arrhythmia, congestive heart failure, acute myocardial infarction) differed significantly between the two groups. Only when the five were combined did statistical significance emerge.

In the 300-bed community hospital, checklist use was associated with a significantly lower mortality rate than non-use, 0.2% vs. 1.9% respectively (p = 0.02), but no mortality difference was seen in the 1100-bed tertiary care hospital.

The tertiary care institution enrolled 3,811 patients, while the 300-bed hospital contributed 1,083. If more patients had been in the latter group, the difference may have disappeared due to the principle of regression to the mean.

Despite the heightened vigilance associated with an ongoing research project, compliance with checklist use was only 73.4%.

Before you go off on me, I will remind you that I do not oppose checklists, and most things we do in medicine are not based on Class 1 evidence.

Just don’t tell me that checklists have been proven to reduce complication rates or save lives.

Skeptical Scalpel is a retired surgeon and was a surgical department chairman and residency program director for many years. He is board-certified in general surgery and critical care and has re-certified in both several times. He blogs at SkepticalScalpel.blogspot.com and tweets as @SkepticScalpel.

Author