In the last Brief, we reported on a new study from the University of Illinois finding that their wellness program had “no causal effect on the activities, health, productivity and medical spending” for its 5,000 participants. This negative study led to an expected storm of headlines in the media, summed up by the one from Bloomberg: “Workplace Wellness Programs Really Don’t Work” – letting one study stand in for all.
It’s created much discussion. HERO, the non-profit dedicated to identifying best practices in the field of workplace health, has written an important rebuttal available below. And experts from the Global Wellness Institute, including members of the Wellness at Work Initiative, chaired by Renee Moorefield PhD (CEO, Wisdom Works) and the Mental Wellness Initiative, chaired by Prof. Gerry Bodeker (University of Oxford, Columbia University) have given important input. For instance, Kenneth Pelletier MD (Clinical Professor of Medicine, UCSF) and Dr. Fikry Isaac (Founder & CEO, WellWorld Consulting and former chief medical officer, Johnson & Johnson) explain in a nutshell why this study, praised for its methodology as a randomized controlled trial, is actually essentially flawed…
HERO noted that despite a large and growing evidence base demonstrating the positive outcomes of best-practice health and well-being initiatives in the workplace, media coverage of occasional studies with negative findings sometimes creates confusion among employers about what really works in wellness.
So they created a commentary to better assess such findings by providing tips on how to critically examine research on wellness program effectiveness. These tips were applied to the University of Illinois Workplace Wellness Study, and provide a strong rebuttal.
Dr. Pelletier noted that this was a bold attempt for a good study, but was so methodologically flawed that it simply isn’t of much use.
Here is his overarching critique…
- This ONE study appears to refute a huge body of countervailing research. Either they have discovered the secret of the universe or they are just wrong. In this case … it is clearly the latter!
- Employees were randomized but there is no consideration in the study of the widely accepted “best practice” components such as health supporting policies, leadership support, and/or comprehensive communication strategies. ALL of which are required for any workplace wellness program to work.
- Program duration was less than a year. Simply not enough time for any changes to take place or be measured.
- Interventions were VERY limited (not a “state of the art” wellness program) and participation was measured by being in any one of the array of courses offered. Researchers offer no evidence to support their hypothesis that such a limited “intervention” can be expected to result in change. As he study demonstrated … it predictably did not.
- As for their claim of the impact of “screening” and “advantageous selection” … there is ZERO evidence that there was higher participation by higher wage earners. This error in the study is known as “confirmation bias” and is a common error.
- They cite the spending differences in ONE year of a multiyear program. Again … that is never done in worksite (or any other) R&D. Since the University of Illinois has on an ongoing wellness program, the researchers could create a true control group but have not done so.
Dr. Fikry Isaac stresses that he believes that the study suffered from the same key flaws as outlined by Dr. Pelletier – and adds:
- Establishing a “culture“ of health and wellbeing is a critical success factor in these programs vs. providing “some” intervention for such a short period of time.
- In several of the studies HERO has published over the past ten years on value and impact, you don’t see such impact until year 3 of implementation, as you are looking for impact on care cost/claims as well as improvement in lifestyle factors all together. (HERO studies were cited in this paper.)
Dr. Pelletier notes that each of the flaws he outlined requires more elaboration and that there are many other errors in the study than the ones he pointed out above. He called its results a “tempest in a teapot”, commenting that there are always ill-conceived studies that get PR. The Univ. of Illinois study will not be the last.
The upside: the study provides so many lessons for a future more methodologically sound study. And what came to life as a provocatively negative headline – workplace wellness programs don’t work! – has quickly evolved into a more meaningful dialogue about what needs to be in place in a study to make meaningful conclusions about outcomes.
Real dialogue will always outlast a headline. What do you think?
I fully concur with Dr Pelletier’s comment, policy, leadership support and a comprehensive communication strategy are key to the success of the programme. I wish to add that this should be underpinned by a proper needs assessment for buy in. As much innovation has a part to play to motivate workers they also like to see that they are listened to and that is the role of a needs assessment report in developing and launching a programme.