Is the NCCN Undermining its Credibility by Failing to Acknowledge Bias and Value in its Guidelines?

In Policy and Practice by H. Jack West, MD0 Comments

The ostensible role for the National Comprehensive Cancer Network (NCCN) is laudable. Essentially, this consortium of 27 leading cancer centers organizes a large group of practice committees with experts in a wide range of cancer subtypes and clinical challenges like supportive care issues, specific complications, age-related challenges, and cancer prevention and screening. These expert committees review the data and create an ever-evolving collection of evidence-based guidelines along with an indication for each recommendation of the quality of the evidence and the strength of consensus supporting it.  The NCCN guidelines are arguably the most widely consulted and recognized benchmark for practicing clinicians to follow what should be considered a standard of care; importantly, they are also used as a benchmark for what payers should be expected to pay for.  They are a remarkably important and helpful tool, with complexity of cancer care escalating and new tests and treatments emerging seemingly by the week.  As it becomes all but impossible for any human (but not necessarily artificial intelligence platforms) to keep up with the deluge of new data and evolving best practices, we should expect to rely on expert-defined algorithms for best practices more and more.

It is important, however, to recognize the limitations of these guidelines. First, the NCCN itself is supported by a wide range of corporate sponsors who have a clear stake in the recommendations of the committees. They did not come down from God, written on stone tablets. They are a product of human deliberation, with the committees comprised of members with their own potential conflicts of interest. Traditionally, we have viewed conflicts rather narrowly, based on money institutions or individuals receive from companies, but biases are not limited to that, and members and their employing institutions also stand to gain profoundly by recommendations that promote their careers and research efforts and emphasize management strategies that are preferentially available at their centers.  Beyond the direct and indirect biases of the members, my colleagues who participate directly often describe how the idea of consensus sounds nice but too often ends up as “groupthink” dominated by the loudest and most opinionated members (as opposed to typical corporate “HIPPO” decision making shaped by the HIghest Paid Person’s Opinion) rather than true convergence of independent perspectives.


There is a danger of the NCCN undermining its mission of defining the optimal treatment for patients if the recommendations run ahead of the evidence, especially when its committees make no pretense of considering the value of their recommended interventions.  In their defense, I believe that many members see a key role of the NCCN guidelines as being as broadly supportive of a broad latitude of choices as possible, in order to strongly encourage if not mandate payers to cover most of what an oncologist and patient may hope to pursue, provided that there is a premise of evidence to support a strategy.  I think there are two key challenges here.

1) First, it is important to have NCCN members not overstate the case in favor of any approach that also happens to serve their personal and institutional best interests. To take an example from the setting of lung cancer, the current version of the NCCN guidelines in non-small cell lung cancer (NSCLC) “strongly endorses” broad genomic testing of advanced lung cancers in hopes of identifying and learning optimal treatments for rare molecular drivers.  This recommendation may just be a well-intended statement to ensure that clinicians wishing to get next generation sequencing (NGS) testing for a broad array of molecular markers covered by an insurer, but it is heavy-handed to imply, as I believe the current guidelines do, that NGS testing is or should be a standard of care. There is no evidence that this approach can improve survival in a population of patients who were tested with NGS or another broad testing mechanism compared with a population of patients who received standard and more limited testing for the markers of established value in guiding treatment decisions.  Moreover, there is also a very real risk that oncologists and patients will make hand-waving treatment decisions that amount to creative guesswork of the right treatment for rare mutations that will actually prove to be less effective, more costly, and more toxic than well established but standard therapies that are now dismissed as last year’s model. We cannot promote broad molecular testing based on a faith-based conviction that it is the right approach. The NCCN guidelines for NSCLC also note that broad molecular testing is the best way to accelerate research, but I would contend that it is subjugating the appropriate use of the NCCN in defining standards of care to make recommendations that essentially strong-arm clinicians to go beyond the confines of current clinical evidence.  It is also quite relevant that the individuals and institutions whose research, career trajectories, and institutional marketing efforts best served by promoting molecular testing beyond its established benefit are the ones articulating these guidelines. Though I would not presume that this is a conscious motivating factor for the NCCN members, the fact that their biases and self interests align with these recommendations should only lead us to hope to keep them grounded in solid data.

2) Even if NCCN members are judicious in using compelling evidence in forming these guidelines, we should now be forced to recognize that as the costs of these tests and treatments rise dramatically every year, there must be sufficient consideration of the relative value of recommended interventions that form an evolving standard of care. As a rule, the current members of these committees, usually sub-specialists in a particular cancer, have no focus or even interest in health care policy are not qualified to dictate how society must spend finite resources.  Just as we should readily recognize that patients with a particular disease have an implicit profound bias about how many resources should be allocated to the management of their disease, we should belatedly recognize that experts in lung cancer cannot objectively weigh the value of additional resources directed to treating advanced lung cancer versus funds being spent for another cancer, for prevention, or simply not allocated — the last option allowing families to hope for a year without a dramatic escalation in their health care premiums raised unilaterally because of costs incurred by people who don’t see or consider the consequences of their decisions.  Accordingly, the standard of care, which may potentially include combinations of therapies costing $15,000/month for each one, without any endpoint, cannot be defined by a group that does not acknowledge the zero sum game of our health care system and the reality that there are other potentially valuable ways to spend financial resources than maximal interventions for the patients with one particular cancer.

Cancer care will only rely increasingly on guidelines and algorithms to shape management of patients, and with the NCCN as the leading contributor of recognized guidelines, we need to ensure that this group protects its credibility by being mindful of the appearance of conflict of interest, not just financial but in terms of professional/career incentives, and that defining standards of care must now include consideration of the costs of recommended interventions.

What do you think? Do others share these concerns?


Leave a Reply