March 27, 2014
Good afternoon. My name is Laurén Doamekpor, and I am a senior fellow at the National Research Center for Women & Families. Our non-profit research center assesses scientific and medical data and provides objective health information to patients, providers and policy makers. Our organization does not accept funding from drug companies, and therefore I have no conflicts of interest.
We have carefully reviewed the data provided to your committee by the sponsor and the FDA and are here to share our perspective and concerns.
As you can see, Serelaxin had NO benefit in terms of one of the primary endpoints: shortness of breath during 3 time points in the first 24 hours. It had no benefit in terms of the two secondary endpoints – which really are the crucial outcome measures: days alive and out of the hospital through Day 60 OR in terms of cardiovascular death or rehospitalization from heart failure or renal failure through Day 60.
The so-called success of the RELAX-AHF clinical trial is based on just one primary end point: patients on Serelaxin reported more improvement in shortness of breath from baseline through Day 5 compared to the placebo. This was only statistically significant because of an imputation protocol that was inappropriate and methodologically unsound.
In addition, this measure is a subjective measurement that has been found in the research literature to not be very accurate for measuring changes due to treatment. (We will provide those citations to the FDA, which show, for example, a test-retest reliability score that is only half as reliable as what is considered “satisfactory.”) Those of you who have taken courses on research methods know that a 7-point Likert scale is more accurate because the gradations are greater than for the 100-point VAS. (That’s because most people can’t really distinguish between discomfort rated as 33 compared to 43 on a 100-point scale, but they can distinguish between 4 and 6 on a 7-point scale.)
The VAS can provide useful information but also can have lots of “false positives” in terms of improvement. Scores can change as patients get used to being short of breath, for example, not because they are getting better. We agree with FDA scientists that the results of this test – whether statistically significant or not – is unlikely to be clinically meaningful.
Given these methodological problems, NONE of us can have any confidence that this drug will actually benefit patients.
That may seem harsh, but as many of you know, clinical trials must be designed correctly, patient outcomes must be measured using very accurate instruments (which VAS is not), and results must be analyzed correctly to be meaningful. It is always possible to get what seems like statistically significant results if the data are manipulated in particular ways.
It doesn’t matter if the manipulation of data is intentional or not – in this study, the only significant finding used the weakest scale and was contaminated by the imputation protocol. The other 3 endpoints showed no improvement. And 2 of those endpoints were the ones that really matter: survival and staying out of the hospital for 60 days.
As noted in the FDA’s review, the imputation protocol allowed subjects with worsening heart failure to be given the VAS score of 0 for the remainder of the observation period. As a result, this end point did not take into account the severity or the length of each episode of worsening heart failure. When this was corrected, the sensitivity analyses showed that there was no difference in the degree of patients’ reported dyspnea. That means, there was no reported benefit to the patients regarding shortness of breath or anything else.
The sponsor submitted an addendum last week that includes a series of analyses addressing this methodological problem. That is not how the FDA process is supposed to work. All data presented at this meeting should be fully vetted by FDA scientists and statisticians and provided to the Advisory Committee in advance. The new analyses need to be carefully and cautiously analyzed by unbiased FDA scientists to determine whether or not this drug is effective at all.
Let me emphasize that 3 out of 4 endpoints found no improvement compared to the placebo. The sponsor is claiming a very modest benefit in shortness of breath through day 5 – a difference that is very subjective and that the FDA believes is not clinically meaningful. And, if the data were not analyzed correctly, there is no benefit at all. We can’t assume that this last-minute analysis by the sponsor proves that the drug is beneficial. This needs to be carefully vetted by the agency and since the Advisory Committee did not have access to that vetting, you should not take this last-minute analysis into consideration when you vote today.
Another important concern about this trial is the lack of diversity. An overwhelming majority of patients in this trial were White and less than 5% were African American. This should not be acceptable in a study for a drug for heart failure, the #1 killer in every racial and ethnic minority group in the U.S. In particular, African Americans are more likely to have heart failure, more likely to suffer more severely, and more likely to get worse faster.
Even though they used a definition of Latino that is not the one that FDA requires, at most, less than 10% of the patients were Latinos. Latinos are at very high risk of heart disease, and as the largest minority group in the country, the sponsor should have included more in the study. .
Research tells us that some naturally occurring genetic variations may influence the way certain drugs are metabolized and work in women compared to men, older patients compared to younger, and certain racial and ethnic groups. For this reason, subgroup analyses are essential to make sure a drug is safe and effective at particular doses for these major groups.
There were not enough African American patients in the study to conduct subgroup analyses to find out if this drug is safe or effective for African Americans. There were perhaps enough Latinos to do a subgroup analysis, but the company didn’t do one. FDA should have required the company to have enough African American and Hispanic patients to separately analyze safety and efficacy for those two large minority groups.
But they didn’t. So, in addition to not knowing if this drug has any meaningful benefits for any patients, we know nothing at all about the benefits or risks for people of color.
The FDA has very clear guidelines that urge sponsors to include women and minorities in their clinical trials. When sponsors ignore those guidelines, using one excuse or another, Advisory Committees like this one should speak up about it. I hope some of you will. It isn’t enough to require more diversity in post-market studies, for the simple reason that companies too often ignore that requirement – they no longer have any incentive to follow FDA guidelines once the drug is on the market. The FDA’s own data confirms this in a report they released last year (the 907 report).
I don’t think that our country’s largest minority groups — African Americans and Latinos — should be paying for drug treatments that have not been adequately studied on them.
But in the case of this drug, I don’t think ANY patients should pay for treatment that is not conclusively proven to work either.
The company pointed out that the Serelaxin patients had lower all-cause mortality. But, as you know, with all the statistical analysis, this could easily occur by chance. There is no evidence it was a result of the drug, given that 60-day mortality and hospitalization did not differ from placebo.
Given the lack of credible evidence that this injected drug is more effective than placebo, we urge you to advise the FDA to NOT approve it at this time, and to require better research as well as subgroup analyses on African Americans and Latinos.
This is not a life-saving drug. Post-market studies usually take 10 years to be completed. Proof that it works should be required before it is on the market. It would be terribly unfair to patients and providers to approve it without clear evidence and then wait 10 years for post-market studies to be completed.
The FDA Advisory panel voted unanimously against approval at the end of the meeting.