NRC Comment on a FDA Guidance on the Evaluation of Sex Differences in Medical Device Clinical Studies

March 19, 2012

Division of Dockets Management (HFA-305)

Food and Drug Administration
5630 Fishers Lane, Rm. 1061
Rockville, MD 20852

Re: Comments on the Draft Guidance, “Evaluation of Sex Differences in Medical Device Clinical Studies” [Docket No. FDA-2011-D-0817]

The National Research Center for Women & Families, a nonprofit think tank that uses research to improve health programs and policies, is pleased to comment on the draft guidance Evaluation of Sex Differences in Medical Device Clinical Studies. We strongly support the agency’s efforts to ensure that more women are included in clinical studies of medical devices, and to notify device companies of the importance of separately analyzing outcomes for male and female patients. For a variety of reasons, women and men may have different experiences with medical devices, especially diagnostic and implantable devices, and it is essential that adequate numbers of women are included in medical device clinical studies, that men and women’s outcomes and experiences with devices be analyzed separately, and that if there statistically significant sex differences in safety or effectiveness, or if there are potential differences that might require follow-up studies, that the data on risks and benefits be described separately for women and men on the labels.  We ask you to also ensure adequate representation of women of color, women in different age groups (when relevant), and subgroup analyses of these different groups to determine any differences in safety or efficacy.

Despite improvements regarding the inclusion of women in clinical trials over the last few decades, women are under-represented in clinical studies of cardiac devices and many other devices, and even when they are included there are often no subgroup analyses.  For example, a study published last year reported that more than one in four studies of high-risk cardiovascular devices didn’t even mention the sex of the participants, and only 41% of the studies included analyses of results by sex or commented on possible sex differences in their FDA Summaries of Safety and Effectiveness Data (SSEDs) for premarket approval. Even if the information is included in the SSED, it is commonly not provided in the package insert.[1]

Of course, the weakness of a FDA guidance is that it is nonbinding.  There are two ways to strengthen the impact of this guidance.  One is to use the guidance as the basis for the FDA promulgating a rule to require the inclusion of sufficient numbers of women and separate analyses of women and men.  However, even without a rule it is also possible for the FDA to convey the importance of this guidance to device companies, letting sponsors know that they are unlikely to have their devices approved for women and men if there are not sufficient numbers of women and men in their studies and if safety and efficacy are not established separately for women and men.  Since device manufacturers want to sell their products to as many people as possible, we believe that warning them that approval for each sex will be dependent on adequate data analysis for each sex would be a powerful incentive to manufacturers.

We also have several comments regarding specific recommendations in the guidance document.

Recommendations for Achieving Representative Enrollment

While FDA states that there are fewer women than men in many types of clinical trials, there is no evidence that this lower participation is inevitable.  We believe that better recruitment efforts can solve this problem of under-representation.  The agency has recommended sponsors take several actions to increase the participation of women in clinical trials of medical devices, and we strongly support those. We also strongly urge that device manufacturers be encouraged to improve analyses and information about sex differences, even for studies that have already been completed.  Rather than focusing only on new studies, FDA should ask sponsors to revise ongoing studies and analyses to respond to the guidance.

We support maintaining open enrollment for women until a pre-specified proportion is reached. We strongly agree with the FDA’s suggestion that device sponsors use the Women’s Health Initiative (WHI) as a model for study recruitment, informed consent, and labeling. Device sponsors would benefit from utilizing WHI strategies for enrolling and retaining female study participants.

We agree that sponsors should provide sex-specific background information on the disease or condition the device is intended to treat. This should include information on the sex-specific prevalence; diagnosis and treatment patterns; as well as the proportions of women included in previous studies that are relevant to the current one, and any known clinically significant sex differences in safety or effectiveness of other treatments. We also strongly recommend that the FDA ask sponsors to provide information on racial or ethnic differences with regard to the disease or condition the device is intended to treat. That information can be used to ensure that the studies include samples of women and men of color that are sufficiently large to detect sex and race-specific differences.  As an example of the importance of this effort, we recall an FDA Advisory Committee meeting on a device for treating uterine fibroids that included very few African American women and did not analyze them separately.  Uterine fibroids are a more common and more serious problem for African American women compared to white women, and it was therefore inappropriate for the FDA to approve the device for all women despite the lack of data on African American women.  Unfortunately, the FDA did so.

We are more concerned about analyzing adequate numbers of women in each study and less concerned about whether medical device clinical studies enroll a proportion of women that accurately represents the proportion of women in the population with the disease or condition the device is intended to treat. Even if relatively small percentages of patients with particular diseases are women, if there are tens of thousands of women with the disease it is important that substantial numbers be included in clinical trials.  We are especially concerned about settling for proportional representation because the clinical trials submitted as part of the PMA process often include relatively small numbers of patients; less than 30% of a sample of 100 patients is too small to provide meaningful data.  It is more important that the number of women and men be large enough for the sex-specific subgroup analyses to be powered for statistical significance.

Although the focus on this guidance is on women, under-representation of men is sometimes a problem in medical device clinical studies as well.  For example, last year the FDA expanded approval of gastric lap bands for men and women based on a very small number of men, many of whom did not benefit from the lap bands.  The lack of men in that study was just as inappropriate as the lack of women in other studies.  And, as previously stated, we strongly urge that the studies be powered to make it possible for the sponsor to conduct sub-group analyses evaluating health outcomes for women of color, particularly the racial or ethnic groups most likely to need the device These sub-group analyses are particularly important in trials of devices intended to treat diseases or conditions for which there is evidence of racial or ethnic health disparities

Recommendations for Sex-Specific Statistical Analyses

Currently, many PMA clinical trials include a small percentage of women, the data for women and men are combined and presented as patient outcomes, and the results are interpreted as if they applied equally to women and men.  This is unacceptable, because the results are not necessarily accurate for either men or women, although the outcome data tend to be a more accurate representation for the larger sample (usually men).   The proposed guidance emphasizes analyzing sex differences in study outcomes.  Instead, we strongly urge that the focus be on analyzing outcomes separately by men and women, and reporting those outcomes separately.

This is an important distinction because many CDRH studies are quite small, and a sex difference that does not reach statistical significance might possibly be a clinically significant difference that would be statistically significant if the sample sizes were larger and adequately powered.  If the samples of women and men are not adequately powered to accurately assess sex differences, the results should be reported separately for women and men.  In contrast, a large study of women and men with small, non-significant sex differences in outcomes (adverse reactions and benefits) would justify combining the men and women and reporting the results together.

In general, women considering a specific device should not be concerned about how well it works for men, they should only be concerned about how well it works for women like themselves.  For example, if hip joint X lasts 8 years on the average woman, that is more important to a potential patient than whether X lasts more years or fewer years for women compared to men, or whether that sex difference is statistically significant.  Whether hip X lasts 4 years in men or 20 years in men does not affect X’s benefits for women.  What is important to women is whether there is another hip (Y or Z or whatever) that lasts significantly longer than 8 years in women. That is why sex differences are not the key issue for patients and consumers, subgroup analyses are.

We also urge the agency to recommend that sponsors report safety and effectiveness endpoints for women of color and women in different age groups. Consequently, device study designs should always include plans to do sex and sex/race/age-specific sub-group analyses. If device studies are designed and conducted with a representative and adequate number of women of color in different age groups, the sponsor will be more likely to detect and assess such differences if they exist.

If sex-specific analyses show that the safety or efficacy of the device differs based on sex, the safety and efficacy results should never be pooled across sex. The pooling of results would skew the findings, providing less useful information for women and for men than the results for women and men separately.   For example, a device that is effective for 20% of women and 80% of men would average out to 50% of adults if the sample included an equal number of women and men, but a 50% success rate

would not accurately predict outcome for a typical man or woman.  The subgroup analysis results would be much more useful for patients and their doctors.

Reporting Sex-Specific Information in Summaries and Labels

We strongly support the agency’s recommendation that sponsors include sex-specific analyses and outcomes on device labels. This is long overdue.  We also agree that if analyses suggest a likely sex difference but statistical significance is not reached, the findings should be reported descriptively, separately by sex, and the label should report if analyses were done and no significant differences were found. If a pre-market study signals a possible sex difference in safety or efficacy and the device is approved, the agency should require sponsors to conduct a post-market study with a larger, more representative sample to evaluate the signal more fully. We also strongly urge the agency to recommend that the label include the results of race-specific outcomes analyses about the evidence of safety and effectiveness for women of color, and age-specific outcomes.  For example, implants may have a much different safety or effectiveness profile for older patients.

Conclusions

In conclusion, as more than half the adult population of the United States, women should be appropriately represented in all clinical trials for devices that they are likely to use, and sub-group analyses are essential to determine safety and efficacy for women.  Determining whether there are statistically significant sex differences is not as important as conducting clinical trials that provide specific information to women and their health care providers about the likely risks and benefits of a device on female patients.  This guidance for sponsors should be strongly worded to clarify the agency’s expectations for sex-specific health outcome information, and thus provide a persuasive incentive for sponsors to comply as they collect, analyze and report the safety and effectiveness of their products in women.  However, we believe a rule is needed to provide a greater incentive to achieve these goals.

For more information, contact Paul Brown at (202) 223-4000 or pb@center4research.org


1 Dhruva SS, Bero LA, Redberg RF. Gender bias in studies for Food and Drug Administration premarket approval of cardiovascular devices. Circ Cardiovasc Qual Outcomes. 2011;4(2):165-171.