The Matrixx Zicam Verdict

It is not often that the concept of statistical significance reaches the status of headline news. But such was the situation after March 22, 2011, when the United States Supreme Court announced its decision in Matrixx Initiatives, Inc. v. Siracusano (1, 2). In a unanimous decision, the Supreme Court held that the lack of a statistically significant association between Matrixx's product Zicam and reports of adverse-drug events could not be used as justification to withhold important information from stockholders. The pharmaceutical industry has questioned whether the Matrixx decision will have broader implications for drug products. The authors summarize the statistical and regulatory implications of the Matrixx decision on pharmaceutical quality and manufacturing.

Background

Matrixx is a securities-fraud class-action suit filed by investors against Matrixx Initiatives, Inc., a pharmaceutical company headquartered in Scottsdale, Arizona, and three of the company's executives, for violating federal securities laws by failing to disclose reports of a potential link between its leading product, Zicam Cold Remedy, and anosmia (i.e., loss of smell) (3). The investors alleged that Matrixx's past statements to the market, in which the company predicted revenue increases between 50 and 80%, were misleading in light of information available to Matrixx regarding the risk to its main revenue-generating product (4). While providing optimistic sales estimates, Matrixx allegedly failed to disclose adverse-event reports regarding consumers who had lost their sense of smell after using Zicam Cold Remedy (5).

Procedural history. On Dec. 15, 2005, the US District Court for the District of Arizona granted Matrixx's motion to dismiss the suit based on the investors' failure to properly plead the elements of a "material misstatement or omission" and knowledge of wrongdoing (6). In reaching this decision, the District Court relied in part on a ruling made by the US Second Circuit Court of Appeals in the case In re Carter-Wallace, Inc. (7). The In re Carter-Wallace decision affirmed the dismissal of a securities-fraud class action suit based upon the investors' failure to sufficiently allege knowledge of wrongdoing on the part of the pharmaceutical company due to the absence of a statistically significant connection between the targeted drug and aplastic anemia (8).

Seizing on the language in In re Carter-Wallace, the District Court in Matrixx noted that, "courts have found adverse information related to the safety of a product is not material unless such reports provide reliable statistically significant information that a drug is unsafe" (9). In September 2003, a study conducted by the University of Colorado School of Medicine identified 11 Zicam users who suffered from anosmia (10). According to the investors in Matrixx, not only did the pharmaceutical company fail to disclose the University of Colorado study, but it continued to make positive public statements regarding the product's growth potential and safety (11). Nonetheless, the District Court held that the investors "failed to present evidence of a statistically significant correlation between the use of Zicam and anosmia so as to make failure to publically disclose complaints and the University of Colorado study a material omission" (12).

The US Ninth Circuit Court of Appeals, however, reversed the District Court's decision in favor of the investors, stating that the District Court erred in requiring statistical significance to establish materiality (13). Relying on the Supreme Court's rulings in Basic Inc. v. Levinson and TSC Industries, Inc. v. Northway, Inc., the Circuit Court rejected the "bright-line" rule (i.e., a simple or straightforward judicial rule of decision) to determine materiality in favor of fact-specific inquiry in which an omission is material "if there is a substantial likelihood that a reasonable shareholder would consider it important in deciding how to vote" (14–16).

Thus, based on the factual allegations, the Circuit Court held the investors had sufficiently pleaded materiality to survive Matrixx's motion to dismiss (17). Thereafter, Matrixx filed a petition for certiorari, which was granted by the US Supreme Court.

The court's decision

From the outset, the US Supreme Court made clear that the issue at stake was: "Whether a plaintiff can state a claim for securities fraud … based on a pharmaceutical company's failure to disclose reports of adverse events associated with a product if the reports do not disclose a statistically significant number of adverse events" (18). The short answer provided by the Supreme Court was clear and unanimous: yes. The Supreme Court affirmed the decision of the Ninth Circuit in finding that the bright-line materiality rule sought by Matrixx was inconsistent with its ruling in Basic (19).

The Supreme Court noted that because multiple factors are considered in determining causation, the bright-line rule sought by Matrixx "would 'artificially exclud[e]' information that 'would otherwise be considered significant to the trading decision of a reasonable investor'" (20). Thus, Matrixx's argument in favor of statistical significance as the only reliable indication of causation was deemed flawed (21). After citing a litany of circumstances under which causation is determined in the absence of statistically significant data using other evidence, the Supreme Court held that "assessing the materiality of adverse event reports is a 'fact-specific' inquiry ... that requires consideration of the source, content, and context of the reports." This is not to say that statistical significance (or the lack thereof) is irrelevant—only that it is not dipositive of every case (22). Consequently, the Supreme Court, citing its ruling in Basic, which in turn relied on the language of the TSC Industries decision, held that because (taking the investors' allegations as true) information provided to Matrixx revealed a plausible causal relationship between Zicam Cold Remedy and anosmia, it was "substantially likely that a reasonable investor would have viewed this information as having significantly altered the 'total mix' of information made available" (23).

Statistical significance

Statistical discussions are exacting and circumscribed because the field of statistics uses the language of mathematics as well as English to define, describe, and present statistical concepts and results. Words that are used freely in a lay discussion take on mathematically precise meanings. Sentences are worded carefully to present the statistical concepts correctly. Generally, this does not present a problem because the context usually reveals whether the discussion is technical or not. In the both the Basic and Matrixx cases, it was clear that the discussions were not statistically oriented, and that the word "significant" was used in the general sense.

In the Carter–Wallace case, it is clear that the discussion is statistical in nature, and the word "significant" is used in its statistical technical sense. Thus, there is a need for a definition of the term "statistical significance."

Statistical analysis provides scientists with a tool along with theory and common sense for making scientific interpretations and conclusions. Often, the analysis focuses on identifying significant differences, that is, practical and statistical differences. Practical significance comes from comparing an observed difference (i.e., a signal) with an absolute reference. Practical significance always takes precedence over statistical significance. In fact, statistical significance should not be determined until practical significance is found.

Statistical significance, on the other hand, comes from comparing an observed difference with a relative reference that incorporates a noise or random variability. Statistical significance testing compares a signal with noise and is often expressed as a ratio of signal to noise. The result is not meant to be a statement of causality, truth, or reality. That is because if the signal can be shown to be stronger than the noise (i.e., more of a difference than expected by chance variation alone), then the scientist may conclude it to be "statistically significant." Otherwise, it cannot be shown to be significant. If more data are obtained, the noise could be reduced, perhaps helping to demonstrate that the signal is significant. In fact, if the noise is small enough or if the sample is large enough, even wildly anomalous differences can be shown to be statistically significant. This difference is the reason that significance cannot be a single "bright-line" rule for causality. Rather, the primary purpose of statistical testing is to prevent the declaration of an apparent practical significance when, in fact, it could be caused by random variation.

Implications for the pharmaceutical industry

The Supreme Court's discussions of "bright-line" decision rules and the "total mix"of information applies to the use of statistical analysis of quality assurance and manufacturing data. Without too much difficulty, the Supreme Court's position could apply equally to stability studies, assay validation, process validation, design space, and process capability.

Anyone who has hosted an FDA inspection knows that an investigator need not wait for a control chart to exceed one or more of the classic "out-of-control" signals before beginning a review of suspicious current and historical data.

Sampling-plan acceptance should not be used as a defense for accepting some lots when it is clear from all the available information that the defect rate is higher than desired for the process as a whole. Waiting for a statistically significant event before evaluating a trend simply pushes the problem down the road; such an approach is also inconsistent with the concept of quality by design.

Manufacturers should be looking at the total mix of information throughout process design, process qualification, and process verification. FDA's guidance for industry on Process Validation: General Principles and Practices, issued in January 2011, makes clear that understanding the manufacturing process and potential variations—that is, the total mix of information—is essential for product quality. Interestingly, the FDA process-validation guidance makes no mention of statistical significance. The document does refer to statistical confidence several times, but only in a general sense. It does not suggest or require a specific confidence level.

As stated above, statistical significance is only the determination that the difference we observe is larger than we would expect by random chance alone given the available estimate of variability. It is not necessarily truth, fact, or causation. It is only one tool for understanding all of the available information. FDA expects pharmaceutical manufacturers to use science-based reasoning to determine which tools are appropriate for evaluating any given issue.

In the laboratory. An out-of-specification (OOS) determination is the result of comparing a single reportable value collected at a specific point in time with a specification. Statistical significance has no role in this determination, nor does practical significance. A reportable result is either within specification or OOS. Thus, the Matrixx case has little or no implication for OOS determinations.

A determination of out-of-trend (OOT) is the result of comparing one or more reportable results collected over a time period with a statistically defined model or to statistically summarize those historical data. If a trend is large enough to be of practical importance to an experienced analyst, then data should be reviewed even if they do not rise to the level of statistical significance. If the data are statistically significant, then chance is ruled out and a cause-and-effect relationship is supported but not proven. This position is supported by Matrixx, in that a lack of statistical significance does not justify failing to investigate a change in the data that could be of practical importance. Statistical significance cannot be used, to quote the Supreme Court, as a single "bright-line" decision rule to the exclusion of other sources of supporting information. The evaluation of OOT data must be an evaluation of the "total mix" of information.

Product liability lawsuits. Given that the underlying facts in Matrixx relate to adverse drug events, it is reasonable to ask whether the Supreme Court's decision has any bearing on product liability lawsuits. Although the case does not appear to change the Supreme Court's view of the standard for causation (i.e., that the product "more likely than not" caused the injury, and that statistical significance is not necessary to prove such a claim), it does seem to reinforce the Supreme Court's holding in an earlier decision, Wyeth v. Levine, regarding a drug manufacturer's responsibility to act upon new information about adverse events (24).

In Wyeth, the Plaintiff, Diana Levine, claimed that she was injured by the inappropriate administration of Wyeth's antinausea drug Phenergan, which caused her to develop gangrene and resulted in the amputation of a portion of her arm. In this case, a Vermont jury found that the Phenergan label was inadequate and should have contained a stronger warning regarding intravenous push administration because of the known risk of inadvertent intra-arterial injection and consequent development of gangrene. Wyeth maintained that Levine's state tort claims were preempted because FDA had approved the Phenergan label without such a strong warning (25).

The doctrine of federal preemption is based upon the Supremacy Clause in the Constitution, which states that the "Constitution and the laws of the United States ... shall be the supreme law of the land [notwithstanding] anything in the constitutions or laws of any state. …" (26). Federal law preempts state law where a federal statute contains specific language indicating that federal law must control (express preemption), where federal law "occupies the field" (i.e., field preemption) or where state law conflicts with federal law and thus making it impossible to comply with both (i.e., conflict preemption).

Wyeth argued that it would have been impossible for it to modify its labeling without violating federal law and that allowing Levine's state tort claim was unacceptable because it would permit a state jury to override FDA's scientific judgment regarding a product's labeling (27, 28). The Supreme Court rejected Wyeth's arguments, holding that state tort claims for injuries caused by drug products were not preempted by the Federal Food, Drug, and Cosmetic Act, and that drug manufacturers have primary responsibility for strengthening warnings in their labeling based on new information about adverse events. The Supreme Court reasoned that Wyeth could have strengthened the warnings on the Phenergan label without prior FDA approval, and added that Wyeth could not simply rely on its approved labeling when faced with new information of adverse events (29, 30).

The Supreme Court noted that FDA does not have sufficient resources of its own to enforce drug-product safety. FDA has historically viewed state tort law as a supplementary form of drug product regulation. Ultimately, however, the responsibility for safety of the product resides with the manufacturer (31).

Conclusion

The Wyeth decision put drug manufacturers on notice that they bear primary responsibility for their labeling, and they must be vigilant in monitoring and evaluating adverse events associated with their drug products. Moreover, the evaluation of risk information must be ongoing: an event that seems insignificant today may be quite significant a year from now, in light of other adverse events. As shown in Matrixx, courts may not look solely to the concept of statistical significance in order to determine whether or not such adverse events indicate causation. Instead, as reaffirmed by the Supreme Court, drug manufacturers must continue to evaluate multiple sources of information when assessing the risks associated with their products.

While the Matrixx case made no mention of Wyeth, it seems that the two cases are consistent in requiring drug manufacturers to evaluate carefully any new information about a drug product that trigger an obligation to act—whether to notify stockholders of material facts or to change the warnings on a product's labeling. However, such cases merely illustrate that the courts, investors and federal and state governments expect this obligation to extend well beyond mitigation of litigation risks and instead to include every aspect of pharmaceutical product design, manufacture, and stewardship. Statistical significance, while still a valuable tool for assessing variation, is only one of many considerations in the "total mix" of information available to drug manufacturers.

Related reading

• S.T. Ziliak and D.N. McCloskey, The Cult of Statistical Significance (University of Michigan Press, Ann Arbor, 2008).

• R. Hooke, How to Tell the Liars From the Statisticians (Marcel Dekker, New York, NY, 1983).

• H.F Spirer et al., Misused Statistics, 2nd ed., (Marcel Dekker, New York, NY, 1998).

• C.Wang, Sense and Nonsense of Statistical Inference (Marcel Dekker, New York, NY, 1993).

Cathy L. Burgess is a partner in the Washington, DC, office of the law firm Alston & Bird LLP, tel. 202.239.3648, cathy.burgess@alston.com. Sean A. Simmons is a senior associate in the Atlanta, Georgia, office of Alston & Bird LLP, tel. 404.881.4576, sean.simmons@alston.com. Lynn D. Torbeck is a statistician at Torbeck and Assoc., and a member of the Pharmaceutical Technology Editorial Advisory Board, tel. 847.424.1314, Lynn@Torbeck.org, www.torbeck.org.

References

1. D.G. Savage, Chic. Trib., Mar. 23, 2011; C. Bialik, Wall Street Jrnl. Apr. 1, 2011.

2. Matrixx Initiatives, Inc. v. Siracusano, 131 S. Ct. 1309 (2011) [hereinafter Matrixx].

3. Matrixx, 131 S. Ct. at 1313.

4. Id. at 1315. Zicam Cold Remedy Reportedly Accounted for Seventy Percent of Matrixx's sales. Id. at 1314.

5. Id.

6. Id. at 1317. "To prevail on their claim that Matrixx made material misrepresentations or omissions in violation of § 10(b) and Rule 10b–5, respondents must prove '(1) a material misrepresentation or omission by the defendant; (2) scienter; (3) a connection between the misrepresentation or omission and the purchase or sale of a security; (4) reliance upon the misrepresentation or omission; (5) economic loss; and (6) loss causation.' " Id. at 1317 (quoting Stoneridge Investment Partners, LLC v. Scientific–Atlanta, Inc., 552 US 148, 157 [2008]).

7. 220 F.3d 36 (2d. Cir.1998).

8. Id.

9. Siracusano v. Matrixx Initiatives, Inc., No. CV 04 0886 PHX MHM, 2005 WL 3970117, *5 (D. Ariz., Dec 15, 2005).

10. Id. at *1.

11. Id.

12. Id. at *7.

13. Siracusano v. Matrixx Initiatives, Inc., 585 F.3d 1167, 1178 (9th Cir. 2009) [hereinafter Siracusano].

14. Basic Inc. v. Levinson, 485 U.S. 224, 236 (1988).

15. TSC Industries, Inc. v. Northway, Inc., 426 U.S. 438, 449, (1976).

16. Siracusano, 585 F.3d at 1178.

17. Id. at 1183.

18. Matrixx, 131 S. Ct. at 1313.

19. Id. at 1319.

20. Id.

21. Id.

22. Id. at 1321–22.

23. Id. at 1318.

24. Wyeth v. Levine, 129 S. Ct. 1187 (2009).

25. Levine v. Wyeth, 944 A.2d 179, 182 (Vt. 2006).

26. U.S. Const. art. VI.

27. Wyeth, 129 S. Ct. at 1193.

28. Id. at 1193-94.

29. FDA allows manufacturers to "add or strengthen a contraindication, warning, precaution or adverse reaction" in order to increase the safety of the product through use of a "changes being effected" (CBE) supplement. 21 CFR 314.70(c) (6) (iii) (A), (C).

30. Wyeth, 129 S. Ct. at 1197.

31. Id. at 1197–98.