'Capability of the Art' versus 'Fit for Use'

The US Pharmacopeia is replacing its current Chapter <111> "Design and Analysis of Biological Assays" with a suite of five chapters. The draft of Chapter <1033> "Biological Assay Validation" has been published for public comments and clearly takes fit for use as its guiding principle (1). The key breakthrough is that it advocates setting acceptance criteria for precision based on the relative spread of the product versus spread of specification.

If a manufacturing process delivers products over a large portion of the specification spread and risks exceeding specification limits, the validation acceptance criteria on precision of the respective bioassay needs to be tightened. Because it uses the process capability index (C_pk) for formal assessment this approach is sometimes referred to as the "C_pk approach". In this context, the criteria on precision are meant to apply to format variability (variability of the reportable value), which is a new term defined by the following equation:

Only when the "specifications have yet to be established" does the draft suggest the application of precision criteria to intermediate precision:

Intermediate precision describes the fundamental variability of the assay, independent of the "use" or "purpose" (i.e., the process and its specification) and replication format. The draft chapter uses geometric standard deviation (GSD), which is also new terminology, in place of the common term of relative standard deviation (RSD). The two are only a reasonable approximation of each other when they are small (less than 20%). GSD and RSD, as well as another closely related metric, are discussed in a previous article (2).

During the third USP Bioassay Workshop held at USP headquarters on Aug. 11–12, 2010, some chemistry, manufacturing and controls reviewers and statisticians from FDA's centers for drug and biologic evaluation and research provided feedback on this chapter. Their central concerns were basing acceptance criteria on format variability based on the C_pk approach, and the complete omission of repeatability in the draft chapter. To make this discussion concrete, repeatability can temporarily be defined by the following equation if using the logic of the draft chapter:

The feedback from members of FDA referred repeatedlyto ICH guidelines and its traditional approach of setting acceptance criteria on repeatability and intermediate precision. Feedback from some industry sources raised similar concerns and objections.

Capability of the art and fit for use

The trend toward fit for use assay validation in recent years represents a step forward. The C_pk approach is a clear attempt to put fit for use philosophy into practice. However, there is something to be said about regulators' desire to see repeatability and intermediate precision calculated, and to apply acceptance criteria to them. It is because these two quantities are independent of the intended use or purpose and replication format that they give regulators a chance to assess whether the basic bioassay is optimized for capability of the art.

Repeatability and intermediate precision are related to the standard deviation of individual (relative potency) determinations, not the standard error of averages. With increasing number of individual (relative potency) determinations, calculated repeatability and intermediate precision should converge to the truth, not diminish to zero. They reflect the fundamental variability of the biochemical reaction and readout platform, which can be improved only through assay optimization. However, there is a natural limit because of the underlying biochemistry and technology.

The capability of the measurement procedure and that of the manufacturing process are two independent factors. The intended use of the bioassay often requires much tighter precision than repeatability and intermediate precision suggest. Once it is demonstrated that the basic bioassay has attained capability of the art through repeatability and intermediate precision, manufacturers should be free to design the replication format of the reportable value to take advantage of the power of averaging. Decisions on product quality are made only on the basis of reportable value, not individual determinations.

A compromise between the two doctrines may represent the right path forward: apply acceptance criteria on repeatability and intermediate precision to verify that the basic bioassay has attained the capability of the art, then apply acceptance criteria to format variability to verify that the chosen replication format is fit for use. The goal is to encourage manufacturers to optimize the bioassay as far as technology permits, then allow replication to further improve the bioassay for its intended "use" or "purpose". This approach eliminates unoptimized bioassays where precision is improved only through brute force replication.

Three kinds of precision

Though the current draft Chapter <1033> introduces additional precision terminology, such as format variability, it fails to clearly differentiate different precision types. For example, in Section 2.4 (Validation Strategies for Bioassay Performance Characteristics), intermediate precision is contrasted with format variability as being the one that is independent of replication format; however, in Section 2.5 "Validation Target Acceptance Criteria", σ²_RA is described as intermediate precision when it appears before the C_pk formula, but is also described as "(with associated format)" when it appears after the C_pk formula.

Three useful species of precision can and should be defined; all of them can be expressed as GSD, following the draft chapter, or RSD, the more common practice. Let σ² denote any kind of variance on the natural log scale (of the relative potency), then:

The three kinds of precision are: source specific precision components; format-independent cumulative precisions; format dependent uncertainty of reportable values.

If σ²_w and σ²_b denote within run and between run variance on the natural log scale, respectively, then the source specific precision components are expressed in the following equations.

The between run component can either be larger or smaller than the within run component. In fact, the relative magnitude of the two informs the choice of the efficient replication format.

The traditional concepts of repeatability and intermediate precision, which are format-independent cumulative precisions, are expressed in the following equations.

Because of its cumulative nature, intermediate precision is always larger than repeatability. These two quantities describe the fundamental short- and medium-term variability independent of the replication format. They are useful metrics to evaluate whether the assay has attained its capability of the art via careful development and optimization.

The format variability introduced in the draft chapter describes the variability of the reportable value under a given replication format (k within run replicates and n replicate runs):

It is closely related to ISO VIM3's definition of "Type A (statistical) uncertainty" (3). It is analogous to the standard error of an average, not standard deviation of individuals. It is not fundamental variability but predicted variability derived from experimentally determined components, and can be manipulated via replication format. However, this is the precision that speaks to the variability or uncertainty of the reportable value directly. Because decisions are made based on reportable value, it is the link to fit for use.

Given that there are so many different kinds of precision, sentences, such as "a bioassay with %GSD between 2% and 20%," are too vague to be useful and should be avoided.

Conclusion

Both capability of the art and fit for use concepts are relevant to assay validation. By setting capability of the art acceptance criteria on repeatability and intermediate precision, proper assay development and optimization can be ensured. By setting fit for use acceptance criteria on format variability, reportable values with sufficient quality to support our intended use are ensured.

When specifications are determined by clinical information, say potency specifications based on efficacy and safety, the fit for use acceptance criteria could be established by comparing the specifications to the process variability (i.e., the C_pk approach in the draft chapter). However, many specifications are determined based on process consistency (i.e., historical process capability). In this context, the C_pk approach is no longer appropriate; the fit for use acceptance criteria need to be established by other means.

Fit for use acceptance criteria are likely to be highly specific to each product's clinical profile and manufacturing process. capability of the art acceptance criteria, however, could be established by proper categorization and an industry norm survey, for which USP is particularly well suited.

Charles Y. Tan is director of biostatistics at Pfizer Inc., 401 North Middletown Road, Pearl River, NY 10965, charles.y.tan@pfizer.com

Submitted: Feb. 23, 2011. Accepted: Mar. 14, 2011.

References

1. USP Proposed General Chapter <1033>, "Biological Assay Validation," Pharmacopeial Forum 36 (4), 986–1005 (2010).

2. C.Y. Tan, Pharmacopeial Forum 31 (2), 653–655 (2005).

3. Joint Committee for Guides in Metrology, International Vocabulary of Metrology—Basic and General Concepts and Associated Terms (JCGM, Sèvres, France, 3rd ed., 2008).

Citation: When referring to this article, please cite it as "C.Y.Tan, "'Capability of the Art' versus 'Fit for Use,'" Pharmaceutical Technology 35 (5) 70–73 (2011)."