Jon Tuckwell

Much has recently been written regarding the US Food and Drug Administration's (FDA's) dose content uniformity (DCU) tests contained in its draft Guidance for Industry.1 This article is not intended to be merely another overview of the poor statistical relevance of the test, nor is it simply a commendation of the International Pharmaceutical Aerosol Consortium on Regulation and Science (IPAC-RS) for the work it has done to raise the profile of this issue and propose an alternative. Instead, the authors intend to propose a novel approach that optimizes the level of testing required for each batch and reduces the risk of high quality batches of drugs being discarded unnecessarily.	

Before alternative approaches can be discussed, it is necessary to highlight the existing testing procedures and also to examine the proposal presented by the IPAC-RS.	

FDA's 'Guidance for Industry - Metered Dose Inhalers (MDI) and Dry Powder Inhalers (DPI) Drug Products' was released in draft version in November 1998.1 Within this document, FDA outlines the recommended method for assessing the DCU of inhalation drug products and describes this test as "Providing an overall performance evaluation of a batch."	

Figure 1:  The results of the FDA test, applied to batches of varying quality. The poor discrimination of the test can clearly be seen.		

The FDA test examines two aspects of DCU - the DCU for doses from multiple containers within a batch and the DCU for doses within the same container. This article focusses on the former.	

The DCU test is familiar to many in the industry because it forms the backbone of the quality assurance (QA) testing for inhalation drug products. However, for those who are less familiar with it, it can be summarized as follows:	

Dose content uniformity test. For each of 10 containers, determine one dose. The test is passed if:	

no more than 1 in 10 values is outside 620% of the label claim (LC)

the average of the 10 values is within 615% LC.

the average of the 10 values is outside 615% LC

If the test is not passed or failed, 20 additional containers are tested in a second tier. The test is passed if:	

no more than 3 of the 30 values are outside 620% LC

the average of the 30 values is within 615% LC.

Otherwise the test is failed. The test may not be repeated on the same batch.	

The FDA test has been devised so that it is simple to understand and easy to implement, thus reducing confusion when the test is interpreted. For any given set of delivered dose data and corresponding LC, it is possible to quickly analyse the data and to pass or fail the batch.		

However, such a simple approach has resulted in a test with poor statistical relevance. The test fails to accurately distinguish between high and low quality batches of drug and, as a result, leads to a high frequency of either poor batches being accepted or good batches being rejected. It is this poor statistical relevance that has been the driving force behind recommendations to change the test. 		

Figure 2: The 10/30 IPAC-RS test applied to the same batches as in Figure 1.  The data drawn from each batch are identical to that used in Figure 1.	

Another criticism often directed at the test is that it does not permit drug manufacturers to reduce the level of testing required if they produce a high quality product. The test demands the same level of QA testing for all batches of product, however close they are to the pass/fail criterion. It would appear to be in everyone's interest to encourage the production of high quality drugs by reducing the burden of QA testing on such a product.	

During November 2001, IPAC-RS submitted its response to the FDA draft Guidance for Industry detailing its recommendations for an alternative DCU test.2 The test is a parametric tolerance interval (PTI) test containing three acceptance criteria:		

An acceptance criterion requiring that the standard deviation not exceed a sample-size dependent maximum value. This requirement controls the batch variability when the mean is close to the LC.

An acceptance criterion requiring that an acceptance value (AV) not exceed a fixed limit. The AV is the sum of the absolute deviation of the sample mean from the LC and the standard deviation scaled by a sample-dependent coefficient. The AV simultaneously controls the batch mean and standard deviation, so that less variability is allowed the more the mean deviates from the LC.

An acceptance criterion of 85–115% LC is imposed on the sample mean to further control mean deviation from the LC when variability is low (for a multi-dose product, this criterion is applied to each tested life stage; for example, the beginning, middle and end of container life).

There is no doubt that the IPAC-RS proposal makes significant strides towards a more rational and statistically relevant DCU test. Figure 1 and Figure 2 contain graphs comparing the existing FDA test with the 10/30 test plan proposed by IPAC-RS. The data clearly demonstrate the poor differentiation provided by the FDA test, with unacceptable batches being passed and, most significantly, a high percentage of acceptable batches being failed. It is evident from the graphical representation of the IPAC-RS test that the proposed PTI test improves the discrimination of the testing process and more accurately assesses the quality of the drug product. Fewer acceptable batches were rejected and fewer low quality batches were accepted.			

Another key benefit of the proposed PTI test is the ability of a manufacturer to determine its own testing programme. IPAC-RS has proposed six test plans with increasing levels of sampling, from a sample size of only 10 doses per batch up to a first tier sample size of 24. The smaller the test sample, the higher the probability of an anomalous dose resulting in the batch failing the acceptance criteria. However, manufacturers with a high quality product will be confident of passing the acceptability criteria, even with smaller sample sizes. The PTI test allows these manufacturers to benefit from their high quality product by reducing the volume of QA testing and, therefore, the cost associated with QA testing.	

The IPAC-RS proposal offers significant improvements compared with the current FDA DCU test programme; however, it still has a number of features that are undesirable. For example, a manufacturer must decide what level of testing it will undertake before it commences QA testing. The manufacturer may decide to perform a test plan requiring a high level of testing and yet when the first ten results come back, it may be evident that the batch is of excellent quality and further testing will not be required.			

Figure 3:  Central section of prior on batch mean.	

Conversely, a low sample quantity test plan may be adopted and one erroneous result may cause the test to fail. The manufacturer is then faced with discarding a potentially high value batch of drug because it is forbidden to retest the batch and discard earlier results. An ideal DCU test would optimize the level of testing required depending on the quality of the drug product shown in the results.	

To optimize the levels of testing required, the authors have applied Bayesian mathematics to the DCU test. The test addresses the problems identified with the FDA and IPAC–RS tests whilst building on the improvements made by the IPAC-RS PTI test.		

The proposed test is based on Bayes' theorem of conditional probability. By applying this theorem to the data collected during a DCU test, an accurate assessment of the batch quality is achievable.	

Frequentist statistics and Bayesian Inference

Much has been written about the differences between frequentist and Bayesian statistics, and the debate will continue for years to come. However, to give a broad overview of the proposed test, a brief explanation of Bayesian statistics is required for those unfamiliar with the principles.				

Figure 4:  Cumulative distribution function for lower part of prior on batch standard deviation.		

Bayes' theorem on conditional probability can be expressed as Equation 1. This theorem can be used to calculate the probability of a theory being true given a certain data set (Equation 2).	

Therefore, if we hypothesize that a batch of product falls within given acceptability criteria, then the probability of this theory being true can be calculated. The only uncertainty exists in determining P(theory). This is an initial probability that the theory is true, otherwise known as the 'prior.' Standard frequentist statistics can calculate P(data|theory), the probability of the data set occurring given the theory that the batch is acceptable is true. However, it is only using Bayesian Inference that we are able to answer the true question - what is the probability that this batch of product is acceptable?	

Many would argue that arbitrarily assigning a value for the prior is unjustified and will result in a distorted final probability. However, it can be shown that if a suitable common prior is assigned to all DCU tests, then, after more than a few samples have been taken, the prior selected has only minimal influence.				

Figure 5:  The results of the Bayesian test applied to the same batches as in Figure 1. 	

Application of Bayesian Inference to DCU testing

As discussed earlier, by using Bayesian Inference we are able to accurately determine the probability that a tested batch falls within a given acceptability region. If a limit is then set on this calculated probability, a pass/fail criterion is defined.		

The IPAC-RS test contains acceptability coefficients that are calculated so that a bad batch will be rejected with a probability of at least 0.95. For the Bayesian test, we arranged to pass a batch if the probability that it is good is at least 0.95; that is, IPAC-RS promises that P(reject|bad)>0.95: Bayes accepts if P(good|bad)>80.95.	

Performing the Bayesian test on a batch is straightforward. An initial measurement is performed and P(batch is good|data) is calculated using the standard prior agreed by regulatory authorities. This result is then fed back into Bayes' theorem and, if necessary, combined with further readings to establish new values of P(batch is good|data). When P(batch is good|data) is found to be 0.95 or greater, the test is passed. 				

Figure 6: The number of measurements made on each batch during the Bayesian testing of Figure 3. 	

Articles

It has been demonstrated that the existing FDA dose content uniformity test has very poor statistical relevance, which has resulted in the acceptance of poor quality batches and the rejection of good quality batches. By using Bayesian Inference, a much improved test has been produced that allows the quality of a batch of drug product to be determined accurately, using a suitable number of samples for the quality of the batch.