Stephan O. Krause

The first part of this article discussed general strategies for validation extensions to other test method components, laboratories and even different test methods.

  This second part provides practical tips on how to maintain test method suitability long after the formal completion of analytical method validation (AMV) studies. 

Case studies on how to meaningfully derive acceptance criteria for validation extensions and the validation continuum (maintenance) are described as well as an example on how to reduce analytical variability in validated systems. 

There are several points to consider when running an AMM programme. Usually, assay control results that are within established limits (e.g., ±3 s.d.) will yield valid assay results for the test sample(s). Whenever possible, the assay control should yield a similar assay response when compared with the test sample. Monitoring the assay control will then indicate when unexpected assay result drifts or spreads may have occurred. Whenever the assay control results are close or over their established limits, there is a high probability that test sample results are also going in the same direction. This is something that should not be ignored because it causes predictable, although not exactly measurable, errors in test results (cases 1B and 2B).

 Table 1 Instrument equivalency execution matrix			

Because production samples are run simultaneously with assay controls, both results should be reported and overlayed on a single statistical process control (SPC) chart (Figure 1). This will make "non-visible" data elements that lead to cases 1B and 2A-B, more "visible" and useful for information regarding the true production process performance. Had the process development, robustness and validation studies been completed properly by using a variance component analysis matrix, the contribution to overall process variance of effect sampling (timing, number of samples, handling, storage conditions, hold times) could be estimated. If needed, sampling could be better controlled within the standard operating procedures (SOP). The likelihood of all cases (1A-B, 2A-B) occurring could then be monitored. This would also allow a much better process understanding as not only could root-causes for process failures be more readily identified, but also more readily measured. 			

Furthermore, controlling and fixing process problems could save more batches from rejection. These steps would also be very much in line with the recently published principles for quality and process analytical technology leading to faster licence approvals and reduced inspections.

From the robustness studies performed during analytical method development (AMD) and the intermediate precision results during AMV, the identity of the method component that may be responsible for changing test results when replaced is clarified. 

Figure 1 Graphical representation of SPC potencies and "invisible" assay variance. 

This should indeed be the main purpose of AMD and AMV reports to not only provide results for variance components, but to also clearly identify potential bias and how they can/will be controlled as part of test system suitability. The validation is ideally only a confirmation of something already expected to be suitable from the AMD studies. It is the recommendation of the author that all method components and operational limits are sufficiently studied and properly documented in the AMD report. This information should then be used to set meaningful limits for sample handling/storage during testing, the number of replicates, and the overall system suitability limits in the AMD/AMV report. It will otherwise be more difficult to set limits and control the implementation of new method components without this knowledge from the AMD/AMV studies.  

Qualifying another instrument, reference standard, critical reagent, assay control or operator for testing may arise from the need to get test results faster. Often, use of an alternative instrument or operator is inevitable. In any case, whenever critical validated method components are exchanged, equivalency in test results between the validated method component and the alternative component should be verified.  

As discussed, the AMD/AMV report should indicate whatever those "critical" method components might be and what associated risk might be involved when changing particular components. Equivalency can be accomplished similarly to AMT, provided that limits set initially still hold, by using an equivalency matrix and appropriate acceptance criteria for accuracy (matching) and (intermediate) precision. 

An example for an execution matrix for instrument equivalency is illustrated in Table 1. The testing of various production lots, unlike the obligation to include this for AMTs (current regulatory expectation), should not be involved as this may not help for the purpose of the equivalency studies.

Understanding the relationship of all components

There are numerous ways process and method performance knowledge can be used to monitor and improve overall process and product quality. In reality, unlimited time and resources are not always available and we should first identify the most critical components and maintain committed to implement critical improvements before we get lost in too much data. A hypothetical example is provided below to illustrate how acceptance criteria for AMTs and AMMs could be derived with respect to estimated probabilities for 1A-B and 2A-B. Following from this example, potential improvements are discussed to illustrate what could be done to reduce the undesirable probabilities for cases 1B and 2A-B. 

A potency bioassay is used for downstream in-process and final container testing of a licensed biopharmaceutical drug. This method is monitored in our AMM programme. After months of commercial production, transfer of this analytical method to a different laboratory for product release testing is needed. If the downstream in-process stage is yielding inconsistent potency results (Table 2), the following data should be reviewed: 

 method performance (assay control) data.

Table 2 Historical process, sampling, assay performance data, and suggested limits for accuracy and (intermediate) precision. 			

We may want to start with the easiest variance component, the assay variance, as monitored with the assay control. Figure 1 illustrates the relationship between the assay performance and observed recent SPC potency results (last n=60). 			

The statistical likelihood for failures now needs to be estimated and a component variance analysis performed to estimate the contribution of each component. Then, focus can be put on how to set limits on post-validation activities from understanding the potential impact on the likelihood for all cases (1A-2B) to occur. The situation can be most effectively improved by having primarily in mind patient safety/dosing and regulatory expectations, but secondarily also the firm as the need to pass specifications and stay profitable is important. Similar to propagation-of-error calculations, an estimate for the sampling variance (batch-uniformity, stability, protein adsorption losses, etc.), could allow immediate estimation of the actual (true) process performance for potency by simply solving for it from Equation 1 (V=variance). 

The (hypothetical) historical results are presented in Table 2. The estimated probabilities for cases 1A and 2A, and the measurement errors ("invisible" component of SPC) are given in Table 3 along with probability estimates for the worst-case scenario when AMT results would reveal that the receiving laboratory will test at a bias at the acceptable AMT protocol limit of +1.5%. 			

Although simplified, several observations can be made from the data in Figure 1 and Table 2. Most important, the overall process performance is out of a desirable ± 3 s.d. SPC state. As said earlier, this is the "visible" SPC trigger that warrants action when limits are exceeded. The test system has recently (last n=60) yielded higher (2.0%) results for the assay control. The assay control is the same molecule in a highly similar matrix as the test sample and both are run simultaneously in each assay run. 

The "visible" process mean can therefore be expected to be about 2% higher. Looking chronologically at the assay control may show the root cause, that is, which assay component was changed and caused this difference over time. Alternatively, several smaller changes could have added up to this difference of 2.0%. For example, the reference standard may unknowingly be unstable even when frozen and may have lost 2.0% potency over time, providing proportionally higher (2.0%) results to the assay control and test samples. 

Although the 2.0% expected difference in process data may in reality be buried within several small sampling and/or production changes, it nevertheless constitutes a 2.0% bias for testing from the time that specifications were set to match clinical data, and the then-existing process variance (PV) and AMV. 

The estimate for the overall sampling process during production (from PV) is used to estimate the actual variance in process performance. However, If small- and/or large-scale process studies to estimate sampling variance were not done well or are no longer representative, this estimate may not be sufficiently accurate to provide a good estimate for the actual PV (using Equation 1). In the hypothetical example, the estimated true PV (2.0%) is smaller than the estimated sampling variance (2.3%), and smaller than the assay variance (3.0%) and always smaller than the overall "visible" process variance (4.3%). This may have been why the specifications had been set to 90–110 units in the first place. Often, the assay and sampling variance could indeed be greater than the actual process variance for downstream in-process potency testing for biopharmaceuticals because of various reasons. Some examples are listed below  				

 Protein adsorption losses (sampling, testing). 

 Sampling procedures lacking detail where needed. 

 Inappropriate sample handling before testing. 

 Poorly set test system suitability criteria. 

 Insufficiently monitored and controlled assay reproducibility. 

 Poorly developed or optimized test method. 

Estimations for the true process mean (99 units) and variance (2.0%) were made from the assay control performance and from Equation 1, respectively. For the AMT, both accuracy and precision are areas of concern because several method components (operators, instruments, location and others likely) will change when the method is executed at the receiving laboratory. Tests should therefore be undertaken for the overall matching of the receiving laboratory results to those of the reference laboratory and for equivalent (intermediate) precision. Accuracy and precision limits are treated independently here for simplicity although they both impact all cases (1A-2B).  

The assay performance is at 3.0% (last n=60) — or potentially higher if control outliers were included — when run under routine conditions with small changes over time. AMV results yielded an intermediate precision of 2.4%. An imprecision higher than 3.0% should not be allowed because the method component is the highest contributor to the overall variance (Table 2). There is already a relatively high likelihood (about 1.73%) of observing an out of specification (OOS) result with two observed over the last 60 (Table 3 and Figure 1). The limit of 3.0% appears balanced between the likelihood of passing to achieve compliance and/or project advancement and the likelihood for all cases 1A-2B to be continued in the future. 			

A recovery or matching of the expected (reference) potency of 100 ± 1.5% appears reasonable. Allowing a maximum of ± 1.5% difference constitutes about one-half of the recent assay variability. The receiving laboratory should not be allowed to test with a greater bias because further increase in the likelihood of OOSs because of the potential shifting in the process mean compared with the target (100 units) may be seen. Overall, recoveries inside this range (98.5– 101.5%) as evidenced by the data from AMV, historical assay control, and SPC should be possible. 

When exchanging or adding a single method component, such as a second instrument, maintaining accuracy or the matching of historical performance should be the main considerations for reasons given above. However, accuracy/matching and precision could both be readily studied to monitor both method performance characteristics. The acceptance criteria for accuracy and precision for AMT and AMM should be derived from the product specifications with regards to assay performance (control) and process performance (SPC data). inebreak/>

Table 3 Estimates of release probabilities and measurement errors (cases 1Aâ2B) for AMT. 

Articles

The first part of this article discussed general strategies for validation extensions to other test method components, laboratories and even different test methods.1 This second part provides practical tips on how to maintain test method suitability long after the formal completion of analytical method validation (AMV) studies.

A guide for testing biopharmaceuticals Part II: acceptance criteria and analytical method maintenance

The successful completion of analytical method transfer (AMT) is a regulatory expectation for the extension of the validation status to other laboratories. The demonstration of equivalent test results and, therefore, an acceptable level of reproducibility when testing at a different location can limit the potential risk to the patient (hence the regulatory expectation). Acceptable reproducibility also limits the risk of failing test results for the biopharmaceutical firm as established probabilities of passing specifications can be maintained. Similar, post-validation changes in method components should be monitored and controlled to avoid significant (negative) changes for material or product release probabilities. 

Analytical method validation (AMV) guidelines exist from several recognized sources.

  Detailed validation guidelines for alternative microbiological test methods also exist.

  In addition, a series of practical tips and discussions for AMV and related topics was recently published.

  However, some topics are currently not sufficiently covered in recognized sources. For example, how can we demonstrate method comparability for new methods, extend the validation status onto other laboratories and/or other test method components, and maintain this validation status over time? What are acceptable levels for differences in method performance and when is a method no longer suitable?  

This article will focus on all post-validation work that may be required to ensure process and product quality over time. The first part, published in this issue, discusses practical concepts on how to ensure successful validation extensions. The second part of this article, to be published in the next issue of 

, will include practical tools to ensure a validation continuum (maintenance) for validated methods. The second part will also include case studies for deriving meaningful and risk-based acceptance criteria for validation extensions and validation maintenance and will, furthermore, include a case study on how to reduce analytical variability in validated systems. 

When replacing approved test methods with improved ones, analytical method comparability (AMC) data should be submitted together with the method description and validation results.

  Once a method is approved and in routine use, it should be maintained in an analytical method maintenance (AMM) programme that can be administered through the validation master plan (VMP).

  If done well, this will ensure — like all post-validation activities — consistent (accurate and precise) production process and product quality measurements.

				  What exactly are the critical elements of good validations, validation extensions, or suitable validation maintenance? The answer lies mostly in the pre-set acceptance criteria for method performance and, of course, the actual validation results obtained. For example, if changes in analytical method components cause a change in test results and ,therefore, in process or product quality measurements, we should capture when we will have exceeded method suitability limits. In other words, if the analytical method change will cause a predictable shift or spread of results with respect to specification(s), and therefore negatively impact the probability of releasing material, we should monitor this. To monitor and possibly compensate, we must first set reasonable suitability limits, then continuously control the overall method performance. Often, the most difficult part may be to estimate the associated risk of changed results with respect to both, patient and firm, and from this, to set reasonable acceptance criteria. Once we truly understand why and when there will be a need for method improvement, we will likely know what should be done to compensate for the difference. 

Each production process has an associated probability for the rate of rejections that can be readily calculated by relating specifications and production process performance. However, instead of having to deal with only two probabilities (pass or reject) that are "visible" and monitored by statistical process control (SPC), we should consider two additional possibilities for all reported results. Therefore, there are a total of four possible cases for releasing product/material of which three should be avoided as often as practically possible. The four cases for reported test results are illustrated here.  

Measured results are within established specifications.

Measured results are outside established specifications (OOS).

Cases 1A and 2A are routinely monitored by SPC. Cases 1B and 2B originate from other uncertainties such as imperfect test method performance or poor sampling and are not readily visible by SPC. Cases 2A and 2B are obviously not desirable because the firm cannot process nor sell this product/material. Case 1B constitutes a risk primarily to patients, but also means a risk to the firm if adverse product-related reactions and/or over-/under-dosing would actually occur. Case 2B constitutes a loss solely to the firm and should also be avoided mainly for profit reasons although other problems may also arise from this situation.  

For our validation extension acceptance criteria, we should primarily set acceptable protocol limits from SPC with relation to specifications. We should consider the likelihood and impact for cases 1B and 2B, and avoid as much as possible measurement errors as part of the AMM programme. Inaccurate and/or imprecise measurements will always cause a lower than ideal probability of observing results within specifications. The acceptance criteria for AMV and its continuum requirements must, therefore, ensure the low likelihood for all cases but 1A. 

Table 2 General guidance for using appropriate statistics to assess method comparability from validation characteristics per ICH Q2A/B.			

To meaningfully estimate risk to patients and the firm, we must understand our process data, and, integrate test measurement aspects into our risk-based validation strategies. Good risk management tools will dictate how much assay performance characteristics can deviate from ideal. This will then set limits on how much we can tolerate over time for a test method to deviate from ideal (100% accurate and precise). It should also now become apparent why it is so important to maintain our validation status with an AMM programme as we may otherwise observe, when this is ignored, that we negatively effect all four cases. (Negative here means increased risk to patient and/or firm). Although undetected, negative effects will occur for the "invisible" cases 1B and 2B because measurement errors are not captured by regular SPC. This may also cause the lack of process understanding and control, and may also lead to conflicts with current regulatory expectations (process analytical technology, [PAT]) and may impact a firm's profits in the long run.

Validated analytical methods can be transferred from one laboratory to another without the need for revalidation at the receiving laboratory.

  A typical AMT is accomplished by testing at the sending and receiving laboratories in a round-robin format. Testing is performed on three different product lots over three days using two operators and two instruments in each laboratory.

  Reproducibility of test results, within and between laboratories, is demonstrated according to Table 1 by evaluating intermediate precision (different operators, instruments, days and product lots at each site) using an analysis of variance (ANOVA) and by comparing the differences in mean results for each lot between both sites.

  For each AMT, pre-set acceptance criteria for intermediate precision and for the absolute differences between sites are derived and justified from the validation at the sending laboratory.

The AMT reports should include descriptive statistics (means, standard deviations and coefficients of variance), comparative statistics (ANOVA ρ-values) for inter-laboratory results, and the differences-of-mean values for both (or each) laboratories. Each report documents evidence that the transferred test method is suitable (qualified) for testing at the receiving laboratory.  

For cases where the ANOVA ρ-value is less than 0.05, secondary acceptance criteria should be established for the comparison-of-means and variability of the results to demonstrate the overall lab-to-lab reproducibility of test results. It is advisable to include a numerical fall-back limit (or percentage) because the likelihood of observing statistical differences may increase with the precision of the test method. In addition, some differences (bias) between instruments, operator performances and days are expected.

  We should tailor our acceptance criteria for overall (intermediate) precision and for the maximum tolerated difference between mean laboratory results (accuracy or matching) to minimize the likelihood of obtaining OOS results (2A and 2B) or 1B results.

  The setting and justification of all acceptance criteria must strike a balance and is a critical part of each protocol. A detailed AMT case study was presented elsewhere.

As we need to demonstrate equality or improvement whenever approved methods are replaced, which and how are method performance characteristics compared? Table 2 provides guidance on which validation characteristics to use for comparability protocols for each assay type per ICHQ 2A/B. All qualitative tests should contain a comparison of hit-to-miss ratios (for "specificity") between the approved method and the new method. If a qualitative limit test is exchanged, the detection limit (DL) of the new method should be compared and should be equal or lower for the new method. For all quantitative methods, the method performance characteristics accuracy and precision (intermediate precision) should be compared.

It is of great regulatory concern whether results may change overall by drifting (change in "accuracy" or "matching") or by an increase in data spreading ("intermediate precision"). An increase in data spreading or lack of precision will mostly increase the likelihood of observing cases 1B or 2B and should be avoided. A drift in results or "lack of matching" may require a change in the specifications. This drift in release results can occur in two directions, lower or higher results. Both directions are not acceptable outcomes for the demonstration of accuracy, and testing for equivalence between methods is therefore correct.

  The goal of comparisons of other characteristics (e.g., DL) is different as at least two outcomes are acceptable that is to be equivalent or superior meaning a lower DL.

  A third comparison category is the demonstration of non-inferiority and is usually the easiest to pass for comparability. However, we should keep in mind that the use of any of the comparability categories (non-inferiority, equivalence, superiority) using ICH E9 and Committee for Proprietary Medicinal Products (CPMP) guidance documents should be properly chosen and justified.

  In other words, non-inferiority testing may be justified for the comparison of some primary characteristic such as DL if other secondary criteria (e.g., increased number of tests or test samples) can compensate for the small level of inferiority of the primary comparison characteristic.

Quantitative limits (QLs) could also be compared. However, both QLs would have to be estimated by the same principle (e.g., estimated by regression analysis). A low QL is desirable as it will let us quantitatively report and monitor low-value results by SPC. There are several ways to compare QLs. For example, we could compare the regression coefficients of both linear assay response curves to estimate both QLs and would also get a general idea how accuracy and precision characteristics compare over the assay range. Table 2 constitutes a general guidance. Particular examples for non-inferiority, equivalence, and superiority testing to demonstrate method comparability were provided and discussed elsewhere.

No matter which comparability category we may use for a statistical comparison (with ρ=0.05), a protocol should provide the design of experiments to be done and the pre-specified value for the allowable difference in results. The pre-specified maximum allowable difference is illustrated in CPMP's 

Points to Consider On The Choice Of Non-Inferiority Margin

  The allowable difference should be set similar to AMV/AMT. The difference should be set and justified by relating specifications to SPC data and with consideration of the likelihood of observing any of the four cases (1A, 1B, 2A, and 2B).

 is manager, QC analytical services and compendial liason at Bayer HealthCare LLC, USA. 

1. Guideline for Industry Text on Validation of Analytical Procedures, ICH Q2A, 60, 1995. 

2. Guidance for Industry Q2B Validation of Analytical Procedures: Methodology, ICH Q2B, 62, 1996. 

The Fitness for Purpose of Analytical Method

, Eurachem/CITAC, Teddington, UK (2003). 

www.eurachem.ul.pt/guides/EC_Trace_2003.pdf

Guidance for Industry, Bioanalytical Method Validation (2001).

Draft Guidance for Industry, Analytical Procedures and Methods Validation (2000). 

Technical Report 33, Evaluation, Validation and Implementation of New Microbiological Testing Methods

8. Alternative Methods For Control of Microbiological Quality, EP. Supplement  5.5 [07/2006:50106] (December 2005). 

12. S.O. Krause, BioPharm International Validation Guide, a supplement to 

13. S.O. Krause, BioPharm International Guide to Bioanalytical Advances, a supplement to 

14. S.O. Krause, BioPharm International 18(

15. Guidance for Industry PAT — A Framework for Innovative Pharmaceutical Development, Manufacturing, and Quality Assurance (2004). 

16. ISPE Good Practice Guide: Technology Transfer (International Society for Pharmaceutical Engineering, Tampa, FL, 2003). 

17. Statistical Principles for Clinical Trials, ICH E9 (1998). 

www.emea.eu.int/pdfs/human/ich/036396en.pdf

18. Points to Consider on Switching Between Superiority and Non-Inferiority, CPMP (2000). 

www.emea.eu.int/pdfs/human/ewp/048299en.pdf

19. Points to Consider On The Choice Of Non-Inferiority Margin, CPMP (2004). 

home.att.ne.jp/red/akihiro/emea/215899en_ptc.pdf

Firms can use practical tools and criteria to implement advanced analytical test methods for biopharmaceuticals. They an also extend and maintain the validation status for a licensed test method from an understanding of the relationship of historical process data, specifications and test method capability.