Scott Stern

Although there is awareness among the biopharmceutical industry of the challenges and trade-offs associated with monitoring the safety of drug products, there does not seem to be a clear and consistent understanding of characteristics correlated with the presence of critical-to-quality deviations of a product at the commercial scale (1). The intent of this paper is to initiate an understanding of product and site characteristics that statistically correlate with quality performance of biopharmaceutical commercial products. Towards this objective, a comprehensive survey on related topics is underway at the Massachusetts Institute of Technology (MIT) Center for Biomedical Innovation (CBI). The interim results described in this research represent survey results from 34 commercial-scale biopharmaceutical products at 11 international sites. This research builds on analysis done by the MIT CBI team earlier this year (2). 

Some regulatory agencies, including FDA, have actively pursued a risk-based approach to manufacturing site inspection because their inspections are site-specific and not restricted to an individual product (1). Explicit inputs that FDA uses for its risk-based selection of sites to inspect include the types of products manufactured (e.g., prescription versus over-the-counter, approved versus unapproved, therapeutic classes); the control and/or contamination potential of the manufacturing environment (e.g., facility size, facility type, number of drug products, sterility requirements); and cost of inspection (e.g., domestic versus international locations). These inputs are logical for consideration of total risk to quality at the level of a site, but, in this research, the focus regarding quality is on the product. 

A comprehensive survey on topics related to site characteristics, quality approaches, quality activities, perception of relative consistency of regulators, product-level process details, and product-level compliance performance has been developed by MIT's CBI. The survey includes questions that result in a total of 57 variables across these topic areas. The developed survey was tested with industry representatives and then deployed over the past 18 months as a secure web-based questionnaire directly to biopharmaceutical manufacturing sites through several different channels. The individuals targeted for inclusion as survey respondents have been manufacturing plant managers or their representatives.  

In total, 20 sites representing 52 manufactured products have completed the survey. These responses have been filtered to focus on commercial-scale manufacturing and products for which all relevant information was provided. This reduced the number of products included thus far to 34 across 11 sites.  

The research is supported by the Alfred P. Sloan Foundation of New York City. 

Characteristics of the products and sites. 

The demographics of the 34 products included in the analysis are organized into the following groups: 

29 of the products surveyed are marketed in the US; 24 in Canada; 33 in Europe; 26 in Japan; 17 in South America; and 20 in Asia (outside of Japan).

Product type: 27 of the products surveyed are therapeutic proteins; 12 are antibodies

Processing: 25 of the products surveyed are produced by mammalian cell culture; 9 by microbial fermentation

Complexity: 15 of the products are perceived as above average in process complexity

Role of quality by design (QbD): 10 of the products are significantly impacted by use of a QbD approach.

Characteristics of the manufacturing site:

Geographic location of commercial manufacture: 6 are based in in Asia; 12 in Europe; and 16 in the US

Contract manufacturing: 14 of the products are manufactured in a site that does greater than 40% of their work as contract work 

Other biologic products at the site: 15 of the products are manufactured in a site that also manufactures more than five biologic products

Use of third-party quality inspections: 24 products surveyed are manufactured in a site that has had multiple third-party quality inspections (for practice) in the past five years.

Based on these overal demographics, the products in the MIT CBI survey represent a reasonably diverse set of biopharmaceutical products, processes, and manufacturing sites. 

Distribution of technical personnel within the manufacturing sites. 

The distribution of technical personnel within a manufacturing site provides some representation of the priorities and/or focus of the site. The sites reported the number of personnel in each of the following technical areas: quality control (QC)—average 12.6%; quality assurance (QA)—average 17.4%; engineering support and services average 8.8%; technical services—average 16.3%; and manufacturing—average 44.9%.  

Figure 1: Fraction of personnel associated with each of the technical areas at different sites. QA is quality assurance; QC is quality control; ESS is engineering support and services; and TS is technical services. (ALL FIGURES COURTESY OF THE AUTHORS) 

A histogram with the fraction of personnel in each of the first four areas is presented in 

. The histogram of fraction of technical personnel in manufacturing is shown in 

. There is a wide distribution of where technical personnel are assigned within the sites (particularly visible from the distribution of technical personnel in manufacturing) suggesting one or more of the following scenarios across the sites that completed the survey:  

Different role definitions for personnel within the technical areas

Structural differences between sites necessitating different distributions of personnel

Different priorities and/or focus of the site management

Lack of understanding regarding optimal distribution of technical personnel. 

Figure 2: Fraction of technical personnel in a manufacturing position across sites surveyed.  

 Each site was asked to identify significant sources, or drivers, of changes in quality efforts and activities at their site. The respondents indicated whether specific drivers had been experienced by the site in the past and/or are anticipated drivers for the site in the future. The results from the survey of the sites are presented in Figure 3, ordered from left to right in descending order of fraction of sites identifying the type of driver as one that has been experienced in the past. Of particular note are the following findings:  

The two types of drivers most frequently experienced were technical challenges (i.e., process, material, equipment) and regulatory challenges (i.e., change from the regulator, noncompliance).

Drivers potentially directly related to profitability of the product (e.g., cost reduction and pursuit of new markets) have been infrequently experienced.

Drivers expected to be experienced significantly more in the future than the past are those directly related to profitability including pursuit of new technology, cost reduction, and pursuit of new markets.

The number of types of drivers expected to be experienced per site is significantly higher than the number of types of quality drivers historically experienced per site, increasing from an average of 40% to 60%. For example, sites expect to have more drivers of change in quality approach and activities and/or have different outcome priorities (e.g., cost reduction and anticipated regulatory change) in the future compared with the past.

Figure 3: Experienced and expected drivers of quality activities and resources.    

Correlations with critical-to-quality product deviations. 

Using critical-to-quality defects at the commercial scale as the dependent variable, 56 other variables were compared based on likelihood ratios for each of the 34 commercial products. Of these 56 variables, 11 were found to have statistically significant different likelihoods depending upon whether a critical-to-quality defect was experienced for that product. These 11 variables of significance are shown in Figure 4 and grouped based on confidence level of statistical significance (the top row contains variables for which the difference in likelihood ratio demonstrated greatest confidence) and grouped based on whether the correlation was negatively or positively correlated with a critical-to-quality defect at the commercial scale. If the variable is in the negatively correlated section of 

, then presence of that variable is more likely to occur for a product without a reported critical defect.  

Table I: Variables that were found to be statistically different in likelihood of correlating with a critical-to-quality defect. 

For example, a product manufactured at commercial scale in a site having an above-median fraction of technical personnel working in QA was statistically less likely to have experienced a critical-to-quality defect at the commercial scale. On the other hand, a positively correlated variable is more likely to be present for a product that has experienced a critical defect. For example, a product manufactured in Asia is more likely to have experienced such a defect. 

Consistent with prior preliminary discussion regarding variables correlated with product defects, many of the statistically significant variables presented in 

 fall into the categories of history of quality issues in the product lifecycle, geographic region of manufacture, and manufacture in a contract manufacturing site (2). Of note is the dependence upon fraction of technical personnel employed in QA and employed in manufacturing. More generally, this dependence is related to the discussion of the distribution of technical personnel across a site as described above, where the lack of consistency and, perhaps, understanding of an optimal distribution was highlighted. The data presented in 

 demonstrates that above median level of the fraction of technical personnel in QA and below median level of the fraction of technical personnel in manufacturing are both correlated with lower likelihood of a critical-to-quality defect at the commercial scale.  

It is possible that greater emphasis on quality is demonstrated by fraction of personnel in QA compared with an emphasis on execution demonstrated by fraction of personnel in manufacturing. Further investigation in this area and clarity regarding the expectations of technical personnel roles could provide greater insight into this observation. 

Although there were 11 variables significantly correlated with likelihood of a critical-to-quality defect at the commercial scale, there were also 45 variables found not to be so. Several of these variables are notable as not having been found to correlate with critical-to-quality defects at the commercial scale, including:  

Type of product (e.g., protein, antibody).

Mode of production (e.g., mammalian, microbial).

Level of production complexity perceived by the site: The first three variables are related to perceived manufacturing complexity and logic could suggest that a dependence is expected, but has not been found in the current data.

Use of QbD: QbD can take on many forms and has a complex impact on measured quality performance metrics. Generally, this lack of correlation is notable as quality by design is expected to increase the likelihood of product with consistent quality over time (3).

Number of product batches manufactured in the past five years: More batches would be expected to increase probability of a single defect occurring but is logically counteracted by familiarity with the product and the possibility of increased learning from completed batches (4).

With the intent of increasing understanding of variables associated with commercial quality performance of individual biopharmaceutical products, a survey was conducted with questions related to the product lifecycle and site characteristics. The interim analysis presented here was based on 34 products across 11 sites with diverse characteristics.  

The analysis focused on determining which product or site variables were statistically more likely to be present for products that have experienced a critical-to-quality defect at the commercial scale. The themes of variables found to be statistically significant based on the likelihood analysis were:  

A history of quality issues in the product lifecycle 

The manufacture in a contract manufacturing site 

Distribution of technical personnel at the site (particularly employed in QA and employed in manufacturing). 

A number of themes not found to be statistically significant for association with commercial critical-to-quality defects, based on the data available, include:  

Articles

MIT survey results address product and site characteristics that statistically correlate with quality performance.