Defining Quality: Joining the Quality Lab and the Plant Floor

For the past few years, to help pharmaceutical manufacturers improve and sustain better product quality, FDA has been working with industry to define the metrics and key performance indicators that are most critical to product quality (1). This work actually began in 2008, with the release of the International Council for Harmonization of Technical Requirements of Pharmaceuticals for Human Use’s  (ICH) Q10 (2), which articulated the need for a systemic approach to quality that would get beyond the case-by-case approach of current good manufacturing practices (cGMPs) and final product testing.

After FDA released initial guidance on quality metrics in 2015 (3), there were complaints about its broad scope. FDA had asked that manufacturers collect data for 10 metrics. In November 2016, FDA released a second version of the guidance (4), which focuses on the following three main metrics: Lot acceptance rate, or the number of accepted lots within a timeframe divided by the total number of lots started, for primary and secondary distribution and packaging, during a given timeframe. Included will be number of lots started, released, and rejected.

Product quality complaint rate, or the number of complaints received divided by the total number of dosage units of that product distributed during that time frame.

Invalidated out-of-specification (OOS) rate, or the number of OOS batch-release test results and long-term stability test results that were invalidated due to measurement process issues at the facility, divided by the total number of such tests performed at the facility during the time frame. Every OOS result will trigger an investigation, and the guidance specifies best practices.

Long term, the agency’s goal is to furnish metrics that will help process operations and manufacturing teams, quality control departments, and regulators (especially plant inspectors) focus on key principles that determine product quality.  These principles should drive, not only day-to-day operations, but also investment in new technology. In addition, they should help regulators prioritize inspections to focus on facilities and companies that are at the highest risk of quality or compliance failure.

In July 2016, FDA began to fund research that aims to analyze existing quality metrics, and to see whether new measurements that incorporate more of the language of the manufacturing plant floor and the principles of operational excellence might help achieve better results in the future (5).

Working on this project is a team at the University of St. Gallen in Switzerland, led by Thomas Friedli, who has spent the past 15 years studying the application of continuous improvement techniques across different industries. Collaborating with Friedli is a team from the Dublin Institute of Technology, led by Nuala Calnan, and, based in the United States, pharmaceutical industry consultant Prabir Basu, who, for more than 10 years, was head of the National Institute for Pharmaceutical Technology and Education (NIPTE).

Friedli’s team has been analyzing pharmaceutical manufacturing for well over a decade, based on the universal metrics used in automotive, aerospace, and other industries, which include “on-time delivery” and inventory levels.  St. Gallen’s surveys of pharmaceutical manufacturing operations look at such things as whether the facility or company uses total predictive maintenance, or how effectively the workforce is engaged in, or how strongly senior management supports, total quality improvement.

Some pharmaceutical companies have been reluctant to embrace such universal manufacturing excellence metrics, with some insisting that “pharma is different,” due to the special requirements for product safety and testing. There has been debate over this topic for decades, and, even today, one sees uneven acceptance of such concepts as “process capability analysis” or process analytical technology (PAT) among drug manufacturers.

At this point, FDA wants to see whether the language of operational excellence can further enrich the industry’s understanding of quality.  Research is still in a preliminary stage, and could not be discussed for this article, but Prabir Basu shared some of his thoughts on what the industry will need if it is to redefine, and transform, pharmaceutical quality control.

Operational excellence

PharmTech: Why is operational excellence (OpEx) so important to improving both pharmaceutical manufacturing and quality?

Basu: Quality and operational excellence cannot be separated. Operational excellence metrics show how motivated people within a company are to improve overall performance, and quality with it.

FDA has, in the past, taken an approach that has separated the two, as if quality were not a part of operations. The point is, that if a company is not investing in quality, that will show up in the operational excellence parameters, and they will have quality problems too.

A great example is preventive maintenance. If you don’t have a corporate mandate and policy for this activity, you are very likely to have problems with product quality.

PharmTech: How does all this affect regulators?

Basu: Having links to existing OpEx quality parameters would be very helpful in ensuring that FDA can get pertinent information underlying deviations or batch rejections.
Today, most manufacturing is taking place outside of the US, and FDA has limited ability to inspect all the facilities involved. Having indicators in place that are operations related and that suggest which facilities and companies might pose a higher risk of noncompliance or low quality will allow FDA to prioritize inspections. But it would be ideal if we could get to the stage where quality and operational excellence are considered as one. ICH Q10 gives us indications of how to get there.

So, we are beginning a journey that has much potential, and we can get to this goal of a unified definition of quality, if we continue for the next three to five years.

At this point, we are collecting data using benchmarking questionnaires, correlating between existing operational excellence measurements and quality metrics. We hope to expand the questionnaire to reflect on real quality information.

PharmTech: In the past decade, we’ve heard more people in pharma talk about Deming’s approaches to excellence, yet concepts as basic as process capability don’t seem to have been widely embraced. Why is that?

Basu: Some companies are working with the concept of process capability, but they tend to be the larger companies, such as Amgen and Pfizer. And even the larger companies don’t employ this approach for all products.

We need to come up with metrics that will be attractive for all companies.

PharmTech: How about cost of goods?

Basu: That measure is too variable, because the cost of capital varies so dramatically depending on which country the facility or company is based in.

Quality metrics

PharmTech: How about the metrics that FDA is focusing on in its latest version of the draft guidance, including out of specification? Are these adequate?

Basu: The metrics are okay but they aren’t yet tied to processes so they won’t necessarily reflect what is going on internally. For example, lot acceptance rate seems okay, but what happens if lots have to be reworked? Will the figure then be a true reflection of the facilities’ processes?

PharmTech:  What are some concepts that might be more helpful?

Basu: I think that Six Sigma value could provide a better indicator. In the early 2000s, a number of thinkers used to talk about doing this, but it hasn’t yet been fully accepted.

Even if we were to use the lot acceptance rate as a quality metric, ideally some measure of accuracy could be factored in, for example, of the number of batches started, how many came out right the first time, without the need to rework them?

PharmTech: What role should quality complaints play?

Basu: This is an important metric, but first we need to define very clearly what the complaints are and where they are coming from. Are they coming from the warehouse? From distributors? From patients? From regulators?

Some of the metrics that are currently being discussed may not adequately reflect internal processes. The beauty of operational excellence metrics is that they measure how well processes are performing, so they are much better reflections of the actual situation within a given facility or company.

Some of the important operational excellence metrics to consider are on-time delivery and customer satisfaction. These measurements reflect internal processes.

In addition, I believe that metrics must incorporate more of the spirit of ICH Q10, to determine the company’s quality culture, and such things as whether the firm has a continuous improvement program in place, whether its senior management is involved in quality, the degree of employee involvement, and how the company prioritizes projects for improvement (ie, whether it uses ICH Q9 and principles of risk assessment to help make those decisions).

The most important question is whether the facility’s or company’s processes are in a state of control.  Here, key indicators are measures of variability of the critical process attributes. Even if process capability information is not available, at a minimum, trending of critical process variables and data on process drift such as shifting of the averages or changes in slopes of the trend, degree of implementation of ICH-Q9, etc. We need metrics to measure these areas, and also to identify facilities and companies that are at the greatest risk of quality and compliance failure.

In the end, the number of rejected batches may be more important to screen than lot failures. In addition, relative numbers are more important than absolute numbers. For instance, the top 25% should have good systems in place, and the bottom 25% should receive more attention from FDA.

It might be most beneficial to use the pillars that St. Gallen has been using to measure performance: total predictive maintenance, total quality management, and just-in-time inventory levels.

Keeping a focus on process operations will ensure that companies and regulators are monitoring operational principles, looking at stabilizing systems, and developing frameworks for knowledge management and risk management. By definition, these efforts can only make any organization more focused on product quality. 

References

1. Measuring Pharmaceutical Quality through Manufacturing Metrics and Risk-Based Assessment, Meeting Summary, brookings.edu, May 2014. 
2. ICH, ICH Q10, Pharmaceutical Quality System (PQS), 2010, ich.org.
3. M. Mezher, “FDA Releases Long-Awaited Quality Metrics Guidance,” raps.org., July, [2015], July 28, 2015.
4. FDA, Submission of Quality Metrics Data, Draft Guidance for Industry, November 29, 2016, FDA.gov, November, 2016.
5. T. Friedli and P. Basu, “Measurement of Pharmaceutical Quality in an Operational Excellence Environment,” in CPhI Annual Industry Report, 2016, p. 15.

Article Details

Pharmaceutical Technology
Vol. 41, No. 1
Pages: 60–62

Citation

When referring to this article, please cite it as A. Shanley, “Defining Quality: Joining the Quality Lab and the Plant Floor," Pharmaceutical Technology 41 (1) 2017.