Guy Villax

It is now more than a year since FDA brought out its 

 notice asking input from the industry on quality metrics. The goal is to stratify along increasing levels of quality all the manufacturing sites regulated by FDA. 

exemplifies the current view on the matter. This paper argues that while the direction is right, the approach is too complex and unnecessarily detailed. Moreover, two major ingredients are missing from the debate. First, quality culture is barely addressed. FDA, CEOs, and regulators in general have not devoted much attention to this primary driver of product quality. What is uppermost in their mind is the verification of compliance (i.e., the effect of quality), and the current debate seems limited to measuring quality system performance. Second, the willingness to learn from others is absent. There is no evidence that anyone wants to go beyond their comfort zone.

Over the years, FDA has been shown repeatedly to be a thought leader. For example, FDA demanded GMPs for APIs 40 years before such a requirement became law in Europe. FDA has also been behind process analytical technologies (PAT) and quality by design (QbD). Major medicine breakthroughs, such as Vertex’s Kalydeco (ivacaftor) for the treatment of cystic fibrosis patients, would not have been possible without FDA’s innovation of the breakthrough therapy designation. The stratification that FDA is seeking to implement must happen, but it needs a leap of faith and major innovative thinking in a direction that is different from current proposals.

FDA’s thermometer is one that only measures negative temperatures. FDA, as well as other regulators, tends to focus on the “floor” with medicine agencies explicitly defining a threshold level below which quality performance is deemed unacceptable. But what does FDA issue when it wants to say “Well done!”? Presently there is no FDA form for “Well Done.” Yet FDA is keen that we move in the direction of its 21st-century vision with concepts such as process understanding, risk management, and supply reliability. FDA has nothing in its arsenal to encourage the proactive pursuit of greater manufacturing quality.

At the same time, sanctions for non-compliance, especially when non-compliance is a business strategy, are insufficient to be an effective deterrent. Because the type of sanctions hurt primarily shareholders and not management, rogue behavior is encouraged by positive payback. Because non-compliance pays, rogue players often win. For example, has anyone been caught (let alone jailed) for causing hundreds of deaths in the heparin tragedy?

Safety and quality have much in common, with both being crucial aspects of industrial activity. Both depend directly on constant management attention and require the right tone at the top. Yet in terms of metrics, safety is easy (while quality is complex). The number of deaths, lost time due to accidents, the gravity and frequency indices are well-established metrics and are sufficiently comparable across all sectors. What we are trying to measure is simple. Research has also taught us to expect correlations between near-misses and accidents.


The Occupational Safety and Health Administration (OSHA) is the USA’s authority that enforces safety and health in the workplace. It performs regular compliance surveillance through surprise inspections. OSHA’s voluntary protection program (VPP) rewards workplaces that have consistently no lost time accidents by removing them from the inspection roster. VPP Star program recognition drives appropriate behaviors--a reduction in insurance premiums is a financial benefit, but paramount is the pride workers have in a safe workplace, and no effort is wasted in maintaining that status.

Just like FDA, OSHA has defined where the “floor” lies. Below such minimum acceptable safety level, sites get closed down and fines are levied. But OSHA’s thermometer also measures positive temperatures--the VPP Star program identifies the role models. OSHA thus has a three-level stratification of the safety performance of all the sites it regulates. The VPP Star program is an effective reward mechanism and a confirmation of a sound and vibrant safety culture.

The metrics used to assess the safety of workplaces exhibit the attributes that FDA is looking for, which include “not amenable to gaming, objective, cross all sectors, are non-intrusive and relevant” (1). It is simple, has virtually no cost, and enables the deployment of inspections of resources where they are really needed. The quality metrics currently under discussion involve setting up systems that are complex, require annual data input, need supervision and funding, and will necessarily introduce behaviors focused on improving “the site’s position in the league table” rather than just doing the right thing for the patient.

In accidents, harm to the worker is a simple common denominator. No comparable measure exists in quality. All the metrics that have been proposed may be objective and may ultimately correlate with recalls and adverse events, and elegant mathematics may make them comparable across sites and across all sectors of the pharmaceutical industry. Such metrics, however, are unable to measure whether the CEO is really committed to developing and nurturing a quality culture in his or her organization.

Sections 705 and 706 of the FDA Safety and Innovation Act (FDASIA) mandate that metrics be made available so that FDA can deploy its inspection resources appropriately. It does not say it needs to be complicated or costly, or that FDA cannot get inspiration from an OSHA-type VPP Star program. FDA could publish a simple list of manufacturing sites that are role models. When this list is published, FDA will have done enough to define the right behaviors and will have done so at a low cost.

The metrics being discussed tend to focus on the deciles from the bottom all the way to the top, which is both unnecessary detailed and costly. The only stratification the industry has today is a well mapped-out lower level, below which quality is unacceptable. What is needed now is a definition of an upper level, beyond which the quality performance is in the right direction and deserves recognition and applause (see


For FDA to establish a stratification of its sites in at least three classes, such as OSHA, it will need to define the border between “being in compliance” and “being ahead.” And those sites that make it beyond that threshold join what has been described as Commissioner Hamburg’s Dean’s List (2).

 It is unlikely that the shape of the distribution today approximates the normal distribution because there is nothing defining the right side of the bell shape. Once a driver of behaviors, such as a Dean’s List, exists, one could expect a normal distribution.

This paper argues that metrics pulled from a quality system alone will not help define where Commissioner Hamburg’s Dean’s List starts. The industry and regulators need to start acknowledging that what genuinely assures compliance and then takes an organization to levels of quality beyond the minimum is the organization’s quality culture. An organization’s quality culture is embedded in the values of the people that make any site function. A quality system will only assure quality if it is backed-up by a well-nurtured quality culture that permeates the entire manufacturing organization. Without a culture of quality, a quality system assures nothing. An article from

 magazine, “Dirty Medicine,” provides a good description of an organization where incidents of fabrication of compliance evidence were endemic (3). In other words, in an organization that has no culture of quality, it is not a system of controls and documents that will safeguard quality.

The large number of warning letters and import alerts issued to sites based in India in the past two years may not be statistically significant given the considerably large number of FDA sites in that country. What is both significant and of concern is that they are all related to data-integrity issues. The chief executive of an Indian drug maker was recently quoted as saying, “When a company is small, it can be managed by strong supervision. As companies get bigger, supervision can break down … You need systems and a culture to maintain proper supervision—and we are in that process of growing up, I think” (4). Indian generic companies’ grew at a rate so fast that the nurturing of such quality culture needed to be miraculous.

The agency should consider whether an organization that has a culture of fraud instead of a culture of quality can ever be reformed, or whether it should be given a second chance (5). The agency should also consider whether drug shortages are not inevitable when unscrupulous companies compete unfairly in the market through systematic non-compliance and destroy the competitiveness of compliant firms. A regulator’s job is not limited to the oversight of the quality of drugs. Regulators must take decisive action when non-compliance becomes an obvious competitive advantage that hurts the fabric of the industry. “A firm can have all the SOPs, systems, and controls required but, without a quality culture, product quality and business continuity are not assured” (6).


A quality culture is not the result of procedures, specifications, audits, a quality unit, or enforcement by regulators. Although some of these elements do help establish a quality organization, by themselves, they are insufficient.

It is surprising that of the 128 comments to FDA’s docket, only Pfizer’s refers explicitly to quality culture as preeminent: “The indicators mentioned above are only useful when coupled with a strong quality culture, which we believe is the foundation of consistent quality performance” (7).

The conditions where a quality culture  will develop are likely to include:

The right tone at the top, a focus on the right values

A constant display of concern for quality, for putting the patient ahead of everything else

An organization that consistently “walks the talk” 

An organization with great people skills and strong values 

Shareholders who expect more than just financial rewards.

 The indicators of a strong quality culture include: 

Quality-driven productivity brought about by sustained change through the introduction of improvements and innovation

Adoption of and investment in technology including IT to ensure a better state of control over systems and timeliness of event follow-up 

Prompt adoption of best practices, promotion of change, and continuous improvement in the quality system itself

Pro-active and transparent behavior with regulators leading to a rich relationship with regulators based on dialogue, sound science, and mutual respect 

Intense participation in the standard setting-process

Good performance in other areas such as safety, business conduct, and sustainability

  The degree of maturity of the quality culture will correlate with where the site is in terms of the following journeys: 

An evolving mind-set that accumulates as second nature habits such as deviation recording, change-control, trend analysis, risk management, process understanding that defines the space of process capability, and so on

Evolving from a rigid machine organization with inflexible quality systems towards a rationale-based approach that adjusts controls appropriately to each situation.

These aspects mirror the maturity of the quality culture in an organization. As the quality culture and system of an organization evolve and mature, the cost of quality diminishes and its capability becomes greater and more efficient. This approach can help map the progress of a quality culture and could be used to define the criteria and boundaries at the right of the bell shaped curve.

 An organization with a sound quality culture is easilyrecognized as it believes in the following two principles:

Product and process understanding (e.g., sufficient depth of science that provides an explanation of why a process deviation has or does not have impact on the quality of the product).

Sensible business management—Too fast growth and operating at the limit of capacity are often triggers of quality issues or of non-compliance. It is obvious that in many Indian generic-drug companies the product portfolio growth outstripped the company’s quality system. Senior management failed in its oversight. 


The following additional attributes of a manufacturing site could be used by FDA to set the bar for that site to be listed in Commissioner Hamburg’s Dean’s List:

A solid track record of consistent good performance at FDA inspections (frequency of successful inspection in the recent past)

Frequency of filings in the recent past that include a growing pattern of QbD approaches and the application of PAT controls

Providing FDA with IT access to perform at any time and in real-time remote inspection of all quality data related to commercial batch manufacturing and release (including seeing deviations, change control and analytical data, and trending thereof, and the measure of timeliness to close events)

Visible active participation in the standard-setting process 

Allowing FDA inspector training at the site.

Most importantly, FDA should add a further element to its compliance inspections. Inspectors should determine the existence and maturity of the quality culture in the inspected site. Simple cross-examination of operators and candid conversations with management at various levels will quickly tell what tone is set at the top, what costs of non-quality are accepted, and how much power the quality assurance department has over budgets, over purchasing, or finance departments decisions. An assessment of quality culture should be central to any quality metric assessment—this has to be done by FDA on-site during an inspection.

In lieu of the complex metrics that are being discussed, the industry and FDA could agree on what is the roadmap to reach the Dean’s List standard, what are the desired minimum attributes that would allow a site to apply to be invited into the program. It would be up to the site to check whether it meets the criteria, and application would be voluntary. To be accepted as an applicant would require a successful inspection that would assess the maturity of the quality system and the vibrancy of the quality culture. This accepted applicant status itself will be a great motivator to improve quality performance and inspection readiness, while also having a marketing value. It does not require much additional financial cost to the agency and allows any plant to set its sights on an ambitious goal.

Finally, for those that make it to the Dean’s List, FDA needs to go public and be able to say, “Well done!” and publish the names of those sites that made it onto the list. FDA has to decide how it will do this and also how it removes sites from the list (the same way as the Michelin or Zagat guides give and remove stars to restaurants and neither justifies it).

Being on this public list of role models does not need to trigger any special treatment as in the case of OSHA’s VPP Star program. Being listed will be in itself a worthwhile reward. Being listed will be a relevant element in FDA’s risk assessment and many new drug applications or abbreviated new drug applications are likely to be approved without pre-approval inspections if their product is produced in a listed site. FDA needs its best manufacturers to be profitable and to grow. A transparent and public three level stratification of quality performance will enable a better site to charge a price that rewards enhanced quality.

It is disappointing to hear that many in the pharmaceutical industry believe that a site’s quality metrics and ranking should remain confidential. When Deborah Autor of Mylan argued for a site’s ranking to be made public at a recent public meeting, other representatives of large multinational companies said that they were completely opposed. This is wrong. Transparency has become a motto of the 21st century. Our industry needs to change. Of course none of what is being proposed is in FDA’s comfort zone but as the breakthrough therapy designation and other examples illustrate, FDA’s greatest accomplishments happen when it innovates


1.	GPhA/FDA CMC Workshop Meeting (Bethesda, Maryland, June 2013).
  2.	Guy Villax, 

,” Fortune, accessed Aug. 19, 2014.
  4.	Reuters, “FDA bans imports from Sun Pharma plant in India crackdown,” Press Release, March 13, 2014. 
  5. 	IPQ, “

FDA Data Integrity Concerns Continue in India as Three More Firms Draw GMP Warning Letters

,” accessed Aug. 19, 2014.
  6. 	Mary Oates, Pfizer, “Risk-based approach to quality oversight in contract manufacturing - outsourced activities,” presentation at the APIC Annual Conference (Budapest, November 2011)
7.	Pfizer’s March 13, 2014 Response to Docket FDA-2013-N-2014. PT


Guy Villax is CEO of Hovione FarmaCiencia SA, Sete Casas, 2674-506 Loures, Portugal,

Articles

Quality is a matter of culture more than metrics. In this opinion article, the author presents arguments for an FDA Dean's List as a means to nurture a quality culture within the industry.

Guest Viewpoint: A Case for an FDA Dean's List

FDA is moving in the right direction in terms of implementing stratification by levels of compliance across pharmaceutical manufacturing sites. However, quality is a matter of culture more than metrics. In this opinion article, the author presents arguments for an FDA Dean's List as a means to nurture a quality culture within the industry.

A Case for an FDA Dean's List

Innovators (Big Pharma and biotech companies) are today a minority player supplying only about 20% of the world’s medicines. At present, the vast majority of medicines are generic--the fruits of the best science of the 20th century can now be enjoyed for a few cents a pill. 

Tax strategies were the one common manufacturing strategy of Big Pharma, and we can expect manufacturing outsourcing to be the next one. This will happen not only because of speed and the need to ramp-up production without capital expenditure, but also because new compounds have issues that can only be solved by complex manufacturing technologies. Today, no organisation can afford to have all the new tools and relevant competences in-house.

Competition and pressure on margins in generic drugs have caused the delocalisation of their manufacture. In the next decade, India and China will be the pharmacy to the world. The concern, however, is not around science and technology in manufacturing or capacity but in the absence of a culture that drives quality and compliance. Regulators are up-in-arms taking action, issuing warning letters, import alerts and fines to a large number of Indian firms, mostly because of data-integrity issues. But that is unlikely to solve the matter--real quality, like safety, requires the right tone at the top.

Another disconnect is in the standard setting process, which is still in the hands of Big Pharma even though the majority of pills are generic. The question is, are we at risk that the standard becomes disconnected from reality? 

The complexity and porosity of the generic-drug supply chain and the cost of quality have created windows of opportunity for criminals. Falsified pharmaceuticals is the fastest growing pharma segment. Those purchasing their medicines from Internet pharmacies have a 96% probability of getting a wrong, illegal, product that is a threat to patient safety. And this problem is only the tip of the iceberg. When illegal APIs get into the legal supply chain, tragedies occur, for example, the hundreds of children who died as result of contaminated cough syrup in Haiti and Panama, and the case of more than one hundred Americans killed from contaminated heparin. Once inside the legal supply chain, nobody can differentiate between the good and bad medicines, unless the toxicity is acute and by then, it is already too late. 

The armed robbery of trucks carrying palettes of pharmaceuticals (innovator as well as generic medicines) happens weekly in Brazil, Mexico and Italy as highlighted in supply-chain security meetings of Rx-360. A pharmaceutical warehouse theft now has the record for the largest goods theft at $70 million in one heist (1). 

The Falsified Medicines Directive brought a major innovation. The “written confirmation” is the first example of a mechanism of true global regulator collaboration. Regulators are now watching each other’s backs. However, if we want effective solutions and faster pace of progress, we will need tougher sanctions, such as criminal penalties for individuals. The deterrence of jail sentences and debarment of CEOs that do not do the right thing would serve patients well.


  1. NBC News, Robbers take $70M in prescription drugs, Press Release, 16 March 2010. 

A breach in the pharmaceutical supply chain is only the tip of the iceberg.