Behind the Headlines Episode 27

Ask the Expert

Small Molecules, APIs, and Excipients—Trends, Challenges, and Opportunities

Drug Digest September 2025

Artificial intelligence, machine learning, and novel instruments are helping drug developers.

Enhancing Bioanalysis and Drug Development with Advanced Tools and Technologies

J.P. Morgan Report Lists Biologics, Obesity, China

Top Trends Reshaping Pharma R&D

Communication between the manufacturing plant and retail stores. | Image Credit: © Cagkan - © Cagkan - stock.adobe.com

TriLink BioTechnologies, a Maravai LifeSciences Company, announced on Nov. 2, 2023 that it has launched the Analytical Sciences Center of Excellence (ASCE). The new ASCE hub expands the company’s suite of analytical testing capabilities and, according to the company, will minimize cost and delays associated with outsourcing testing to multiple laboratories. The ASCE, which is located in Maravai LifeSciences’s Pacific Center Boulevard facility in San Diego, Calif., features a 4000-square-foot lab to support plasmid DNA and messenger RNA (mRNA) manufacturing and will be responsible for developing analytical methodologies such as mRNA fingerprinting and sequencing.

“TriLink's new Analytical Sciences Center of Excellence will be a transformative force in the development of nucleic acid therapies,” said Khaled Yamout, senior director of Analytical Services at TriLink, in a press release. “This new lab will allow us to expand our analytical capabilities to meet the growing demands of the market, including specific method development.”

TriLink has developed and qualified 10 unique methods for mRNA characterization with 40 various constructs. The ASCE’s new instrumentation to improve nuclear magnetic resonance, next-generation sequencing, and lipid nanoparticles characterization builds on the company’s method development for construct-specific assays.

“We continue to witness exciting growth in nucleic acid therapeutics—and rapid expansion calls for rapid innovation,” added Kevin Lynch, TriLink’s vice-presdient and general manager of GMP Operations, in the press release. “The ASCE’s centralized analytical capabilities and development of analytical methodologies for the characterization of nucleic acids will advance the industry as we continue to find ways to meet the needs of developers working on life-saving therapeutics.”

TriLink, in addition to the new hub, has an extensive mRNA capping portfolio, advanced scale-up capabilities, and expertise in mRNA, oligonucleotide, and plasmid production. It has been a leader in nucleic acid and mRNA solutions for more than 25 years and provides chemical and biological experiences, contract and development manufacturing services, and readymade and custom materials.

Articles

The new centralized hub will provide advanced testing of nucleic acids, which is expected to simplify mRNA substance testing.

TriLink BioTechnologies Launches Analytical Sciences Center of Excellence

Feliza Mirasol is the science editor for 

On Oct. 30, 2023, Aragen, a contract research development and manufacturing organization (CRDMO), announced that it was setting up a new biologics manufacturing facility in Bangalore, India as part of a $30 million investment.

The new 160,000 m2 facility will include process development labs and multiple good manufacturing practice (GMP) suites as well as supporting functions such as quality control labs (analytical and microbiology). The GMP manufacturing suites will be equipped to conduct intensified bioprocessing using single-use bioreactors and will have advanced downstream purification capability, according to a company press release.

The facility will be designed to offer integrated solutions across the biomanufacturing lifecycle—from process development to process validation, analytical development, pilot production, large-scale drug substance manufacturing, and stability services. The facility will also be equipped to develop and manufacture monoclonal antibodies, therapeutic proteins, and fusion proteins to augment the company’s bioproduction capability in California.

The process development laboratory is slated to be operational by the third quarter of 2024. The first manufacturing suite will open in the third quarter of 2025. The facility will include plasmid DNA, messenger RNA, cell and gene therapy, microbial manufacturing, and additional GMP manufacturing suites in the future to meet demand.

“With the opening of our bio-manufacturing facility, we will empower our customers with even faster development timelines and support their full lifecycle journey from clone development to cell culture manufacturing …, ” said Subodh Deshmukh, CEO, Aragen Bioscience, in the press release.

“Setting up of a biologics manufacturing facility in India is part of our forward integration strategy for our US-based biologics business, allowing Aragen to offer integrated gene-to-GMP solutions to our customers. Once this facility is commissioned, Aragen will offer a unique value proposition to our biologics customers where the early stage discovery will be executed in our R&D labs in California … As the molecule advances, Aragen can tech-transfer the program to our … manufacturing facility in Bangalore, offering better COGS [cost of goods and services] to our customers,” said Manni Kantipudi, CEO, Aragen Life Sciences, in the press release.

As part of a $30 million investment, Aragen is setting up a new biologics manufacturing facility in Bangalore, India.

Aragen to Establish $30 Million Biomanufacturing Facility in India

Chris Spivey is director of Industry Relations and Strategic Partnership, MJH Life Sciences.

Felicity Thomas is the former Associate Editorial Director of Pharmaceutical Technology, Pharmaceutical Technology Europe, BioPharm International.

Visual inspection is an integral process in the production of filled and sealed pharmaceutical products to ensure there are no particles or contaminants in the finished parenteral dosage form. Currently, both manual and automated processes are used for visual inspection; however, automated systems can be expensive to implement and not always accessible for smaller companies dealing with smaller batch sizes.

Accessibility of automated visual inspection solutions is a key focus for Luo Automation, a Dutch company researching, developing, and producing automated visual inspection machines. To learn more about the company and how it is making visual inspection more accessible, 

 spoke with the company’s founder, Joren van der Horst, at CPHI Barcelona 2023.

"We make visual inspection [solutions] for very small batches,” explained Horst. As with all visual inspection technologies, the primary aim is to check the quality of the medicine and make sure there are no foreign particles or contaminants present, he added.

To perform the inspection, the finished dosage form—syringes, ampoules, and vials—are swirled and then any movement of particles within can be detected, Horst continued. “Another cool thing is the [current interest] in artificial intelligence and machine learning,” he said. “We use machine learning algorithms that are pretty new to also detect static defects. So, not only particles but also leakages and such.”

For Horst, a major differentiator for Luo Automation within industry is the focus on smaller-scale inspection. “If you want, you can buy a huge inspection machine that will cost you a couple of million, but then you need to produce 100 million containers per year,” he stressed. “But, if you have clinical trials or personalized medicine, you cannot [perform the inspection] with a huge machine.”

Luo Automation was featured at the inaugural Start-Up Market on the show floor at CPHI Barcelona 2023. CPHI Barcelona ran from Oct. 24–26, 2023.

Videos

Pharmaceutical Technology Europe® spoke with Joren van der Horst from Luo Automation, a Dutch company focused on automated visual inspection solutions, at the inaugural Start-Up Market on the show floor at CPHI Barcelona 2023.

Making Visual Inspection Accessible with Luo Automation at CPHI Barcelona

Within the highly regulated pharma industry, adoption of new technologies is generally slower than in other industries, as such, despite three-dimensional (3D) printing being available for some time now, its use in the production of medicines is still very much a novelty. However, 3D printing technology has a great potential within the pharma industry, particularly for personalized medicines.

FABRX, a small start-up company that was created by academics from the University College London in the United Kingdom, is focused on the use of 3D technologies in the future of medicine. At CPHI Barcelona 2023, 

 spoke with Carlos Rial Calvo, lead software engineer of FABRX AI—a subsidiary of FABRX based in Spain and started in 2023—to learn more.

“Our aim is to personalize medicine for patients,” explained Rial Calvo. “This is very useful, for example, for highly potent drugs that require very specific dosing.” Additionally, using 3D printing technology to personalize medicine is very useful for pediatrics, who can find swallowing medication difficult, so, Rial Calvo pointed out that it is possible to produce medication that is easier for children to adhere to.

Similarly, for patients that need to take multiple tablets, Rial Calvo noted that with the 3D technology it is possible to create polypills—a combination of different APIs in one printlet (3D printed tablet). “We create polypills, and then [the] patient only has to take one tablet,” he said.

FABRX is currently collaborating with hospitals across Europe, Rial Calvo specified. “For example, we are running a clinical trial in an [oncology] hospital in France to combine an anti-cancer drug with a drug for the side effects [of the cancer treatment] into the same pill,” he stated (1).

In another clinical study, the company has worked with the University of Santiago de Compostela and the University Hospital in Santiago de Compostela in Spain to develop a 3D printed medicine for a rare disease, Rial Calvo revealed. For this study, FABRX prepared doses of isoleucine for children with maple syrup urine disease—an inherited metabolic disorder (2). Using the 3D printing technology, flexible production of a personalized dosage form was possible.

“We provide the software for controlling the machines, but also the software provides features to help with the development of new formulations,” Rial Calvo concluded. “So, the researchers can use our software as well, with the machine, to test new materials.”

1. FABRX. FabRx and Gustave Roussy Enter into an Agreement to Develop a Novel, Personalised, Multi-Drug Dosage Form for the Treatment of Patients with Early-Stage Breast Cancer. 

, Press Release, June 25, 2021.
2. FABRX. FabRx Performs the First Clinical Study Using Personalised 3D Printed Dosage Forms to Treat Children with a Rare Metabolic Disease. 

Fabrx was featured at the inaugural Start-Up Market on the show floor at CPHI Barcelona 2023. CPHI Barcelona ran from Oct. 24–26, 2023.

Pharmaceutical Technology Europe® spoke with Carlos Rial Calvo from FABRX, a pharmaceutical 3D printing company, at the inaugural Start-Up Market on the show floor at CPHI Barcelona 2023.

3D Printing Medicines with FABRX at CPHI Barcelona

H. Gregg Claycamp, MS, PhD, is a Risk and Decision Analysis Consultant, hgclaycamp@gmail.com.

concept of auditing and evaluating quality and efficiency of personnel, business document evaluation process ,inspection of business finance tax documents, Data analysis reports growth results | chaylek - stock.adobe.com

This paper updates “Expert Judgments in Quality Risk Management: Where Quality Risk Management can ‘Go Wrong’” (1), in which several authors writing on the first 10 years of the International Council for Harmonisation’s (ICH’s) Q9 discussed controlling “subjectivity” in quality risk management (QRM) as a persistent challenge to ICH Q9 implementation. Indeed, “[t]he robustness and validity of a risk management program depend in part on recognition and control of the subjectivity that might be introduced by the many expert-based judgments for decisions that occur during the development and use of risk tools” (1). The revised guideline, ICH Q9 (R1), provides a new section of specific comments on subjectivity, “Managing and Minimizing Subjectivity (5.3),” adjacent to an expanded discussion about the meaning of formality in QRM (2). This paper discusses some of the approaches for controlling subjectivity using elicitation of expert knowledge and probabilities.

This article was peer-reviewed by a member of

The general means for controlling subjectivity in QRM and for risk-based decision making (RBDM) include:

raising awareness about subjectivity through training on probability and expert judgments of subjective probabilities, Heuristics and
biases, and possible impact of subjectivity on RBDM

using formal calibration of individual experts using Cooke’s classical method or other frameworks to elicit subjective probabilities using seed questions for expert calibration, elicit subjective probabilities for the 

 situation, and use formal scoring rules to weight experts’ subjective judgments using subjective probability elicitations in group settings with facilitation, such as Delphi technique, value-focused thinking, and multi-criteria decision making (MCDM) processes.

The first approaches for controlling subjectivity were discussed in the prior paper on ICH Q9 (1). In this paper, the last two approaches are visited, while recognizing that this list is certainly not an exhaustive summary of the many decades of research and application of methods for controlling subjectivity in risk analysis and decision-making under uncertainty.

RBDM often relies on experts for qualitative knowledge about likely sources, predictive factors (parameters), appropriate models, and approaches for controlling identified risks. For complex and significant risks to the patient, RBDM also calls upon experts “to provide quantitative probabilistic assessments of key uncertainties in an analysis when empirical data are unavailable, incomplete, uninformative, or conflicting” (3). This general view of expert elicitation is discussed in the following paragraphs, especially in the context that “[f]ormality in quality risk management is not a binary concept (i.e., formal/informal); varying degrees of formality may be applied […] when making risk-based decisions (3) at §5.1).” In other words, there is no distinct boundary between when qualitative or quantitative elicitations of expert knowledge and probabilities will suffice for a given risk-based decision.

Elicitation of expert knowledge and probabilities

Expert elicitation methods are used as a means of controlling subjectivity and thereby help reduce uncertainty arising in risk management decision-making. For this paper, “expert elicitation” (EE) is used as a general term for processes ranging from a relatively informal and qualitative elicitation of the structure of a risk problem to mathematically formal expert elicitation of risk factors (e.g., variables and parameters), their possible values, and the uncertainties about those estimated values. The latter, known as structured expert judgment (SEJ), is nearly a discipline itself attributed to Roger Cooke and a long list of his colleagues and research collaborators (3–6). SEJ processes apply scientifically justified methods for eliciting experts’ quantitative knowledge to fill critical information gaps in the risk assessment and risk control evaluations.

 The first concept in this use of EE is the qualitative (and possibly informal) elicitation of expert knowledge to help identify decision-making objectives, values, model structure, key risk factors/parameters, and appropriate QRM tools and controls. Here, the SEJ literature is relatively silent on identifying or defining suitable elicitation processes (7).Therefore, the author relies on the volumes of scholarly work about eliciting complex decision-making hierarchies or models for risk-based decisions that are available in the MCDM literature (8–11)(12–14). The benefits of multi-criteria decision analysis (MCDA) for controlling subjectivity in QRM risk modeling and decision-making were previously reported (15). The distinction between a model structure for a quantitative risk analysis and that of MCDA is not trivial. The former seeks a testable mathematical formulation of a risk question while the latter models objectives (goals) performance measured over the possible alternatives (e.g., risk projects). Overall, either approach can inform risk-based decision making.The typical steps in MCDM—which includes MCDA and other decision tools—to explain the two major types of expert elicitation for developing risk models that inform RBDM are given in 

Table I. Typical application of structured expert judgement (SEJ) tools in typical risk-based and multi-criteria decision making (MCDM).

Defining and structuring of a complex risk-based decision begins by eliciting a clear definition of the risk in question and the principal factors contributing to the risk (

). The hierarchical view of a complex multi-objective or multi-criteria decision problem is sometimes referred to as an “objectives hierarchy” acknowledging that the purpose of a decision is to achieve one or more specific objectives (12,16,17). In contrast, the purpose in most classical expert elicitations is to have expert knowledge stand in for data, risk parameters, and the uncertainty in the parameters. The elicited data distributions and parameters are then used to calculate risk. Although the theoretical underpinnings differ, a decision objective to “reduce risk to the patient” can lead to similar results as the predictive endpoint of a quantitative, expert-built risk model. Most importantly, both processes inform decision-making about the need to control risk.

 shows a small section of an elicited objectives hierarchy model that might occur in prioritizing pharmaceutical manufacturing sites for good manufacturing practice (GMP) surveillance inspection. In this view, the model includes inherent product and process factors, among others, that experts identify as contributing to overall risk. The attributes of these factors are measurable parameters that combine (usually linearly) to estimate a defined risk value or a score, “upwards” in the hierarchy.A relevant example of qualitative expert knowledge elicitation is found in the development of FDA’s original “GMP Site Selection Model” for GMP surveillance inspections. The model development began in 2003 using qualitative expert elicitation of the key parameters that might predict risk based on methods in the MCDM literature (18). The important attributes that experts believe contribute to a “risk score” for the purpose of GMP surveillance inspections were elicited through both focus group and survey methods of approximately 80 pharmaceutical manufacturing and compliance experts. Since 2005, the FDA site-selection model has undergone many iterations, continuous improvement, and peer-review; however, many of the original risk factors and attributes are retained (19).

Figure 1. Hypothetical and partial elicited hierarchy for a good manufacturing practice (GMP) surveillance model. Elicitation of expert knowledge about the factors important in predicting risk can be organized hierarchically as an aid to risk model communication. The hierarchies range from a handful of criteria to a complex hierarchy of (e.g.) 60 criteria and attributes the author elicited from experts in adverse drug experience reporting. (Figure courtesy of the author)

Expert elicitation for structured expert judgments. A formal quantitative elicitation of expert judgments of probabilities, such as described in Cooke’s classical method for SEJ (20), is the second concept within expert elicitation as defined herein. Rigorous SEJ is used in high-impact RBDM where eliciting continuous risk estimates and the cumulative distribution—a presentation of uncertainty in the risk estimate—may be crucial. The steps for eliciting experts for SEJ are shown in 

 illustrates using theoretical probability densities—the notion that the expert is elicited to obtain a justifiable estimate of likely value and uncertainty for a risk parameter to use in making risk estimates. 

 illustrates the simplest form of eliciting subjective probabilities, which is a common starting point for novices in the practice. This method is sometimes used to initialize parameters for monte carlo risk models. But SEJ is much more complicated, as outlined in 

. The process steps given in Table II mask the very complicated process of eliciting judgments using interviews of one expert at a time before aggregating experts’ results for modeling. Unfortunately, the one-on-one is notorious for being time consuming and challenging for the experts and facilitators alike.

Table II. Structured expert judgment. After: Cooke (5), Kurowicka and Cooke (6). 

Cooke’s classical model relies significantly on calibrating experts by asking them to estimate value and probability distributions of approximately 10 quantities that are reasonably knowable in their discipline. The results of estimating probabilities in the so-called “seed questions” exercise is used under proper statistical scoring rules to calculate two scores for each expert. The first is a calibration score that roughly measures accuracy using statistical distance measures from the known value of the parameter. The second is an information score measuring the match of the expert’s probability distribution and the known parameter distribution (21). Placing estimates in probability bins or probabilities on values estimates can be used (

). For example, a sterile manufacturing expert might be asked for the probability of sterility failures, given a defined sterile process, and subsequently to make reasoned judgments about the uncertainty around values. Reasonable estimates are likely to exist in the pharmaceutical engineering and manufacturing domains. The expert’s accuracy and precision can then be used as measure of the experts’ calibration. The mathematics to calculate the expert’s performance can be daunting and is beyond the scope of this discussion.

Figure 2. A key purpose of expert elicitation (EE). EE/structured expert
judgement (SEJ) is sometimes used to obtain estimates of a probability
distribution (here shown as probability density functions (PDFs) for
convenience of presentation). The EE/SEJ process also generates
information about a variable’s expected uncertainty according to the expert. (Figure courtesy of the author)


Aggregating expert judgments. Finally, the use of multiple experts in an SEJ or in the qualitative hierarchy or structure building is generally preferrable to relying on a single expert. The methods for quantitatively aggregating expert judgments have been topic of much research in risk analysis, operations research and management, and statistics. In general, the many proposed aggregation methods fall within only two general classes: mathematical and behavioral (

). Both classes may include one-on-one elicitations of experts prior to aggregation of the results. But often, such as in failure mode and effects analysis (FMEA) exercises, groupwise estimates might be “socialized” and replace the aggregation of individual risk and uncertainty estimates.

Figure 3. The simplest notion of eliciting a probability distribution. The expert is asked for the low, highest, and most likely (probable) estimates of the risk event or parameters needed for a risk assessment. Given an assumption about the shape of the distribution (e.g., normal, triangular, …), a crude distribution of parameters can be inferred from the elicited values. (Figure courtesy of the author)

Although the math and statistics for SEJ and aggregating EE results is beyond the scope of this report, SEJ has been scientifically validated on many occasions. Key to its validation is the availability of results from tens of thousands of expert elicitations (6,21) and dozens of risk assessments from diverse areas such as nuclear reactor safety, global climate change, and microbial contamination in the food chain (4,5). To date, the author is unaware of a formal SEJ for GMP manufacturing risks for the reasons discussed in the following. Nevertheless, similar but more direct elicitations along the lines of the Sheffield Elicitation Framework (SHELF) (22,23) value and uncertainty judgments have been used by the author and colleagues.

Figure 4. Elicitation of Probabilities by two methods. One approach is to have the expert assign value(s) of the parameters to percentile bins of an assumed distribution of the parameter. The alternative method is to assign percentiles to binned values of the parameters. (Figure courtesy of the author)

Expert elicitation—as defined by the author—is always necessary for QRM. Even the design of apparently straightforward risk tools, such as risk matrices and FMEAs, begins by convening experts in the domain and risk assessment/management possibly with project managers, process engineers, scientists, and managers. Expert elicitation of the key elements of simple models as the basis of, for example, new FMEAs, implementation plans, scheduling periodic reviews, etc., can benefit greatly from structured thinking that is a characteristic of MCDM (11,24), such as value-focused thinking (12–14,25).In this author’s experience, QRM teams who have taken the time and care for structured thinking with qualitative expert elicitation produce sustainable risk tools as opposed to the teams who were more casual about the elicitation process.

Figure 5. General approaches to aggregating expert judgements. The two major processes for aggregating experts' judgments are mathematical and behavioral. The former uses statistical, Bayesian, or AI methods to aggregate independently elicited judgments from the experts. Behavioral methods are “socialized” processes where adjustments to individual’s judgments can occur in real time. Adapted from the author’s work (16). (Figure courtesy of the author)

 Most of the significant decisions risk managers make involve a complex landscape of known and unknown risks, competing risk management resource needs, and large gaps in the information and data for risk assessments, and there is sometimes a compelling argument for more rigorous QRM methods that might benefit from expert elicitations as defined in the SHELF and Cooke classical method procedures. Fortunately, the need for a completely rigorous SEJ continues to be somewhat rare. Pharmaceutical industry risk events or public health crises that could have benefited from more formal risk modeling include the international heparin contamination event (26), glass particulates in injectable drugs, and the ongoing manufacturing and consumer supply chain issues.

Principles for expert elicitation add value. Training of QRM team and risk managers discussed in the 2015 review of QRM was mentioned at the outset of this paper as an important way to control subjectivity. In addition to the topics mentioned previously, the principles for subjectivity control delineated in the MCDM literature and textbooks can extend the training of risk team members beyond a basic “heuristics and biases” level. This information is also valuable for teaching best practices in structured thinking for intermediate QRM risk assessments and risk control evaluations. Moreover, having teams ready to solve complex QRM using the structured thinking behind expert elicitation contributes significantly to the overall risk maturity in an organization.

Cooke’s principles for structured expert judgments were developed around his quantitative methodology; however, the principles are broadly applicable to group RBDM. These principles for the SEJ process discussed in Cooke (4) are:

Scrutability/Accountability, “all data, including experts’ names and assessments, and all processing tools should be open to peer review and results must be reproducible by competent reviewers.”

Empirical Control, “quantitative expert assessments should be subject to empirical quality controls.”

Neutrality, “the method for combining and evaluating expert opinion should encourage experts to state their true opinions and must not bias results.”

Fairness, “experts should not be prejudged, prior to processing the results of their assessments.”

Clearly, Cooke’s principles extend easily to good science practice and might be applied throughout QRM in RBDM activities that call for eliciting expert knowledge as input.

Limitations to expert elicitation in practice

Important limitations to using formal expert elicitation in practice include resources, adequate expertise among staff, lack of supporting information, rejection of the methodology by experts, and misuse or misunderstanding the utility of formal expert elicitation. The state-of-the-art implementations of expert elicitation among many disciplines are those engaging many experts, facilitators, analysts, and project managers. Given the rarity of major QRM risks problems, it is unlikely that companies and governments will routinely assemble or maintain these resource-intensive teams and processes. In fact, regulatory oversight by legislative bodies is often the impetus for agencies to engage in resource-intensive risk assessments for risk policy decision-making.

Anecdotal reports from QRM practitioners often mention the lack of adequate expertise in QRM with respect to more arcane RBDM tools and frameworks. Having adequate expertise in expert elicitation on site for RBDM is likely a widespread limitation. Perhaps the publicly accessible frameworks and primary literature here and elsewhere will help motivate QRM practitioners to expand their knowledge base in the area. Many of the popular QRM tools (e.g., risk matrices, FMEAs, corrective and preventive actions [CAPAs]) benefit greatly by having identified and well-controlled sources of subjectivity based on training and principles for expert elicitation.

The processes outlined in expert elicitation and MCDM are subject to misuse and misunderstanding. For example, it is easy to be dazzled by current analytical and computational power supporting the practice and then to apply sophisticated analyses to every risk question in QRM. This is a fraught approach for both its overuse of limited resources and for its potential risk miscommunication. Sophisticated quantitative results from SEJs, Monte Carlo analyses, or even overly complex risk dashboards can unintentionally communicate more knowledge about a risk problem than can be supported by data and information and expert probability judgments. Simply stated, such endeavors violate the ICH Q9 and Q9(R1) principle, “[t]he level of effort, formality and documentation of the quality risk management
process should be commensurate with the level of risk”(2).

Finally, risk and decision-making facilitators often recall working with highly credentialed and well-trained experts who adamantly reject the notion of eliciting probabilities and, particularly, their personal subjective probabilities. To these experts, the structure of the elicitation processes is not necessarily the problem. Rather, the expert feels that the use of subjectivism runs counter to basic science training emphasizing objectivity. Moreover, experts can feel as though their personal expertise and judgment is in question. With the most resistant experts, this requires discussion and possibly even managerial intervention stressing the importance of the overall risk management goal.

One of the enduring challenges in QRM is the paucity of QRM-specific procedures or regulatory guidance about specific risk tools. The problem is greatest for the most complex risk problems requiring a formal risk analysis. Procedures are particularly lacking for quality risks to the patient having greater potential severity of harm as these problems require more formal risk management. For example, writing a single “QRM cookbook” for a risk issue, such as the international heparin crisis of 2008 (26), involving many dozens of experts, would be a major task. Nevertheless, elements of qualitative expert elicitation were evident as risk assessment and risk controls moved forward.

QRM facilitators and other interested parties will find abundant supporting literature for both the (qualitative) structural knowledge elicitation and the quantitative SEJ. In the first case, helpful works in the field of MCDM include those by Keeney (12,14,25), Belton, and others (11,24,27–29), in addition to the engineering approaches of Haimes (17,30), Ayyub (31), and Saaty (32). The quantitative SEJ has been discussed above as works attributable to Cooke (4,33,34) and with similar frameworks available from Hemming et al. (35), Gosling (22), van der Sluijs (36), and Burgman (37). Importantly, some of these authors make software available for the quantitative aggregation of experts’ subject probabilities. Finally, there are a few sources for expert elicitation, particularly more qualitative aspects, from regulatory authorities (38–41)

In the author’s experience, the choice of a framework for expert elicitation and MCDM depends on the composition of the decision-making or risk management group. Some groups gravitate to more prescriptive, stepwise RBDM methods while others are comfortable in an open brainstorming format. In the author’s experience and as shown in the rich literature on the subject, many opportunities exist for building hybrid frameworks. Thus, it is likely that the relatively new practice of QRM will continue to see one-off frameworks for RBDM until best practices evolve and are committed to guidelines. This predicament is not a negative; rather, it both enables creativity in solving problems and follows the ICH Q9 principle of using “the right tool for the job!”

The frameworks for expert elicitation of knowledge and subjective probabilities contribute significantly to controlling subjectivity in QRM. Whether following Cooke’s classical method for SEJ or using MCDM theory to guide elicitation knowledge about the hierarchical structure of a risk problems, QRM practitioners will find the support of a rich knowledge base from the peer-reviewed literature. The literature from either of these practice disciplines provides thorough guidance on controlling intrusions of subjectivity in RBDM.

There is no simple “binary decision” (e.g., ICH Q9(R1) § 5.1) to know when either qualitative expert elicitation of knowledge or quantitative structured expert elicitation are necessary. For example, there might be significant uncertainties in judging the base frequencies of identified risks that are of low impact. Thus, the significant uncertainties beg for structured judgments; however, the likely low impacts might argue against the resource-intensive structured expert judgement. In this case, extra precision is not warranted.

Claycamp, G. Expert Judgments in Quality Risk Management: Where Quality Risk Management Can “Go Wrong.” In: Waldron, K.; O’Donnell, K., editors. 

Quality Risk Management in the GMP Environment–Ten Years Since the Finalisation of ICH Q9 Anniversary Special Edition

Q9(R1) Quality Risk Management. Guidance for Industry

French, S.; Hanea, A. M.; Bedford, T.; Nane, G. F. Introduction and Overview of Structured Expert Judgement, 2021 In: Hanea, A.M.; Nane, G.F.; Bedford, T.; French, S. (eds) Expert Judgement in Risk and Decision Analysis. 

International Series in Operations Research &

Experts in Uncertainty: Opinion and Subjective Probability in Science

Uncertainty Analysis with High Dimensional Dependence Modelling

. John Wiley & Sons, Ltd, 2006; pp.1–284.

Cooke, R. M.; Goossens, L. L. H. J. TU Delft Expert Judgment Data Base. 

Hanea, A. M.; Hemming, V.; Nane, G. F. Uncertainty Quantification with Experts: Present Status and Research Needs. 

Franco, L. A.; Montibeller, G. Problem Structuring for Multicriteria Decision Analysis Interventions. In: 

Wiley Encyclopedia of Operations Research and Management Science

; Cochran, J.J., Ed.; John Wiley & Sons, Inc., 2010; pp. 1–14.

Nutt, D.J.; King, L.A.; Phillips, L.D. Drug Harms in the UK: A Multicriteria Decision Analysis. 

Sussex, J.; Rollet, P.; Garau, M.; Schmitt, C.; Kent, A.; Hutchings, A. A Pilot Study of Multicriteria Decision Analysis for Valuing Orphan Medicines. 

Multiple Criteria Decision Analysis, An Integrated Approach

. Boston: Kluware Academic Publishers, 2002; pp. 1–372.

Keeney, R. L. Value-Focused Thinking. Cambridge University Press, 1992.

Keeney, R. L. Making Better Decision Makers. 

Keeney, R. L. Value-focused Brainstorming. 

Claycamp, G. Controlling Subjectivity in Risk Decision-making: The Benefits of Multi-criteria Decision Analysis. InAudience with International Regulators in the Manufacture of Medicines: Quality Risk Management (QRM) and Knowledge Management (KM); Greene, A.; O’Donnell, K.; Calnan N., Eds.; DIT Academic Press, 2018; pp. 15–30.

Bond, S. D.; Carlson, K. A.; Keeney, R. L. Generating Objectives: Can Decision Makers Articulate What They Want? 

Haimes, Y. Y.; Kaplan, S.; Lambert, J. H. Risk Filtering, Ranking, and Management Framework using Hierarchical Holographic Modeling. 

Tran, N. L.; Hasselbalch, B.; Morgan, K.; Claycamp, G. Elicitation of Expert Knowledge about Risks Associated with Pharmaceutical Manufacturing Processes. 

FDA Office of Pharmaceutical Quality. MAPP 5041.1 Understanding CDER’s Risk-Based Site Selection Model (Silver Spring, Md., September 2018).

Colson, A. R.; Cooke, R. M. Expert Elicitation: Using the Classical Model to Validate Expert’s Judgments. 

Hanea, A. M.; Nane, G. F. An In-Depth Perspective on the Classical Model. In 

International Series in Operations Research and Management Science

Gosling, J. P. SHELF: The Sheffield Elicitation Framework. In 

Elicitation, The Science and Art of Structuring Judgement;

 Dias, L. C.; Morton, A.; Quigley, J., Eds.; Springer, 2018; pp. 61–93.

SHELF:the Sheffield Elicitation Framework

” (version 3.0), School of Mathematics and Statistics, University of Sheffield. 2016 (accessed Jan. 5, 2023).

Marttunen, M.; Lienert, J.; Belton, V. Structuring Problems for Multi-Criteria Decision Analysis in Practice: A Literature Review of Method Combinations. 

Keeney, R. L. Foundations for Group Decision Analysis. 

Response to Heparin Contamination Helped Protect Public Health; Controls That Were Needed for Working with External Entities Were Recently Added

. Washington, DC; October 2020 (accessed June 9, 2023).

Decision Analysis and Behavioral Research

Otway, H.; von Winterfeldt, D. Expert Judgment in Risk Analysis and Management: Process, Context, and Pitfalls. 

von Winterfeldt, D.; Fasolo, B. Structuring Decision Problems: A Case Study and Reflections for Practitioners. 

Haimes, Y. Y.; Kaplan, S.; Lambert, J. H. Risk Filtering, Ranking, and Management Framework Using Hierarchical Holographic Modeling. 

Elicitation of Expert Opinions for Uncertainty and Risks

Saaty, T. L.; Peniwati, K. Group Decision Making: Drawing Out and Reconciling Differences. 2013th; 2008th ed.; RWS Publications, 2013.

Cooke, R. M. The Aggregation of Expert Judgment: Do Good Things Come to Those Who Weight? 

, 35 (1), 12–15. DOI: 10.1111/risa.12353

der Fels-Klerx, H. J.; Cooke, R. M.; Nauta, M. N.; Goossens, L. H.; Havelaar, A .H. A Structured Expert Should this Judgment Study for a Model of Campylobacter Transmission During Broiler-chicken Processing. 

Hemming, V.; Burgman, M. A.; Hanea, A. M.; McBride, M. F. A Practical Guide to Structured Expert Elicitation Using the IDEA Protocol. 

van der Sluijs, J. P.; Craye, M.; Funtowicz, S.; et al. Combining Quantitative and Qualitative Measures of Uncertainty in Model-Based Environmental Assessment: The NUSAP System. 

, 25 (2), 481–492. DOI: 10.1111/j.1539-6924.2005.00604.x

Trusting Judgements: How to Get the Best Out of Experts

EFSA. Guidance on Expert Knowledge Elicitation in Food and Feed Safety Risk Assessment. 

MacGillivray, B.H. Characterising Bias in Regulatory Risk and Decision Analysis: An Analysis of Heuristics Applied in Health Technology Appraisal, Chemicals Regulation, and Climate Change Governance. 

Practice EG on QU in. Quantifying Uncertainties in Practice. In 

Good Practice Guidance and Uncertainty Management in National Greenhouse Gas Inventories

; Odingo, R., Ed.; Intergovernmental Panel on Climate Change, 2001; pp. 6.1–6.34.

More attention should be given to how expert opinions and judgments are elicited for reducing uncertainty in quality risk management and risk-based decision making. 

Elicitation of Expert Knowledge and Probabilities for Controlling Subjectivity in Risk-Based Decision Making

Mike Hennessy Jr. is the Chairman and CEO of 

High Angle Shot of a Working Desk of an Successful Person in Office with Cityscape Window View. | Image credit: © Gorodenkoff - stock.adobe.com

As of mid-October 2023, there were 9342 clinical trial studies targeting COVID-19 (1). This is alongside 12,086 patents that relate to COVID-19 (2). Messenger RNA (mRNA) clinical trials, however, stand at only 483, with 112 related patents. mRNA was the talk of the industry during October 2023, as Katalin Karikó and Drew Weissman were awarded the Nobel Prize in Physiology or Medicine for their work in mRNA vaccines against COVID-19.
In an interview with Pharmaceutical Technology®, Laks Pernenkil, Principal and Practice Leader, US Life Sciences, Deloitte, pointed to “an emphasis coming out of the pandemic on supply resilience … with all the ups and downs around the supply of raw materials, there has been significant focus on ensuring patients are not affected by supply shortages, part of the anxiety has manifested around inflation [rising costs] of raw materials.” Pernenkil also drew attention to increased digitalization as improving process design, supply performance, and eventually, profits (3).
At the other end of the spectrum, discovering new types of drugs and therapies, the top five pharmaceutical companies’ investments in R&D in 2022 were Roche with $14.78 Billion, J&J $14.14 billion, Pfizer $12.38 billion, Merck $11.84, and BMS $10.02 billion, ranging between 15% to 30% of income (4). Such investment speaks of self-confidence and perhaps aligns with increasing trust in the newest modalities, including mRNA approaches.


NIH. https://clinicaltrials.gov. Clinical trials.gov (accessed Oct. 12, 2023).

World Health Organization, WHO’s Covid-19 Response Timeline (accessed Oct. 12, 2023).

Playter, G. The Era of Digital Data in Bio/Pharma Manufacturing. 

Statista. Leading 50 Global Pharmaceutical Companies by Prescription Sales and R&D Spending in 2022. 

When referring to this article, please cite it as Hennessy, M. Capitalizing on Success. 

Industry investments indicate increasing trust in the newest modalities, including mRNA approaches. 

Capitalizing on Success

Siegfried Schmitt, PhD, is Vice President Technical at Parexel International, Siegfried.Schmitt@parexel.com.

Q: According to our standard operating procedure (SOP), we perform supplier audits during the selection process and as part of ongoing oversight. Although we try to perform as many of these audits ourselves, we do need to rely on third parties to manage the volume of good manufacturing practice (GMP) audits. Is there an international standard for third-party GMP audits?

A: Interestingly, there isn’t such a standard. There are a large number of articles and presentations freely available on the subject, but these may give you conflicting advice, leaving you perhaps confused. This lack of standardized approach has been realized, and a working group (VDI EE 6306, 3rd Party GMP-Audits) under the auspices of The Association of German Engineers (VDI) was formally established in 2023, with the aim of establishing precisely such a standard. That work has already progressed well, and the document is currently under review by regulators and other interested parties. Its publication in German and English is just a matter of time.

Meanwhile, you could verify that your SOP covers all the key elements relating to audits performed by your third parties, namely:

Contractual agreement: each audit should be covered by a legal contract and a quality agreement (these can be combined in one document). The scope of the audit and the deliverables should be unambiguously described. Very likely, confidential disclosure agreements between all parties will be required.

Auditor qualifications: the competency of the auditor or auditors are decisive for reaching the goals of the audit. The competency assessment should address the technical/scientific knowledge and the social competence (e.g., cultural awareness) of the auditors. Additional requirements apply to audit team leaders. Anyone involved in the audit should certify that they have no conflict of interest.

Types of audits: third parties could be used for on-site, remote, or hybrid audits. Each type should be addressed in the SOP.

Audit preparation: as audits are typically of a very short duration, planning them is of utmost importance. The aims and the scopes of the audits should be fully aligned with the audit approach, the available resources, and the time available for the audit. The agenda should be agreed between all involved parties.

Conducting the audit: most audits follow a standard sequence of opening meeting, site tour, and documentation review, followed by a close-out meeting. Provisions should be made if it becomes necessary to deviate from the agenda for whatever reason.

Audit follow-up: as soon as possible, the third party should provide an update on the audit outcome, whether certain information is still outstanding or if anything else is pending.

Preparing audit reports: Invariably, an audit report is required, and it should follow the style of the contract giver’s documents to achieve consistency.

Audit close-out: in all cases (i.e., whether there were observations during the audit that require follow-up or not), there should be formal close-out of the audit. It has to be clear up to what point in this process the third party has to be involved.

The actual standard will of course provide much more detail on each of these aspects, and it will hopefully provide you with exactly what you need.

Siegfried Schmitt is vice president, Technical, at Parexel.


Vol. 47, Number 11
November 2023
Pages: 42

When referring Schmitt, S. Using a Third Party to Perform Audits. 

There are key elements that should be covered when dealing with third-party auditors, according to Siegfried Schmitt, vice president, Technical, at Parexel. 

Using a Third Party to Perform Audits

Emerson’s Fisher FIELDVUE DVC7K Digital Valve Controller

Emerson’s Fisher FIELDVUE DVC7K Digital Valve Controller is a new digital valve controller featuring Emerson’s Advice at the Device technology. Digital valve controllers are accessories for control and on-off valves are used to provide local analysis of valve data and digital communications with host systems, as well as other features. These features improve uptime by alerting personnel to developing problems, reducing maintenance costs by providing information required for proactive rather than reactive maintenance, and cutting troubleshooting time in the event of an issue by providing recommended corrective actions.

The DVC7K ‘s Advice at the Device technology contains embedded computing and real-time analytics that convert raw data into information with Bluetooth capability. This allows maintenance personnel to receive data via their phone, tablet, or computer wirelessly without having to be in a control room at the plant location. Information can be accessed wirelessly from one or more digital valve controllers from up to 30 ft, and the local user interface displays detailed information via LEDs.

Fisher FIELDVUE DVC7K combines patented technology with real-time analytics to provide real-time awareness of valve health. 

Digital Valve Controller

Charles Ross & Son Company's PD-40-gallon PowerMix Mixer

ROSS Mixers are high-performance mixers that are designed for durability and efficiency. The company’s original equipment manufacturer (OEM) team has reconditioned ROSS Mixers to restore them to “like new” working condition. These reconditioned mixers are intended to enhance existing equipment or expand production and are designed to perform at the same standard as brand-new equipment while offering cost savings and expedited shipping. Purchasers of the reconditioned machines receive the same standard warranty as a new ROSS Mixer: one year from the date of shipment.

Reconditioned equipment may utilize existing components to minimize waste, without compromising quality and value. A recent example is the pictured ROSS PD-40-gallon PowerMix, recovered through the ROSS Buy Back Program wherein owners sell mixers no longer in use back to ROSS. The PowerMix is a patented hybrid planetary mixer featuring a planetary stirrer blade and a High Speed Disperser, designed for heavy-duty mixing of high viscosity formulations up to 2 million cP. Reconditioning entails a thorough inspection of each part of the mixer, all of which are tested, cleaned, repaired, replaced, repolished, or modified as needed.

ROSS’ reconditioned mixers are restored to “like new” working condition by the ROSS OEM team and are designed to perform at the same standard as brand-new equipment while offering cost savings. 

ROSS Reconditioned Mixers

Encapsulating active ingredients within microspheres has proven to be an effective approach to drug delivery. Not only does the coating of the API provide protection against degradation, but it is also possible for developers to control the release of the active ingredient over a set amount of time.

A spin-off company from the University of Twente in The Netherlands, IamFluidics, is pioneering a new technology called in-air microfluidics, which is precise and scalable, as well as being more sustainable than traditional microencapsulation approaches. At CPHI Barcelona 2023, 

 spoke with Dr. Tom Kamperman, co-founder and chief technology officer of IamFluidics, to learn more.

Pharmaceutical Technology Europe® spoke with Dr. Tom Kamperman from IamFluidics, a Dutch high-tech company that develops customized microencapsulation solutions, at the inaugural Start-Up Market on the show floor at CPHI Barcelona 2023.

Featured Content

Behind the Headlines Episode 27

Ask the Expert

Drug Digest September 2025

Enhancing Bioanalysis and Drug Development with Advanced Tools and Technologies

Top Trends Reshaping Pharma R&D