Mika Reijonen

Despite the long history of pharmaceutical validation, process validation in pharmaceutical manufacturing continues to be topical. European regulations regarding process validation were renewed in autumn 2001, which again brought the subject under the spotlight. Many people working in pharmaceutical production are now reviewing the state of their compliance practices and posing the question: "How will we benefit from process validation?"	

To estimate the value of process validation, a systematic evaluation of the collected opinions and experiences of it was performed using technology assessment (TA). TA is an evaluation process that aims to protect people/society from the consequences of rapid technological developments and attempts to identify all the possible impacts of a technology, not just the intended ones.1 Today, technology in this context refers not only to the logical products of science, but also to the attitudes, processes, apparatus and consequences associated with it, and in that wider meaning, the principles of TA are well suited to the evaluation of process validation as a tool for pharmaceutical manufacturing.	

There are a number of different methods of TA; this study used the synthesis (the compilation and evaluation of all available knowledge1) method. Initially, a literature search was performed,2 which revealed a lack of European experts' comments. Therefore, during autumn 2001, an experimental survey was conducted amongst European experts in pharmaceutical fields of manufacturing, regulation and academia to unearth their opinions.	

The main objectives of the study were to explore the value of process validation and to ascertain the best tools to perform it. Additionally, the study provided an opportunity to test how the principles and methods of TA could be used in the field of pharmaceutical quality assurance.				

Given that there is no single solution to process validation and that the value of process validation cannot be evaluated by using solid empirical measurement, but rather by informed judgement, 3 a discussion group was found to be an effective method for collecting information. For this reason, the Delphi method was chosen.				

Delphi method. The procedure used in the Delphi method aims at structuring and distilling the mass of information from a selected group of experts by means of a series of questionnaires based on a structured process with controlled feedback.4 Moreover, this method was chosen for the following benefits:	

it enables participants from various countries and different fields (industry, authorities, schools) to take part

it allows anonymous participation; a benefit that was of special value because the survey intended to facilitate discussion between the industry and its authorities, and to obtain comments from different organizational levels

participants can take part asynchronously; that is, one may choose when to participate

participants can choose to contribute to areas in which they are best qualified.

For development in the Delphi survey, see sidebar "Changes in the Delphi survey."

The Internet and e-mail. To accelerate communication, the Internet and e-mail were used. Using the Internet was also beneficial because it offered supporting tools for group communication, such as the potential for online discussion.6 Furthermore, the Internet provides a better and more illustrative means of informing the participants of the survey's key elements.		

Although TA synthesis methods are frequently used to predict future scenarios, they can also be employed to critically examine the state-of-the-art of a given field.1,7 One of the most popular tools of synthesis is the Delphi technique. The aim of most Delphi techniques is the reliable and creative exploration of ideas or the production of suitable information for decision-making.				

The replies to one round of questions are summarized and used to construct the next questionnaire. This reiterative process is continued until consensus or clear disagreement is reached among the participants.	

One disadvantage of the Delphi method is the definition and selection of experts; that is, whom to regard as an expert and how to create a representative group. 				

For this survey, experts were defined as those people working on process validation in:	

Representatives of the pharmaceutical industry were chosen from quality assurance, production and product development positions; representatives of the authorities had to either evaluate pharmaceutical and chemical aspects of the marketing authorization applications or work in inspection; and the representatives of pharmaceutical education had to teach process validation. Further details were unspecified; educational background was disregarded and the objective was to obtain the widest possible representation of different organizational levels. The level of experience of pharmaceutical process validation would have been of interest, but because of the limited number of experts in the field, this issue was not used as an exclusion criterion. 

Given the pharmaceutical industry's sensitivity concerning knowledge sharing and to get both the regulating authorities and the regulated industry involved in the survey, anonymity was regarded as essential.	

The search for participants took 5 months and was the most challenging part of the survey, particularly contacting industry experts because their e-mail addresses could not be located. To find suitable people from the pharmaceutical industry, a snowball method was used.23 Additionally, addresses for quality assurance, product development and production experts were requested from the representatives of appropriate foreign companies in Finland and from the qualified persons of appropriate domestic companies. However, only a few participants from the foreign companies participated because the representatives did not know who the right people were to ask, or the company refused to participate because of time constraints. The only effective way of obtaining willing participants was finally found to be direct telephone contact; this method also worked well for all three pharmaceutical fields.		

The size of the expert group is also important to the outcome of a Delphi study, but it depends on the homogeneity of the expert population and whether the study searches for qualitative or quantitative results.23 Other Delphi surveys have varied in size from 10–15 up to 2000–3000.23,24 For this survey, the number of experts was limited and the study mainly searched for qualitative results. Thus, a group of approximately 30–50 participants was considered apposite.	

An extremely important part of the Delphi method is the questionnaires, particularly the first round questionnaire (Q1). Q1 needs to be easy and clear, to motivate and encourage the respondents; otherwise they may lose interest.				

The design of the questionnaires was carefully discussed by the advisory group before the start. The objective was to form easy-to-use and fast-to-complete questionnaires. It was agreed to keep the number of questions to a minimum and to limit the use of open-ended questions. Not only would these measures avoid making the survey too time consuming, but they would deter those who would use these factors as reasons for abandoning the survey. The use of open-ended questions is, though, often found necessary to eliminate the possible external bias linked to the guiding role of the investigator.7 Instead of open-ended questions, the external bias was eliminated using a mixture of negatively and positively influenced arguments. The questionnaires were also pre-tested with a pilot study among the advisory group and external experts.	

Although the aim was to identify the attitude on the usefulness of pharmaceutical process validation, a further aim was to establish whether there are differences in attitudes between European countries or the different parties.	

Operationalization of the subject was done under five headings:	

How do you feel about process validation?

What are the benefits of process validation?

What are the negative aspects of process validation?

How can we make process validation easier and more effective to get the most from it?

What hinders positive thinking on process validation?

The first three questions were designed to measure the overall attitude towards process validation and the final two were mostly determined the reasons underlying the attitudes, and also attempted to find out if some specific tools of process validation were known among the participants.

The initial round started with a background information questionnaire, which also included one final question estimating the overall opinion on process validation. This questionnaire was to be completed before commencing Q1, and thus, the last question served as a control, measuring the attitude at the beginning of the study. The same question was repeated at the end of the second round questionnaire (Q2).	

Q1 consisted of 31 questions, most of which were multiple-choice. Only three of the questions were open-ended to increase the inclination and quality of personal expression. Q2 included some new or modified questions, but most were repeated in their original form together with a summary of the answers from Q1. 		

Figure 5:  Pharmaceutical participant demography: gender, education and age.	

In Q2, the opportunity to add comments was provided after every question and was encouraged by offering participants the option to answer in their native language. Also in this round, the questions were posed in a slightly different order, with the addition of one new heading - "Cost of validation." In Q1, questions concerning the cost of validation had been spread under other headings.	

The Internet as an environment for the survey 

The WebCT (WebCT, Inc., Lynnefield, Massachusetts, USA) learning environment26 was chosen as the platform for the survey because it was already being used in Helsinki University and offered the required tools. Extranet homepages were constructed and the questionnaires, together with supporting communication tools and information pages, were available from the Internet public page. The participants were e-mailed the passwords required to access these pages.				

Guidance and other information, including definitions of the critical terms, were offered on the homepages and an online forum was available for anonymous discussion. The discussion area was continuously available and an online forum was organized twice during the survey - before and after Q2. The surveys' Internet public homepage can still be viewed.	

Response rate and expert demography. Of the total 73 used e-mail addresses, 36 experts' responses to Q1 were received, 28 of whom continued to Q2. Thus, the response rates were 49% and 38% respectively, for Q1 and Q2.				

Some of the experts' demography can be seen in Figures 1–5, which show a comprehensive variation in their background. There were participants from Finland, Denmark, France, Germany, the UK, Norway, Sweden, Belgium, Iceland and Switzerland; however, the number of participants from each country was not equal.		

Articles

During autumn 2001, a Delphi survey was conducted using the Internet to discover the opinions of European experts from the pharmaceutical fields of industry, regulation and academia regarding pharmaceutical process validation. The Internet Delphi method was found to be an appropriate tool for exploring the opinions and experiences of pharmaceutical manufacturing because it allowed anonymous participation and discussion between geographically dispersed parties. However, to obtain reliable results, many factors were taken into account, as this article examines.