Robert Shaw is principal scientist statistics, Oral Product Development, Operations, AstraZeneca Pharmaceuticals, Macclesfield, Cheshire, United Kingdom, SK102NA, Robert.Shaw@astrazeneca.com. 

Robert Shaw

	Submitted: August 21, 2019; Accepted: November 21, 2019

Process variability must be assessed over time to ensure that pharmaceutical product quality is maintained. While the application of prospective statistical process control has been widely published, much less emphasis has been given to retrospective statistical analysis and associated methods and approaches. The authors present a simple-to-use Microsoft Excel-based statistical tool that uses cumulative sum techniques to aid retrospective understanding of data trends. Practical recommendations and experience of applying the tool from a pharmaceutical manufacturing context are also provided, including the teamwork needed to fully exploit these approaches through combining input from multiple disciplines. 

In pharmaceutical development and manufacturing, scientists are often involved in assessing process variability over time. For example, having developed a new analytical method, the analyst needs to review data to ensure that the method continues to perform adequately over time; or scientists responsible for manufacturing processes need to demonstrate that product quality is maintained and that there is no evidence of systematic or special cause variability in processes. It can be misleading to apply traditional prospective techniques when analyzing data retrospectively (see below), and a useful alternative is cumulative sum techniques.

	The tool presented in this paper incorporates such cumulative sum techniques and furthermore uses the Autocorners algorithm to automatically identify statistical changes in the average over time, which can aid problem-solving and process improvement.

	The techniques and approaches described in this paper apply to any process where data are gathered in time-ordered sequence and one wishes to understand the process to ensure statistical control and variability within acceptable limits. For many processes, Shewhart control charts work well for prospective monitoring. The implementation of effective Shewhart charts involves a set-up phase prior to the subsequent run phase. The set-up phase includes capturing data over a pre-defined period of time exhibiting statistical control (absent of special cause variability) and calculating control limits based on the mean and standard deviation.  These summary statistics are then applied to calculate control limits to be used subsequently in the run phase. Often, analysts want to review data retrospectively (e.g., in manufacturing, analyze data from previous batches of product to assess process robustness or to identify a root cause of a problem). In these cases, the traditional Shewhart chart does not apply in the conventional way. In particular, care needs to be taken in constructing a Shewhart chart from historical data where the limits applied are based on the same data being analyzed. In this case, the limits might be derived from data exhibiting special cause variability, which would inflate the limits calculated. An effective alternative to Shewhart charts in the situation of retrospective data analysis is CuSum charts and the Autocorners algorithm described in this paper. 

	Cumulative sum (CuSum) charts and Autocorners can be applied both in prospective and retrospective situations.  Many texts (1–7) describe the improved sensitivity in identifying small shifts in the average or variance when using CuSum techniques compared with Shewhart methods. 

distinguishes the benefits of Shewhart charts for prospective monitoring versus CuSums/Autocorners analysis.

			Table I: Comparison of Shewhart and CuSum charts.			

					      special cause variation at

					      reduce the frequency in 

					•   Primary goal: Visualising, 

					      frequent step changes (in

	In constructing a CuSum chart, the data are first suitably ordered (e.g., by manufacturing date) and the following steps applied.

	Suppose there is a set of results in sequence, denoted by x

.  The differences of each result from a target or reference value, T, are calculated, so the ith difference = x

-T.  The cumulative sum of these differences can be calculated as follows:

 = cumulative sum of the first two differences in series = (x

	These cumulative sums, abbreviated to CuSums, are plotted in time sequence to produce a CuSum chart. In many situations, the target or reference value T can be set to equal the average of the raw data, and in this case, the last point in the CuSum chart equals zero.

	The effect of the CuSum is to produce a smoother picture of changes in data over time-a shift up or down in the raw data will appear as a change in slope in the CuSum.

	Having plotted the CuSum data as a chart, it may be useful to assess whether an observed change in slope demonstrates a genuine signal (special or assignable cause) or whether it represents typical noise in the process (common cause variation). Several approaches to making this assessment are described by Taylor 

 (8). The basis for finding significant changepoints programmed into the Manhattan tool is the automatic search algorithm, Autocorners, described by Woodward and Goldsmith (6).

	The tool has been applied successfully in a variety of different pharmaceutical applications, including:

		Assessing animal model performance by monitoring control groups over time

		Aiding understanding of variability in critical quality attributes and processing parameters in manufacturing processes.

	The Autocorners algorithm identifies where statistical changes in slope occur in the CuSum chart. These change points are highlighted in a plot of the original raw data by splitting the data into stages and plotting the average per stage.

	In the retrospective analysis of process data, Woodward and Goldsmith describe several approaches that can be applied to decide whether a change in the slope of a CuSum chart is real or due to noise. These include two manual approaches (the “span” method and the decision interval method) and one automatic search by computer. It is this latter approach, called “Autocorners”, that has been programmed into the Manhattan tool.  The data in 

are used to illustrate the approach described as follows:

		The first step is to inspect the data for unusual observations that may be individual outliers. These are identified by testing whether an observation differs from both of its neighbors by more than 4.12 times the mean range (4.12 is the appropriate 0.1% significance point-see Weatherill 

 (1) Table 5.2).  If any are found, then they are replaced in subsequent calculations by the average of the two adjacent values in the series. It is important to note that these unusual values are not excluded from the final interpretation and are represented in the output plot; however, outliers should not overly distort the CuSum chart or skew the overall trends and associated calculations.

		The overall mean of the data series is calculated, and CuSums are formed using the overall mean as reference value.

		A forward search is carried out through the data series starting at the first point as follows: a chord is joined from each CuSum back to zero, and the maximum difference between the chord and corresponding CuSums is found (see 

):
		a.         The position of this largest difference is used to split the intervening original observations into two groups, and a t-test is carried out between the mean values of the two groups.
		b.         When the value of t is not significant, the program moves to the next CuSum and repeats the process. When the value of t becomes significant, the program, subject to further tests, declares this position to represent a significant changepoint (called a “corner”).

		A forward search is carried out through the data series from each corner as follows: a chord is joined from each CuSum back to the last corner, and the maximum difference between the chord and corresponding CuSums is found (see 

). The same criteria a. and b. are applied as in 3 above.

		When this forward search is completed, the program reverses the order and does a backward search.

		Finally, the two lists of change points are amalgamated, and a list of significant change points made. 

	In the above steps, numerous tests of differences between adjacent segments are made.  The overall significance level (α) has been explored by varying between 1, 5, and 10%, so that the algorithm would find increasing numbers of corners with higher α.  In the authors’ experience, applying a 1% significance level works well for most of the data sets, and this is used as the default setting in the tool.  The overall adjustment is applied for multiple comparisons described in BS 5703: Part 2.  This standard has been superseded by BS ISO 7870-4 (3); however, this more recent version does not include the multiple comparisons details; two alternative references (9,10) have been provided for readers without access to the original standard.

	As explained in the references (4,9,10), for retrospective analysis using CuSum charts, the individual tests should have significance level α√m  /2N, where m=length of segment, N=total number of observations.  The t-test with an ad hoc adjustment has significance level given by Lewis 

 (9), α/(2*√m).   Where multiple tests are carried out in a series of total length N, the adjustment described by Osanaive 

 (10) also applies: (m/N)α.  This leads to a combined adjustment of α (m/N)/(2*√m) = α√m  /2N. 

	Having identified points of change or corners in the raw data, then a convenient representation of the data is provided in Figure 3.  The raw data are shown as green triangles and connected with a blue line.  (In situations where an unusual value has been identified, as in step 1 described previously, then such a point is connected to its neighbors with a green rather than blue line, such as shown in 

, the black line shows the CuSum plot.  The red line shows the means across the regions between the break points, and shifts in this line indicate where the significant change points occur. This plot is called the Manhattan plot as it can look like the skyline of Manhattan.  The changepoints in the Manhattan plot coincide with where the slope changes in the CuSum plot. 

	The Autocorners functionality described in this paper has been written in to the Excel tool using VBA.

Raw data are entered into the spreadsheet. There is some additional functionality that is available including the ability to refresh the spreadsheet by deleting existing data. The tool is flexible to handling different numbers of response variables. 

	When the analysis is run, for each variable, two new worksheets are produced:

		One sheet consists of a plot of the raw data with the Manhattan plot overlaid, together with the CuSum chart (if selected).

		The other sheet provides statistical summary statistics and details of the different stages.

	The choice of whether to include the CuSum chart or not depends on the nature and amount of data entered. In the following example, the CuSum is particularly helpful to visualize alongside the Manhattan plot to help confirm the key change points in the data. When viewing the Manhattan plot, some pragmatic interpretation is required (e.g., it is crucial to take account of the scale of the y-axis and to avoid over-interpreting changes which are not of practical importance). If changes are made to the data entry sheet and the analysis re-run, then the new sheets produced will over-write the previous ones.

Example of application-capsule dissolution investigation

. The dissolution test for this product involves taking a random sample of six capsules from a batch and submitting them to the dissolution test. The dissolution test gives the percent dissolved at 90, 300, and 480 minutes, from which the rate of change between 300 and 480 is calculated.  The average dissolution rate for the six capsules across multiple time-ordered batches has been recorded and analyzed using this tool. See 

 for an example of the graphical output with the following features:

		There is one value deemed to be an unusual value–batch 84. This one value is connected to the ones either side by a green line.  The algorithm automatically identified this point (see point 1 in Autocorners Algorithm Description) and replaced it in calculations by the average of the two adjacent values.

		The CuSum plot shows an important change in slope at batch 315. Prior to this point, the slope is predominantly negative, while subsequently the slope is consistently positive.

		The change in slope of the CuSum corresponds to an upward shift in the raw data and an upward step-change in the Manhattan plot.

	There are some other changes that can be seen in 

between batches 40 and 63, and around batch 275; however, these are more subtle and not sustained over a long time.

Articles

The authors present a simple-to-use Microsoft Excel-based statistical tool that uses cumulative sum techniques to aid retrospective understanding of data trends.