Rounding Results for Comparison with Specification

Chris BurgessRounding a result to the required number of decimal places is easy, isn’t it? After all, we were all taught at college or university that only two rules are needed.

Rule 1: If the digit after the figure to be rounded is less than 5, then don’t change the rounded figure (i.e., round down).

Rule 2: If the digit after the figure to be rounded is 5 or more, then increase figure to be rounded by 1 (i.e., round up).

Simple really, isn’t it? After all, the United States Pharmacopeia (USP) requires the use of just this method:

When rounding is required, consider only one digit in the decimal place to the right of the last place in the limit expression. If this digit is smaller than 5, it is eliminated and the preceding digit is unchanged. If this digit is equal to or greater than 5, it is eliminated and the preceding digit is increased by 1 (1).

Of course, rounding only takes place after the final calculation has been performed. The USP General Notices make this requirement clear:

Numbers should not be rounded until the final calculations for the reportable value have been completed. Intermediate calculations (e.g., slope for linearity) may be rounded for reporting purposes, but the original (not rounded) value should be used for any additional required calculations (1).

The problem is that simplicity is not always correct all the time. From a statistical point of view, applying rule 2 will bias the data over time because one will always round up particularly if 5 is frequently the figure to be rounded. 4 and below and 6 and above are balanced in rounding but what about 5? It has been known for more than 60 years that applying only rules 1 and 2 will cause biased data. What is even more interesting is that the third rule was well known at that time, at least to statisticians. Sadly, this rule is rarely if ever mentioned in modern textbooks and guidelines.

The purpose of this column is to resurrect this “forgotten” rule because the need for simplicity does not take precedence over scientific correctness. In any event, rule 3 is not hard to understand and is merely buried in older textbooks and standards.

Three rules for roundingThe best explanation of unbiased rounding is in a 1947 textbook written by the statistical research group of Columbia University (2). The senior editor, Churchill Eisenhart from the (then) National Bureau of Standards, was one of the most influential statisticians of his era.

Here are the three rules:Rule 1: If the last digit to be dropped is less than 5, the last digit retained shall be left unchanged.

Rule 2: If the last digit to be dropped is greater than 5, or is 5 followed by digits greater than 0, the last digit retained shall be increased by 1.

Rule 3: If the last digit to be dropped is 5 alone or a 5followed by 0 only, the last digit retained shall be rounded to the nearest even number.

Figure 1: Decision tree for an unbiased rounding process.It is usually easier to see how such a rule works if you draw it in a decision tree as shown in Figure 1. In this figure, we designate the digit or figure to be rounded by X, the digit or figure to be dropped by Y, and the difference between the truncated value and the full figure value by Z. Based upon the values of X, Y, and Z, we can arrive at the correct unbiased rounding decision.

Examples

We can now test our unbiased rounding process with four examples.

1. Round π with 7 significant figures to 2 decimal places.

Rule 1 gives the answer ‘Yes,’ so round down to 3.14.

2. Round energy (E) with 7 significant figures to 2 decimal places.

Rule 1 gives the answer ‘No’ and the rule 2 answer is ‘Yes,’ so round upto 2.72.

3. Round 0.005051 to 2 decimal places.

Rules 1 and 2 both give the answer ‘No,’ therefore, rule 3 is invoked. AsZ > 0, rule 3 answer is ‘No,’ so round up to 0.01.

4. Round 0.005000 to 2 decimal places.

Rules 1 and 2 both give the answer ‘No,’ therefore, rule 3 is invoked. As Z=0, rule 3 answer is ‘Yes’ and the digit before X is even, therefore, round down to 0.00.

The last example may give rise to surprise because by the simple rules it would be deemed incorrect. However, it would be unlikely to find such anexample in analysis unless the method is only capable of giving rise to 5 or 0 for Y.

Rounding using Microsoft Excel

Table I: results from Excel 2010 with four examples.Many laboratories use Microsoft Excel for calculations, so it is interesting to see how it performs with the four examples. Excel has three rounding functions: round, round down, and round up. In each function, it is necessary to specify the number to be rounded and the number of decimal places required. The results from Excel 2010 for the examples to 2 decimal places are shown in Table I. The incorrect rounded values are highlighted in yellow.

The round function works well with the exception of example 4. However, Excel can be easily programmed to perform a unbiased rounding decision tree automatically. Suppose the values to be rounded are put starting in cell C9 and the number of decimal places to be rounded to in cell E9.

Excel's trunc function can be used to extract the values of X, Y and Z; using the formulae:

For X = TRUNC($C9,$E9)*10^$E9-TRUNC($C9,$E9-1)*10^$E9

For Y = TRUNC($C9,$E9+1)*10^($E9+1)-TRUNC($C9,$E9)*10^($E9+1)

For Z = $C9-TRUNC($C9,$E9+1).

These values of X, Y and Z are stored in the relevant rows (9 to 12) of columns G, H, and I. The rounding calculation formula (a nested “if” formula) is now placed in cell J9 for the first example and copied down for rows 10, 11, and 12 for the others:

=IF($H9<=4,ROUNDDOWN($C9,$E9),IF($H9>=5,

IF(AND($H9>=5,$I9>=0),ROUNDUP($C9,$E9),ROUNDDOWN($C9,$E9))))

Table II: Results of the unbiased rounding formula using Excel.The Excel results of the unbiased rounding formula are shown in Table II and agree with the manual evaluation for the four examples.

The mysteries of rounding are exposed here and strict unbiased rounding can be applied.

References

1. USP 35, General Notices 7.20, “Rounding Rules” (US Pharmacopeial Convention, Rockville, MD, 2012).

2. C. Eisenhart, M.W. Hastay, W.A. Wallis, Selected Techniques of Statistical Analysis, (McGraw-Hill, New York, 1947).