A simplified, visual version of Westgard's model sheds light on QC issues.
By Jeffrey A. Freed, MD
"Make everything as simple as possible, but not simpler." - Attributed to Albert Einstein
Laboratories that routinely apply the same multirule to control all of their quantitative tests, regardless of individual test performance characteristics, may have no idea whether any given test is well or poorly controlled, or indeed whether the test is performing acceptably at all. (A well-controlled test is one in which a critical change in test performance can be detected reliably by statistical quality control methods.) Westgard and Klee1 offer a rigorous model for assessing the quality of quality control and for planning how to perform it. In contrast, this article presents a simplified version of Westgard's model which is visual and intuitive, and which clarifies the factors that affect the quality of your quality control.
In Control
Please recall that, for one tail of the normal distribution, the area beyond 1.28SD is 10%, and the area beyond 1.65SD is 5%.
Turning the Levey-Jennings chart from an in-control test on its side, we get the approximate normal distribution shown in Fig. 1, with mean and standard deviation calculated from the data points. The mean exceeds (in this example) the manufacturer-assigned target value for the control level by the bias (Target+Bias=Mean). Let's assume a control rule of 13s. We want the probability of false rejection (Pfr), which is the proportion of the area under the curve outside the interval from Target-3SD to Target+3SD, to be not more than 5%. Pfr=5% when Target +3SD=Mean+1.65SD.Consequently, Bias=1.35SD, as shown. What happens if an in-control test has an absolute value of bias [abs(Bias)] greater than 1.35SD? In that case, Pfr>5% and the control rule is not usable. Later, we will answer the question of what can be done.
Out of Control
We will consider what happens when there is a sudden change in test performance such that it is borderline unacceptable for clinical use ("critical error").The condition of critical error exists when the total allowable error (TEa) is exceeded 5% of the time.This may happen through an increase in systematic error (bias), through an increase in random error (standard deviation), or through a combination of the two. Figure 2 shows this condition of critical systematic error in which New Mean+1.65SD=Target+TEa, leaving 5% of the area under the curve outside of the interval from Target-TEa to Target+TEa. If we wish the probability of error detection (Ped) to be 90% (i.e., 90% of the area under the curve is outside of the region from Target-3SD to Target+3SD), then Target+3SD+1.28SD+1.65SD=Target+TEa. It follows that TEa/5.93=SD. If SD is greater than TEa/5.93, then Ped will be less than 90%.
Considering Figs. 1 and 2 together, it is apparent that changing the control rule will alter both Pfr and Ped in the same direction. If we adopt 12.5s instead of 13s, Ped increases (desirable) and Pfr increases (undesirable). If we adopt 13.5s, Ped decreases (undesirable) and Pfr decreases (desirable). Thus, we might wish to remedy a high Pfr (recall that Pfr>5% when Bias>1.35SD) by adopting 13.5s. This is appropriate if the SD is sufficiently small compared to the TEa. Also, use of a multirule tends to reduce Pfr (somewhat) without reducing Ped. For those seeking accurate guidance, a rigorous analysis such as Westgard's is recommended.
Critical random error is illustrated in Fig. 3, showing why random error is difficult to detect.
Assessing the Quality of Your Quality Control
Considering the limitations of the above approach, the following is for illustrative purposes only. Take the QC data of a test. If abs(Bias)<1.35SD, then Pfr<5% (acceptable). If abs(Bias)>1.35SD you might consider changing the control rule (e.g., 13.5s instead of 13s), but be aware that the Ped will be decreased. This approach will work best when SD<<TEa/5.93.
If SD<TEa/5.93, then Ped>90% (desirable). If SD>TEa/5.93, the error detection rate may be suboptimal and therefore increased preventive maintenance and/or more frequent control runs may be appropriate.However, any increase in control runs entails an increase in false rejection.
If abs(Bias)+1.65SD>TEa, the test is not suitable for clinical use (see Figs. 2 and 3).
Limitations of the Approach
In practice, many laboratories apply a multirule rather than the single 13s rule considered in this article.Furthermore, at least two control levels are run. Finally, there are components of imprecision that are not normally distributed. Because of these differences between actual practice and the simplified model presented above, a rigorous analysis is recommended for actually assessing and planning your quality control. The above analysis, however, makes clear the factors that affect the efficacy of quality control.
Dr. Freed is medical director of Laboratory, Grays Harbor Community Hospital, CellNetix Pathology and Laboratories.
Reference
1. Westgard JO and Klee GG in Tietz Textbook of Clinical Chemistry and Molecular Diagnostics (Elsevier, Inc., St. Louis, 4th ed, 2006), Chapter 19.
©Copyright 2013 Merion Matters. All rights reserved.