The modern clinical laboratory is an extremely demanding field that requires to produce faster results without compromising on the quality. Laboratory systems are complex involving multiple procedural steps and many technical personnel with variable training and handling abilities. More than 70% of clinical decisions rely on test results and recommendations thereby increasing the need for high precision and throughput.1, 2
Quality control (QC) is the core aspect of good laboratories that assures that the quality of results critical for clinical diagnosis and patient care are performed with utmost precision with no window of error. Therefore, the quality management model is an important tool that looks at the entire system and helps the laboratory in achieving efficient laboratory performance. Various practices are adopted by clinical laboratories to maintain quality; these are monitoring Levey Jenning (LJ) graphs, following Westgard rules, and recording coefficient of variation (CV%) for internal quality control purposes. Medical laboratory technicians are often trained to test and rerun controls till it reaches acceptable limits, only after which patient samples can run. To further assure quality, external quality assurance (EQA) programs are established and Z score or Standard deviation index (SDI) is calculated. These tools allow an estimation of precision by minimizing random errors and ensures accuracy by reducing bias respectively. The backbone of a good laboratory thus rests on the quality control program adopted by the laboratories.3
Although, these tools are important, the exact number of errors occurring in the system cannot be quantitated and it is difficult to provide a direct and integrated assessment of the performance of the analytical system.4
In recent years, a quantitative assessment method based on Sigma metrics is often used. The Six sigma model measures the degree to which any process deviates from its goal. Sigma (σ) is the mathematical symbol of standard deviation (SD). It was first introduced by Motorola company as part of a quality improvement project in early 1980s; since then it has been widely used in other fields to help reduce cost, eliminate errors, and detect variability in the system.5
The six sigma model advocates five steps in contrast to the traditional total quality management (TQM) model. The five steps are define, measure, analyze, improve and control (DMAIC). In contrast to the TQM model, an extra step of 'Control' guards against errors returning to the total process and this is a crucial step. In sigma metrics, errors identified are quantified as percentage errors or DPM (defects per million). 1 sigma (σ) represents 6,90,000 errors/million reports, 2 sigma represents 3,08,000 errors/million reports, 3 sigma represents 66,800 errors/million reports, 4 sigma represents 6,210 errors/million reports, 5 sigma corresponds to 230 errors/million reports and 6 sigma represents 3.4 errors/million reports.6, 7 Therefore any process with greater than six sigma, indicates very low variability and defect rate. Based on the sigma obtained the process can be divided into the following categories:8
>6: world class performance
A Six Sigma classification in the clinical laboratory can result in fewer controls and fewer rates of false rejections for methods with a sigma metric of 5 or better. The higher the number of methods with a sigma metric of 5 or better, the lower the costs for reagents, supplies, and control material required to monitor the performance of the methods. The first study utilizing sigma metrics in the clinical lab was published by Nevalainen et al., in the year 2000 and since then many similar studies have been done throughout the world.7 The studies utilizing Sigma metrics to gauge laboratory performance is of limited number in the Middle East region and hence this study was undertaken.
The aim of this study is to evaluate the errors in analytical phase of quality control by sigma metric and compare analyzer performance with other similar studies in the Middle East.
Materials and Methods
This is a retrospective study for a period of 12 months from January to December 2020 conducted at Aster Medical Centre clinical laboratory, Doha, Qatar using biochemistry analyzer ERBA-XL 200. Internal quality data of 16 analytes including, albumin (ALB), alanine amino transferase (SGPT), aspartate amino transferase (SGOT), alkaline phosphatase (ALKP), bilirubin total (BIL T), bilirubin direct (BIL D), calcium (CAL), cholesterol (CHOL), creatinine (CREAT), gamma glutamyl transferase (GGT), glucose (GLUC), high density lipoprotein (HDL), triglyceride (TG), total protein (PROT), uric acid (UA) and urea was extracted.
The control materials used were Erba Norm and Erba Path (normal and high levels respectively) supplied by the manufacturer (Erba Lachema s.r.o, Brno, CZ). These controls are routinely assessed once a day before processing patient samples. The same lot of quality control materials was used for this study and instrument calibration done according to manufacturer guidelines.
Coefficient of variation (CV%)
For each level of control, the monthly CV% was calculated from the mean and standard deviation (SD) of internal quality control data which is automated by the analyzer and then average was taken. The CV% was calculated as follows:
CV% = SD/ mean x 100
The laboratory uses the Randox international quality assessment scheme (RIQAS) where, one sample of varying concentration is analyzed every month and the subsequent bias% was used. The Bias% was calculated as follows:
Bias% = mean of all laboratories using same instrument and method-lab mean/mean of all laboratories using same instrument and method×100.
Total allowable error (TEa)
The total allowable error was taken from CLIA '88 (Clinical and Laboratory Improvement Act) guidelines.
Table 1 and Table 2 shows average CV% for all parameters in Level I and Level II while Table 3 and 4 shows bias and sigma calculation. In this study, the CV% was found to be the lowest for albumin (Level 1- 1.35%) and SGOT (Level II-1.2%) and the highest for urea (Level 1 and 2). Furthermore, the minimum average bias was observed for albumin (1.2%) while maximum bias was observed for bilirubin total (10%).
Out of the 16 analytes studied at two levels of concentration, it was found that five analytes in level 1 and eight analytes in level 2 had greater than 6 sigma performance indicating world class quality. Moreover, many of the parameters showed >3 sigma performance which is considered acceptable performance. The problem analytes having <3 sigma was identified as Urea (both levels) and GGT (level 1). (See Figure 1 and Figure 2).
Sigma metrics is an improvement method which concentrates on reducing variability in process outputs. Furthermore, it is an excellent tool to predict and compare assay and instrument quality and is a pointer to the tests that require minimal quality control rules to monitor the performance of the method. Based on the sigma values obtained the QC can be tailored as follows: 8, 7
>6σ (Excellent performance): IQC can be run once per day with one level (alternating levels) and follow 13.5s rule.
4σ–6σ (Suited to purpose): IQC can be run once per day with two levels per day and follow single IQC rule.
3σ–4σ (Poor performers): IQC can be run twice per day with two levels of IQC per day and use multi-rule system.
<3σ (Problematic): IQC should be run three times per day with three levels; consider testing in duplicate and use maximum IQC rules.
As is evident from this classification, analytes that display >6 sigma require very minimal QC rules to monitor the method performance. If the sigma is <3 or shows a wide variation between two levels, a close monitoring of the method with use of multiple QC rules or even a change in method is mandated.9 In our study, Urea (both levels) and GGT (level 1) was identified as problem analytes with <3 sigma metrics. Our study correlates well with similar studies from Asia and Middle East. In these studies, urea was also found to be <3 sigma while ALKP was found to have >6 sigma10, 11, 12, 13, 14, 15, 16, 17, 18 (Table 5)
A recent study in India by Vijatha et al., shows excellent correlation with the present study.18 Our study also showed good correlation with a similar study by Nanda et al where >6 sigma was observed for uric acid, total bilirubin, SGOT, SGPT, TG and ALP. In contrast, albumin and cholesterol had <3 sigma. In our study, these parameters showed better performance.19 Similar studies in India by Adiga US et al., using Erba XL-640 showed >6 sigma for HDL, ALKP, UA, TG and Alb, whereas <3 sigma was obtained for SGPT and SGOT. This contrasts with our study as these parameters showed better sigma metric in our study.20 The discrepancies observed in sigma metrics in various studies can be attributed to various factors such as difference in methods, reagents, IQC material, bias calculations and varying EQAS providers. Creatinine showed a wide variation between L1 and L2 with L1 performing at 4 sigma, whereas L2 showed >6 sigma. The parameters which demonstrates wide variation in the sigma values for both the levels of QC should be evaluated with caution. The method needs to be be re -evaluated and the Westgard multi-rules have to be strictly followed. Also the number of QC runs need to be increased so as to abolish this discrepancy.
Another study analyzed the reason for the lower sigma in analytes using another quality parameter known as Quality goal index ratio (QGI). A QGI value less than 0.8 (QGI < 0.8) indicates that the precision of the corresponding analyte needs to be improved, whereas a value greater than 1.2 (QGI > 1.2) indicates that the accuracy of the analyte needs to be improved. A QGI value between 0.8 and 1.2 (0.8 ≤ QGI ≤ 1.2) indicates that the accuracy and precision of the analyte need to be simultaneously improved.9
El Sharkawy et al., in 2018, proposed a harmonized protocol for sigma calculation and highlighted the importance of selecting TEa goals.11 Sigma metric changes according to the chosen TEa goal and each lab should have a standardized criterion for selecting the same, as under-or overestimation of sigma metric will affect patient results. The world is yet to reach a consensus regarding the most ideal quality goal to be used, and herein lies the biggest challenge of using sigma metrics.21 Nevertheless, it cannot be denied, that sigma metric analysis is a revolutionary quality assessment tool. The old ‘one size fits all’ model of quality management is now recognized as insufficient to meet the time and cost-saving demands of the modern lab. Newer QC graphic tools have also been developed like the Method Decision Chart, OPSpecs chart, and QC Frequency Nomogram which can further enhance the quality in the lab.5
Few limitations of this study include the use of manufacturer supplied controls for calculation of precision instead of third-party controls. This was due to financial constraints. Another limitation is the lack of a pilot study using the new proposed IQC frequency demonstrating process improvement in comparison to the existing one.
This is one of the first study in Qatar to gauge analytical clinical laboratory performance using six sigma metrics. Before the sigma study, our laboratory was using two level IQC and Westgard multirule blanket approach for all analytes. Based on the study findings, we conclude that the six sigma metrics can be used to customize IQC frequency for effective and improved quality control.