International Journal of Clinical Biochemistry and Research

Print ISSN: 2394-6369

Online ISSN: 2394-6377

CODEN : IJCBK6

International Journal of Clinical Biochemistry and Research (IJCBR) open access, peer-reviewed quarterly journal publishing since 2014 and is published under auspices of the Innovative Education and Scientific Research Foundation (IESRF), aim to uplift researchers, scholars, academicians, and professionals in all academic and scientific disciplines. IESRF is dedicated to the transfer of technology and research by publishing scientific journals, research content, providing professional’s more...


  • Article highlights
  • Article tables
  • Article images

Article statistics

Viewed: 63

PDF Downloaded: 68


Koshy and Raza: Sigma metrics in quality control- An innovative tool


Introduction

The modern clinical laboratory is an extremely demanding field that requires to produce faster results without compromising on the quality. Laboratory systems are complex involving multiple procedural steps and many technical personnel with variable training and handling abilities. More than 70% of clinical decisions rely on test results and recommendations thereby increasing the need for high precision and throughput.1, 2

Quality control (QC) is the core aspect of good laboratories that assures that the quality of results critical for clinical diagnosis and patient care are performed with utmost precision with no window of error. Therefore, the quality management model is an important tool that looks at the entire system and helps the laboratory in achieving efficient laboratory performance. Various practices are adopted by clinical laboratories to maintain quality; these are monitoring Levey Jenning (LJ) graphs, following Westgard rules, and recording coefficient of variation (CV%) for internal quality control purposes. Medical laboratory technicians are often trained to test and rerun controls till it reaches acceptable limits, only after which patient samples can run. To further assure quality, external quality assurance (EQA) programs are established and Z score or Standard deviation index (SDI) is calculated. These tools allow an estimation of precision by minimizing random errors and ensures accuracy by reducing bias respectively. The backbone of a good laboratory thus rests on the quality control program adopted by the laboratories.3

Although, these tools are important, the exact number of errors occurring in the system cannot be quantitated and it is difficult to provide a direct and integrated assessment of the performance of the analytical system.4

In recent years, a quantitative assessment method based on Sigma metrics is often used. The Six sigma model measures the degree to which any process deviates from its goal. Sigma (σ) is the mathematical symbol of standard deviation (SD). It was first introduced by Motorola company as part of a quality improvement project in early 1980s; since then it has been widely used in other fields to help reduce cost, eliminate errors, and detect variability in the system.5

The six sigma model advocates five steps in contrast to the traditional total quality management (TQM) model. The five steps are define, measure, analyze, improve and control (DMAIC). In contrast to the TQM model, an extra step of 'Control' guards against errors returning to the total process and this is a crucial step. In sigma metrics, errors identified are quantified as percentage errors or DPM (defects per million). 1 sigma (σ) represents 6,90,000 errors/million reports, 2 sigma represents 3,08,000 errors/million reports, 3 sigma represents 66,800 errors/million reports, 4 sigma represents 6,210 errors/million reports, 5 sigma corresponds to 230 errors/million reports and 6 sigma represents 3.4 errors/million reports.6, 7 Therefore any process with greater than six sigma, indicates very low variability and defect rate. Based on the sigma obtained the process can be divided into the following categories:8

>6: world class performance

5-6: Excellent

4-5: Good

3-4: Acceptable

2-3: Poor

<2: Unacceptable

A Six Sigma classification in the clinical laboratory can result in fewer controls and fewer rates of false rejections for methods with a sigma metric of 5 or better. The higher the number of methods with a sigma metric of 5 or better, the lower the costs for reagents, supplies, and control material required to monitor the performance of the methods. The first study utilizing sigma metrics in the clinical lab was published by Nevalainen et al., in the year 2000 and since then many similar studies have been done throughout the world.7 The studies utilizing Sigma metrics to gauge laboratory performance is of limited number in the Middle East region and hence this study was undertaken.

The aim of this study is to evaluate the errors in analytical phase of quality control by sigma metric and compare analyzer performance with other similar studies in the Middle East.

Materials and Methods

This is a retrospective study for a period of 12 months from January to December 2020 conducted at Aster Medical Centre clinical laboratory, Doha, Qatar using biochemistry analyzer ERBA-XL 200. Internal quality data of 16 analytes including, albumin (ALB), alanine amino transferase (SGPT), aspartate amino transferase (SGOT), alkaline phosphatase (ALKP), bilirubin total (BIL T), bilirubin direct (BIL D), calcium (CAL), cholesterol (CHOL), creatinine (CREAT), gamma glutamyl transferase (GGT), glucose (GLUC), high density lipoprotein (HDL), triglyceride (TG), total protein (PROT), uric acid (UA) and urea was extracted.

The control materials used were Erba Norm and Erba Path (normal and high levels respectively) supplied by the manufacturer (Erba Lachema s.r.o, Brno, CZ). These controls are routinely assessed once a day before processing patient samples. The same lot of quality control materials was used for this study and instrument calibration done according to manufacturer guidelines.

Coefficient of variation (CV%)

For each level of control, the monthly CV% was calculated from the mean and standard deviation (SD) of internal quality control data which is automated by the analyzer and then average was taken. The CV% was calculated as follows:

CV% = SD/ mean x 100

Bias%

The laboratory uses the Randox international quality assessment scheme (RIQAS) where, one sample of varying concentration is analyzed every month and the subsequent bias% was used. The Bias% was calculated as follows:

Bias% = mean of all laboratories using same instrument and method-lab mean/mean of all laboratories using same instrument and method×100.

Total allowable error (TEa)

The total allowable error was taken from CLIA '88 (Clinical and Laboratory Improvement Act) guidelines.

Sigma metric

For each analyte, Sigma metric was calculated using the formula:

Sigma Σ(σ) = (TEa %− Bias%) / CV%

Results

Table 1 and Table 2 shows average CV% for all parameters in Level I and Level II while Table 3 and 4 shows bias and sigma calculation. In this study, the CV% was found to be the lowest for albumin (Level 1- 1.35%) and SGOT (Level II-1.2%) and the highest for urea (Level 1 and 2). Furthermore, the minimum average bias was observed for albumin (1.2%) while maximum bias was observed for bilirubin total (10%).

Figure 1

Graphical representation of Sigma metrics Level 1 QC(Jan- Dec 2020)

https://typeset-prod-media-server.s3.amazonaws.com/article_uploads/36b3378a-0cd2-4eb2-b78b-655221ad391c/image/296af4e4-7850-4abf-b003-571c8e9f8cc0-uimage.png

Figure 2

Graphical representation of Sigma metrics Level 2 QC (Jan- Dec 2020)

https://typeset-prod-media-server.s3.amazonaws.com/article_uploads/36b3378a-0cd2-4eb2-b78b-655221ad391c/image/bf56f24d-3c7f-4333-bd96-8adceb157d21-uimage.png

Table 1

CV% of all 16 analytes during the study period for level 1

Level 1

Jan

Feb

Mar

Apr

May

Jun

July

Aug

Sep

Oct

Nov

Dec

AVG

CV%

ALB

1.34

1.35

1.26

1.57

1.25

1.32

1.0

1.3

1.17

1.56

1.16

1.86

1.35

ALKP

6.32

3.03

2.17

1.42

1.74

1.37

1.74

2.0

3.5

4.33

4.51

3.83

3.00

BIL T

2.19

1.48

2.03

1.28

2.41

2.3

2.74

2.36

2.69

2.46

3.55

5.14

2.55

CAL

2.03

1.99

2.54

2.06

2.31

4.3

1.69

2.26

3.3

2.85

2.12

1.45

2.41

GLUC

2.06

1.55

1.16

1.42

2.26

1.59

1.39

2.5

1.93

2.0

2.77

2.33

1.91

GGT

2.54

2.06

2.18

2.52

2.93

2.33

2.51

2.28

2.24

1.95

2.01

2.12

2.31

PROT

3.91

2.85

2.28

2.45

2.04

1.34

1.39

1.86

2.07

1.71

2.24

2.58

2.23

UREA

2.43

3.4

2.93

3.16

2.21

2.87

2.0

3.11

3.25

3.34

3.8

3.56

3.01

SGPT

2.37

1.51

2

1.74

1.66

1.84

1.79

1.66

2.97

3.58

3.21

5.21

2.46

SGOT

1.73

1.13

2.23

2.17

2.35

1.49

2.1

2.01

2.32

1.18

2.02

2.27

1.92

HDL

1.77

2.39

2.4

2.97

2.91

2.11

1.87

2.31

2.07

2.05

1.63

2.81

2.27

CHOL

1.33

1.51

1.79

1.79

1.13

1.88

1.08

1.47

1.24

1.48

2.74

3.08

1.71

UA

1.15

1.26

1.28

1.78

1.24

2.67

2.37

2.57

3.64

3.87

3.16

3.42

2.37

TG

2.26

2.85

2.8

3.11

2.55

3.31

3.39

2.79

3.84

3.27

2.57

2.99

2.98

CREAT

1.88

2.35

2.64

3.06

4.62

2.69

4.42

2.04

2.17

2.34

2.96

2.95

2.84

BIL D

2.52

2.66

2.7

2.83

1.85

3.58

1.91

4.02

4.13

2.83

4.97

4.71

3.23

Out of the 16 analytes studied at two levels of concentration, it was found that five analytes in level 1 and eight analytes in level 2 had greater than 6 sigma performance indicating world class quality. Moreover, many of the parameters showed >3 sigma performance which is considered acceptable performance. The problem analytes having <3 sigma was identified as Urea (both levels) and GGT (level 1). (See Figure 1 and Figure 2).

Table 2

CV% of all 16 analytes during the study period for level 2

Level 2

Jan

Feb

Mar

Apr

May

Jun

July

Aug

Sep

Oct

Nov

Dec

Avg CV%

ALB

1.12

0.96

1.33

1.75

1.1

1.51

1.55

0.87

1.23

1.46

0.78

1.45

1.26

ALKP

5.28

2.75

1.66

2

1.03

1.64

1.66

1.42

3.93

3.89

3.22

2.57

2.59

BIL T

1.37

1.88

2.53

2.01

1.76

2.21

2.25

1.95

1.64

2.06

2.88

3.07

2.13

CAL

1.85

1.96

1.87

1.95

2.53

3.25

1.82

1.8

2.39

2.81

2.39

1.39

2.17

GLUC

1.67

1.08

1.65

1.53

1.26

0.92

1.35

1.25

1.52

2.02

2.18

1.63

1.51

GGT

1.83

1.84

1.26

1.6

1.99

1.53

1.27

1.19

1.1

2.02

2.11

1.88

1.64

PROT

2.37

2.28

1.44

2.09

1.69

1.32

1.33

0.99

1.31

1.51

2.66

2.1

1.76

UREA

2.74

3.78

2.55

2.89

3.09

2.5

0.94

2.93

4.14

3.41

3.8

2.83

2.97

SGPT

1.8

1.63

2.41

1.82

1.42

1.64

1.33

0.86

2.06

2.05

2.12

2.02

1.76

SGOT

1.33

0.77

1.54

0.97

0.72

1.38

1.02

0.99

1.44

2.05

1.09

1.24

1.21

HDL

3.2

2.32

2.84

1.55

1.68

2.21

1.65

1.87

1.42

2.49

1.36

2.74

2.11

CHOL

1.21

1.18

1.5

3.04

1.59

1.43

1.8

1.3

1.4

2.17

1.72

2.06

1.70

UA

0.96

1.1

1.34

1.2

0.89

2.75

0.88

3.12

3.36

2.79

2.9

4.06

2.11

TG

1.99

2.5

1.87

2

2.05

2.23

3.18

1.8

2.19

2.71

1.58

1.99

2.17

CREAT

1.17

1.44

1.21

1.55

1.93

1.85

3.16

1.12

1.82

2.05

0.99

2.06

1.70

BIL D

1.46

2.07

4.25

3.78

3.45

3.92

1.64

2.61

3.14

2.3

5.7

3.22

3.13

Table 3

Bias % and sigma metrics for level 1 QC

Analyte

Avg CV%

AVG BIAS%

TEa

AVG Sigma1

ALB

1.35

1.3

10

6.4

ALKP

3.00

6.6

30

7.8

BIL T

2.55

10.0

20

3.9

CAL

2.41

2.1

11

3.7

GLUC

1.91

2.2

10

4.1

GGT

2.31

3.5

10

2.8

PROT

2.23

1.8

10

3.7

UREA

3.01

3.6

09

1.8

SGPT

2.46

6.0

20

5.7

SGOT

1.92

2.6

20

9.1

HDL C

2.27

8.3

30

9.5

CHOL

1.71

3.1

10

4.0

UA

2.37

4.2

17

5.4

TG

2.98

4.8

25

6.8

CREATN

2.84

2.3

15

4.5

BIL D

3.23

8.7

20

3.5

Table 4

Bias % and sigma metrics for level 2 QC

Analyte

Avg CV%

Avg BIAS%

TEa

Avg Sigma 2

ALB

1.26

1.3

10

6.9

ALKP

2.59

6.6

30

9.0

BIL T

2.13

10.0

20

4.7

CAL

2.17

2.1

11

4.1

GLUC

1.51

2.2

10

5.2

GGT

1.64

3.5

10

4.0

PROT

1.76

1.8

10

4.7

UREA

2.97

3.6

09

1.8

SGPT

1.76

6.0

20

7.9

SGOT

1.21

2.6

20

14.4

HDL C

2.11

8.3

30

10.3

CHOL

1.70

3.1

10

4.1

UA

2.11

4.2

17

6.1

TG

2.17

4.8

25

9.3

CREATN

1.70

2.3

15

7.5

BIL D

3.13

8.7

20

3.6

Discussion

Sigma metrics is an improvement method which concentrates on reducing variability in process outputs. Furthermore, it is an excellent tool to predict and compare assay and instrument quality and is a pointer to the tests that require minimal quality control rules to monitor the performance of the method. Based on the sigma values obtained the QC can be tailored as follows: 8, 7

  1. >6σ (Excellent performance): IQC can be run once per day with one level (alternating levels) and follow 13.5s rule.

  2. 4σ–6σ (Suited to purpose): IQC can be run once per day with two levels per day and follow single IQC rule.

  3. 3σ–4σ (Poor performers): IQC can be run twice per day with two levels of IQC per day and use multi-rule system.

  4. <3σ (Problematic): IQC should be run three times per day with three levels; consider testing in duplicate and use maximum IQC rules.

As is evident from this classification, analytes that display >6 sigma require very minimal QC rules to monitor the method performance. If the sigma is <3 or shows a wide variation between two levels, a close monitoring of the method with use of multiple QC rules or even a change in method is mandated.9 In our study, Urea (both levels) and GGT (level 1) was identified as problem analytes with <3 sigma metrics. Our study correlates well with similar studies from Asia and Middle East. In these studies, urea was also found to be <3 sigma while ALKP was found to have >6 sigma10, 11, 12, 13, 14, 15, 16, 17, 18 (Table 5)

Table 5

Comparision of present study with similar studies in the Middle East and Asia

Country

Instrument

Year

CV source

Bias source

TEa source

Sigma

>6

Sigma

<3

Ref

Iraq

RXL Max

(Siemens)

2018

Randox

Randox

Ricos 2014

ALT, AST

Urea, Creat

10

RXL Max

(Siemens)

2017

Randox

Riqas

Ricos 2014

TG

Gluc

11

Egypt

Olympus AU400

Cobas 8000

Cobas c501

Olympus AU480

2018

Manufacturer IQC

Biorad

Variable

Nil

ALT, AST,

BIT, BID,

Chol, Prot, UA, Urea, GGT

12

UAE

Architect Ci 8200 Abbott

2015

Biorad

CAP

CLIA

ALP, Amy, BIT, Creat, Gluc

AST, Prot

13

Turkey

COBAS

8000

2019

Roche

-

CLIA

ALP(L2), BID

TG(L2)

Alb, Cal, Crea, Gluc, Prot, Urea

14

Abbott architect

2021

Multichem S

Calculated

CLIA

ALP, AST, ALT, GGT, UA(L2)

Alb, BID, urea, TG

15

Beckman Coulter

2018

Manufacturer IQC

Calculated

CLIA

HDL

Urea, Gluc

16

Abbott- C8000,

Roche-Cobas 8000,

Siemens-ADVIA2400

TG

ALT, BIT

Pakistan

Architect c8000

2017

Randox

RIQAS

CLIA

Creat

TG

Alb, BIT,

Prot

17

India

Cobas Integra 400 plus

2017

Manufacturer IQC

Biorad

CLIA

ALP, ALT, AST, BIT, HDL, UA

Urea

18

A recent study in India by Vijatha et al., shows excellent correlation with the present study.18 Our study also showed good correlation with a similar study by Nanda et al where >6 sigma was observed for uric acid, total bilirubin, SGOT, SGPT, TG and ALP. In contrast, albumin and cholesterol had <3 sigma. In our study, these parameters showed better performance.19 Similar studies in India by Adiga US et al., using Erba XL-640 showed >6 sigma for HDL, ALKP, UA, TG and Alb, whereas <3 sigma was obtained for SGPT and SGOT. This contrasts with our study as these parameters showed better sigma metric in our study.20 The discrepancies observed in sigma metrics in various studies can be attributed to various factors such as difference in methods, reagents, IQC material, bias calculations and varying EQAS providers. Creatinine showed a wide variation between L1 and L2 with L1 performing at 4 sigma, whereas L2 showed >6 sigma. The parameters which demonstrates wide variation in the sigma values for both the levels of QC should be evaluated with caution. The method needs to be be re -evaluated and the Westgard multi-rules have to be strictly followed. Also the number of QC runs need to be increased so as to abolish this discrepancy.

Another study analyzed the reason for the lower sigma in analytes using another quality parameter known as Quality goal index ratio (QGI). A QGI value less than 0.8 (QGI < 0.8) indicates that the precision of the corresponding analyte needs to be improved, whereas a value greater than 1.2 (QGI > 1.2) indicates that the accuracy of the analyte needs to be improved. A QGI value between 0.8 and 1.2 (0.8 ≤ QGI ≤ 1.2) indicates that the accuracy and precision of the analyte need to be simultaneously improved.9

El Sharkawy et al., in 2018, proposed a harmonized protocol for sigma calculation and highlighted the importance of selecting TEa goals.11 Sigma metric changes according to the chosen TEa goal and each lab should have a standardized criterion for selecting the same, as under-or overestimation of sigma metric will affect patient results. The world is yet to reach a consensus regarding the most ideal quality goal to be used, and herein lies the biggest challenge of using sigma metrics.21 Nevertheless, it cannot be denied, that sigma metric analysis is a revolutionary quality assessment tool. The old ‘one size fits all’ model of quality management is now recognized as insufficient to meet the time and cost-saving demands of the modern lab. Newer QC graphic tools have also been developed like the Method Decision Chart, OPSpecs chart, and QC Frequency Nomogram which can further enhance the quality in the lab.5

Few limitations of this study include the use of manufacturer supplied controls for calculation of precision instead of third-party controls. This was due to financial constraints. Another limitation is the lack of a pilot study using the new proposed IQC frequency demonstrating process improvement in comparison to the existing one.

Conclusion

This is one of the first study in Qatar to gauge analytical clinical laboratory performance using six sigma metrics. Before the sigma study, our laboratory was using two level IQC and Westgard multirule blanket approach for all analytes. Based on the study findings, we conclude that the six sigma metrics can be used to customize IQC frequency for effective and improved quality control.

Source of Funding

None.

Conflicts of Interest

None.

Author Contributions

JSK conceptualized, performed, interpreted the data and wrote the manuscript. AR contributed to the study design and provided critical revision on the manuscript. All authors read and approved the final manuscript.

Acknowledgments

Special gratitude to Dr Rekha Nirwan for her valuable inputs and Farhan Mohamed for the technical assistance rendered for completion of this study.

References

1 

JO Westgard Internal quality control: planning and implementation strategiesAnn Clin Biochem200340Pt 6593611

2 

GG Klee JO Westgard AC Burtis ER Ashwood DE Brurns Quality ManagementTietz Textbook of Clinical Chemistry and Molecular Diagnostics5th editionPhiladelphiaSaunders2012163204

3 

JO Westgard Six Sigma Quality Design and Control2nd EditionWestgard QC IncMadison, WI2006

4 

D Angmo S Kant Six sigma implementation in healthcare industry: Past, present and futureInt J Eng Res Technol201546107882

5 

S Westgard H Bayat JO Westgard Analytical Sigma metrics. A review of Six Sigma implementation tools for medical laboratoriesBiochem Med (Zagreb)2018282020502

6 

A Coskun TC Inal M Serteser Six Sigma Projects and Personal ExperiencesInTechCroatia2011

7 

D Nevalainen L Berte C Kraft E Leigh L Picaso T Morgan Evaluating laboratory performance on quality indicators with the six sigma scaleArchives of Pathology & Laboratory Medicine20001244516519

8 

H Kinns S Pitkin D Housley DB Freedman Internal quality control: best practiceJ Clin Pathol20136612102732

9 

M Kumar T Mohan Sigma metrics in evaluating the performance of clinical chemistry laboratoryJ Lab Physicians 20181021949

10 

B Mahmood MK Rasheed A Khazal MK Rasheed Assessment of Sigma Metric Results of Serum Parameters of Liver and Kidney Function Tested by Automated Chemistry Analyzer in Medical City HospitalIraqi Postgrad Med J201817330714

11 

AN Kaftan AK Yaseen Z Hasan Assessment of sigma metrics results of serum glucose and lipid profile tested by automated chemistry analyzer in medical city hospitals in IraqInt J Res Med Sci201751146906

12 

R El Sharkawy S Westgard AM Awad AKOI Ahmed EH Iman A Gaballah Comparison between Sigma metrics in four accredited Egyptian medical laboratories in some biochemical tests: an initiative towards sigma calculation harmonizationBiochem Med (Zagreb)2018282020711

13 

MMNH Al-Hassan Assessment of Quality Control in a Clinical Chemistry Laboratory using Six Sigma Metrics2017http://repo.uofg.edu.sd/handle/123456789/2979

14 

G Çakıra T Yazıcı Evaluation of Analytical Performance of Clinical Chemistry Assays Using Six SigmaTürk Klinik Biyokimya Derg20191727984

15 

Basak Oktay Analytical process performance and Six Sigma method: A comparison of two biochemistry analyzersInt J Med Biochem20214297103

16 

Ö Gülbahar The use of six sigma methodology to evaluate the analytical performancesTurk J Biochem201843118

17 

S Iqbal Application of Sigma Metrics Analysis for the Assessmentand Modification of Quality Control Program in the Clinical Chemistry Laboratory of a Tertiary Care HospitalInd J Clin Biochem20173211069

18 

V Thomas PB Desai AT Mithrason Evaluation of clinical biochemistry laboratory performance using sigma metricsInt J Clin Biochem Res2018546047

19 

SK Nanda L Ray Quantitative application of sigma metrics in medical biochemistryJ Clin Diagn Res2013712268991

20 

US Adiga A Preethika K Swathi Sigma metrics in clinical chemistry laboratory-A guide to quality controlAl Ameen J Med Sci2015842817

21 

K Hens M Berth D Armbruster S Westgard Sigma metrics used to assess analytical quality of clinical chemistry assays: importance of the allowable total error (TEa) targetClin Chem Lab Med201452797380



jats-html.xsl

© This is an open access article distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.


Article type

Original Article


Article page

253-259


Authors Details

Juby Sara Koshy*, Afsheen Raza


Article History

Received : 30-10-2021

Accepted : 23-11-2021

Available online : 07-01-2022


Article Metrics


View Article As

 


Downlaod Files

   








Open Abstract (Increase article citation) Wiki in hindi