Laboratory medicine has undergone a spectacular evolution in the last decades
and has become today of crucial importance for supporting diagnostic
and therapeutic decisions. The increase of the volume of laboratory analyses
has not gone without an emerging risk of measurement errors that may
have far-reaching consequences, even on the patient’s life. External Quality
Assessment (EQA), already established since several decades in various
countries and often running on an international level, aim at going further
than the "internal quality control" procedures of every laboratory and at improving
laboratory quality by inter-laboratory comparisons. An EQA round
generally consists of sending aliquots of the same sample to various laboratories
for assaying selected tests. After finishing the assays, results are
reported back to the EQA organizer. Subsequently these results are subject
to a statistical analysis, which is performed globally, for all the participants,
or for each analytical technique separately. Finally, a report is sent to every
participant that informs about the acceptability of the individual results,
with respect to predefined limits, and with respect to the group of peers.
This thesis, structured in five chapters, focuses on the External Quality Control
of clinical laboratories by a critical analysis of existing methods and by
creating new approaches that permit to improve the current procedures.
The first chapter of this work emphasizes the evolution of the role of the
clinical laboratory and EQA in the quality improvement. After the report
’To Err is Human: Building a Safer Health System’, numerous scientists
became interested in investigating the frequency, source and impact of laboratory
errors. The Total Testing Process (TTP) became recognized as the
best framework to investigate laboratory errors. The three different phases
of the TTP - respectively, the pre-analytical, analytical and post-analytical
phases - are described in detail and the nature and frequency of errors in
each phase explained. For each phase, possible improvements are described
and the role of EQA is suggested. Today, EQA principally focuses on the
assessment and improvement of the analytical phase. Proposals are made to
improve the role of EQA for assessing and improving pre- and post-analytical
error as well, by using specific sample material and by automating the reporting
of data and laboratory reports to the EQA participants. The principle
of the comparison of results of a laboratory with those obtained by the other
laboratories is traditionally based on the calculation of "z-scores". An indepth
study comparing different techniques has been made, shedding new
light on the shortcomings and strong points of the different approaches. We
concluded that robust techniques may exhibit weak performance for smaller
sample size, while techniques that eliminate outliers before calculating zscores
should be recommended.
The second Chapter discusses the role of EQA as a tool to assess harmonization
between methods. The role of EQA is described, together with the
pitfalls and current shortcomings for assessing harmonization. A major problem
in assessing standardization between methods is the possible presence of
matrix effects in control samples, in which a method-specific bias may appear.
Several explanations for matrix effects are mentioned and statistical
techniques are described that assist EQA organizers to split up the data in
homogeneous peer groups using multivariate statistics. The chapter also reviews
several techniques to be used in method comparison studies, and the
preference for the use of orthogonal regression is expressed. In addition, an
example is given of a method-comparison study for Estradiol and Progesterone,
with a novel technique of assessing standardization between various
methods, in the presence of matrix effects for a small number of samples.
The study also reveals that standardization between various methods is not
attained, and that the striving for standardization with standards of higher
order may not be satisfactory.
Chapter 3 introduces different evaluation techniques that combine information
from different samples or parameters: Variance and bias index scores,
Mean ranking scores, counts of z- and u-scores, and a long-term analytical
Coefficient of Variation. Also, a new and original method is introduced that
uses 3 steps to identify outliers in a first step, to find laboratories with exceeding
variability in a second, and to identify laboratories with high bias in
a third step. Each of the techniques are evaluated and discussed by means
of a data set in which accidental outliers, high variability and high bias were
induced. In addition, the comparison between the different evaluation methods
reveals that distinguishing between variability and bias is a tedious task,
and that some long-term analysis methods lack robustness against outliers.
Also, it is proven that evaluation techniques summarizing results of different
parameters may hide useful information. In addition, the 3-step method is
proposed as a method for discerning between errors produced in the pre- or
post-analytical phase, and errors that arise from the analytical phase.
Chapter 4 applies the 3-step method to data obtained from the Belgian EQA.
Data sets from alcohol, flow cytometry, lithium and semen analysis surveys
are examined. The method is extended for applicability to heteroscedastic,
i.e. unequal residual variability, regression models and demonstrates that
it is able to be used in a wide range of surveys. For each of the surveys
under consideration, a follow-up is made of the occurrence of accidental mistakes,
and the evolution of within-laboratory variability and bias for selected
methods. It highlights several conclusions that show a striking similarity for
various EQA surveys: an improvement of laboratory performance has been
attained over time. The major improvement was a reduction of accidental
mistakes. The analytical performance of selected methods, however, did not
show an improvement over time.
In Chapter 5, some graphical representations of EQA data are explored and
a graphical representation of the 3-step method is described. The histogram,
normal quantile plot and box plot are described in detail and suggested for
providing a quick visual overview of EQA data. Other graphical representations
that respond to specific questions are given and discussed as well, like
Shewhart charts, Cusum charts and graphical representations to combine
variability and bias in one graph. In addition, the 3-step method is graphically
explored by means of three distinct graphs. The chapter finishes by
suggesting the use of interactive graphs for improving feedback from the EQA
organizers to the EQA participants by means of Scalable Vector Graphics.
The latter is illustrated with web-accessible examples of long-term evaluation
of z-scores and the results of the 3-step method for the data obtained in the
Belgian EQA for alcohol determination in blood.
In brief, this work describes in a critical and constructive way current statistical
methods used in EQA and proposes novel statistical and graphical
techniques to help alleviating the future needs of External Quality Assessment