BICTEL/e - ULg
Serveur institutionnel des thèses de doctorat

• outlier searching/recherche d'outliers
• external quality assessment/évaluation externe de qualité
• statistical analysis/analyses statistiques
• robust statistics/statistiques robustes

Laboratory medicine has undergone a spectacular evolution in the last decades

and has become today of crucial importance for supporting diagnostic

and therapeutic decisions. The increase of the volume of laboratory analyses

has not gone without an emerging risk of measurement errors that may

have far-reaching consequences, even on the patient’s life. External Quality

Assessment (EQA), already established since several decades in various

countries and often running on an international level, aim at going further

than the "internal quality control" procedures of every laboratory and at improving

laboratory quality by inter-laboratory comparisons. An EQA round

generally consists of sending aliquots of the same sample to various laboratories

for assaying selected tests. After finishing the assays, results are

reported back to the EQA organizer. Subsequently these results are subject

to a statistical analysis, which is performed globally, for all the participants,

or for each analytical technique separately. Finally, a report is sent to every

participant that informs about the acceptability of the individual results,

with respect to predefined limits, and with respect to the group of peers.

This thesis, structured in five chapters, focuses on the External Quality Control

of clinical laboratories by a critical analysis of existing methods and by

creating new approaches that permit to improve the current procedures.

The first chapter of this work emphasizes the evolution of the role of the

clinical laboratory and EQA in the quality improvement. After the report

’To Err is Human: Building a Safer Health System’, numerous scientists

became interested in investigating the frequency, source and impact of laboratory

errors. The Total Testing Process (TTP) became recognized as the

best framework to investigate laboratory errors. The three different phases

of the TTP - respectively, the pre-analytical, analytical and post-analytical

phases - are described in detail and the nature and frequency of errors in

each phase explained. For each phase, possible improvements are described

and the role of EQA is suggested. Today, EQA principally focuses on the

assessment and improvement of the analytical phase. Proposals are made to

improve the role of EQA for assessing and improving pre- and post-analytical

error as well, by using specific sample material and by automating the reporting

of data and laboratory reports to the EQA participants. The principle

of the comparison of results of a laboratory with those obtained by the other

laboratories is traditionally based on the calculation of "z-scores". An indepth

study comparing different techniques has been made, shedding new

light on the shortcomings and strong points of the different approaches. We

concluded that robust techniques may exhibit weak performance for smaller

sample size, while techniques that eliminate outliers before calculating zscores

should be recommended.

The second Chapter discusses the role of EQA as a tool to assess harmonization

between methods. The role of EQA is described, together with the

pitfalls and current shortcomings for assessing harmonization. A major problem

in assessing standardization between methods is the possible presence of

matrix effects in control samples, in which a method-specific bias may appear.

Several explanations for matrix effects are mentioned and statistical

techniques are described that assist EQA organizers to split up the data in

homogeneous peer groups using multivariate statistics. The chapter also reviews

several techniques to be used in method comparison studies, and the

preference for the use of orthogonal regression is expressed. In addition, an

example is given of a method-comparison study for Estradiol and Progesterone,

with a novel technique of assessing standardization between various

methods, in the presence of matrix effects for a small number of samples.

The study also reveals that standardization between various methods is not

attained, and that the striving for standardization with standards of higher

order may not be satisfactory.

Chapter 3 introduces different evaluation techniques that combine information

from different samples or parameters: Variance and bias index scores,

Mean ranking scores, counts of z- and u-scores, and a long-term analytical

Coefficient of Variation. Also, a new and original method is introduced that

uses 3 steps to identify outliers in a first step, to find laboratories with exceeding

variability in a second, and to identify laboratories with high bias in

a third step. Each of the techniques are evaluated and discussed by means

of a data set in which accidental outliers, high variability and high bias were

induced. In addition, the comparison between the different evaluation methods

reveals that distinguishing between variability and bias is a tedious task,

and that some long-term analysis methods lack robustness against outliers.

Also, it is proven that evaluation techniques summarizing results of different

parameters may hide useful information. In addition, the 3-step method is

proposed as a method for discerning between errors produced in the pre- or

post-analytical phase, and errors that arise from the analytical phase.

Chapter 4 applies the 3-step method to data obtained from the Belgian EQA.

Data sets from alcohol, flow cytometry, lithium and semen analysis surveys

are examined. The method is extended for applicability to heteroscedastic,

i.e. unequal residual variability, regression models and demonstrates that

it is able to be used in a wide range of surveys. For each of the surveys

under consideration, a follow-up is made of the occurrence of accidental mistakes,

and the evolution of within-laboratory variability and bias for selected

methods. It highlights several conclusions that show a striking similarity for

various EQA surveys: an improvement of laboratory performance has been

attained over time. The major improvement was a reduction of accidental

mistakes. The analytical performance of selected methods, however, did not

show an improvement over time.

In Chapter 5, some graphical representations of EQA data are explored and

a graphical representation of the 3-step method is described. The histogram,

normal quantile plot and box plot are described in detail and suggested for

providing a quick visual overview of EQA data. Other graphical representations

that respond to specific questions are given and discussed as well, like

Shewhart charts, Cusum charts and graphical representations to combine

variability and bias in one graph. In addition, the 3-step method is graphically

explored by means of three distinct graphs. The chapter finishes by

suggesting the use of interactive graphs for improving feedback from the EQA

organizers to the EQA participants by means of Scalable Vector Graphics.

The latter is illustrated with web-accessible examples of long-term evaluation

of z-scores and the results of the 3-step method for the data obtained in the

Belgian EQA for alcohol determination in blood.

In brief, this work describes in a critical and constructive way current statistical

methods used in EQA and proposes novel statistical and graphical

techniques to help alleviating the future needs of External Quality Assessment

programmes.

Bien que le maximum ait été fait pour que les droits des ayants-droits soient respectés, si un de ceux-ci constatait qu'une oeuvre sur laquelle il a des droits a été utilisée dans BICTEL/e ULg sans son autorisation explicite, il est invité à prendre contact le plus rapidement possible avec la Direction du Réseau des Bibliothèques.