Expert judgement

Ullrika Sahlin

2026-10-13

Linguistic uncertainty

There is life on Mars

Impact on biodiversity is high

Vagueness: natural and scientific language allows borderline cases e.g. the number of mature plants
Ambiguity: some words have more than one meaning and it is not clear which meaning is intended e.g. cover
Underspecificity: unwanted generality in data

The factory constitute a moderate risk

There is a 30% probability of rain tomorrow

Prominent in qualitative risk assessment due to the use of qualitative expressions
Can be avoided by using well defined questions, quantities and outcomes and meaningful scales

A tool from Mark and his colleagues to reduce linguistic uncertainty

Rain - There will be rain tomorrow

Space - Aliens from space will arrive to the earth in our lifetime

Trump - Trump becomes the next president of USA

Use the AUSTRALIAN SCALES!

Make judgements with single scores e.g. 3 or ranges, e.g. 2-4

Make individual judgements
Compare where there are large differences in judgements.
Discuss and resolve the reason for differences. Here is where linguistic uncertainties can be reduced.
Redo individual judgements
Aggregate the judgements

Formal, structured approaches designed to

retrieve judgements from a group of experts by a systematic, documented and reviewable process
minimise cognitive biases

Two scales to collect judgements on the probability of an offence

Question to students in previous class:

How long time does it take to drive from Lund to Umeå in hours?

Wisdom of the crowd

Mathematical aggregation: with or without expert calibration
Behavioural aggregation: facilitated interaction
Delphi method: multiple rounds of individual judgements followed by summary of judgements after every round until convergence is obtained
Combinations of these e.g. IDEA and the SAHLIN-protocol*

*in development

The ideal group of experts will

have sufficient diversity of experience and opinion to achieve the desired coverage, and
be willing to share their opinions and to listen to the opinions of others

An expert who is highly-respected but is arrogant and unwilling to acknowledge any contrary opinion
A very senior person who has become a manager and lacks recent experience ‘at the coal-face’
A junior person who is unwilling to contribute their views, or who will simply defer to more senior members of the group
A representative of a stakeholder organisation who will always assert the organisation’s position
An expert whose knowledge and opinions are expected to be very close to those of another expert in the group

Evidence dossier: