Wisdom of crowds

Ullrika

Quantity to be elicited

The healthy life years for females in Norway 2003

Healthy life years is defined as the number of remaining years that a person at birth is expected to live without any severe or moderate health problems.

[Link to definition on eurostat] (https://ec.europa.eu/eurostat/cache/metadata/en/hlth_hlye_h_esms.htm)

Procedure (1)

  • We discussed the quantity and enusured that it was well-defined

  • The experts (students) made their judgements indpendent of each other

  • Each student fitted a distribution to their judgements and extracted a 80% probability interval

  • The judgements (five quantiles) and the probability interval was shared with the facilitator (Ullrika)

Procedure (2)

  • The facilitator uploaded the judgements into the SHELF app for multiple experts

Procedure (3)

  • Judgements shown by the quartiles

Procedure (3)

  • Judgements shown as “histograms”

Procedure (3)

  • Judgements shown by the best fitting probability distribution

The aggregated judgement - linear pool

Individual judgements

  • Judgements shown as 80% probability intervals

With the aggregated judgement

  • Aggregation made taking the linear pool

With the true value

  • The true value taken from eurostat is 64.2 years (red line)

Comment

  • The average number of healthy years for females in Norway is likely to be below but close to the life expectancy, and unlikely to be close to 0 or above 100

  • Some probability distributions allows for negative values, which is problematic when calculating the probability intervals

  • Take home message:

    • make sure the experts understand the quantity, e.g. by asking them to justify their judgements

    • the final probability distribution must be chosen with care