Decision theory

BADT

Author

Sahlin

Generic setup for decison making

Agents
Decision alternatives
Preferences over outcomes
Impact on outcomes for each decision alternative
An idea of what is a good decision

Decision rules:

Bayesian Decision Theory: Maximise expected utility

There are other decision rules!

DT

Descriptive: What people actually do, or have done
Prescriptive: What people should and can do
Normative: What people should do (in theory, e.g. if rational)

Medium read on Decision theory at Stanford Encyclopedia of Philosophy Archive

Preferences

An agent prefers option \(A\) over \(B\) means that the agents takes \(A\) to be more desirable or choice-worthy than \(B\)

Indifference \(\sim\)

Weak preference \(\preccurlyeq\)

Strong preference \(\prec\)

Rational preferences of options can be seen to be given by:

Completeness

For any \(A, B\in S:\text{ either }A \preccurlyeq B\text{ or } B \preccurlyeq A\)

Transitivity

For any \(A, B, C \in S: \text{ if }A\preccurlyeq B\text{ and } B\preccurlyeq C \text{ then } A \preccurlyeq C\)

Utilities

Utility as a way to measure preferences

\[\text{For any }A,B \in S: u(A)\leq u(B) \Longleftrightarrow A \preccurlyeq B\]

Maximise Expected Utility

Let \(p_{ik}\) be the the probability for outcome \(O_{ik}\) in lottery \(L_i\)

von Newmann Morgenstern representation theorem of the expected utility of lottery \(L_i\)

\[EU(L_i)=\sum_ku(O_{ik})\cdot p_{ik}\]

Subjective Expected Utility (SEU)

Savage

Set of outcomes \(\mathbf{O}\) - target of desire

Set of states of the world \(\mathbf{S}\) - target of belief

An act can be seen as a function from the states to outcomes. We have at least to acts \(f\) and \(g\)

\(f(s_i)\) denotes the outcome of act \(f\) when event \(s_i \in \mathbf{S}\) is actually happening

The expected utility of act \(f\) is given by Savage’s equation

\[U(f)=\sum_i u(f(s_i))\cdot p(s_i)\]

Savage defined six axioms, which when satisfied, generates that a persons belief can be represented by a unique probability function which relates the theory to the theory to maximise expected utility (von Newmann Morgenstern representation theorem)

To maximise expected utility can be seen as a decision rule for decision problems where the probability of different outcomes are known or when the probability for different outcomes represents a persons belief about that outcome.

It is possible to derive a persons belief by asking here to choose between options. This is often done asking them to choose between lotteries with different costs and expected utilities. Betting interpretation of subjective probability.

Bayesian decision theory

Bayesian Belief Network

Wet grass - Rain- Sprinkler -Umbrella

I will demonstrate BBN and influence diagram in genie

Bayesian decision analysis example in stan

https://mc-stan.org/docs/stan-users-guide/decision-analysis.html

# Save this code into a file named "dm.stan"

functions {
  real U(real c, real t) {
    return -(c + 25 * t);
  }
}
data {
  int<lower=0> N;
  array[N] int<lower=1, upper=4> d;
  array[N] real c;
  array[N] real<lower=0> t;
}
parameters {
  vector[4] mu;
  vector<lower=0>[4] sigma;
  array[4] real nu;
  array[4] real<lower=0> tau;
}
model {
  mu ~ normal(0, 1);
  sigma ~ lognormal(0, 0.25);
  nu ~ normal(0, 20);
  tau ~ lognormal(0, 0.25);
  t ~ lognormal(mu[d], sigma[d]);
  c ~ lognormal(nu[d], tau[d]);
}
generated quantities {
  array[4] real util;
  for (k in 1:4) {
    util[k] = U(lognormal_rng(nu[k], tau[k]),
                lognormal_rng(mu[k], sigma[k]));
  }
}

library(cmdstanr)

This is cmdstanr version 0.9.0

- CmdStanR documentation and vignettes: mc-stan.org/cmdstanr

- CmdStan path: C:/Users/ekol-usa/.cmdstan/cmdstan-2.38.0

- CmdStan version: 2.38.0

library(ggplot2)
library(bayesplot)

This is bayesplot version 1.15.0

- Online documentation and vignettes at mc-stan.org/bayesplot

- bayesplot theme set to bayesplot::theme_default()

   * Does _not_ affect other ggplot2 plots

   * See ?bayesplot_theme_set for details on theme setting

generate_fake_data <- function(N){
d = sample(1:4,N,replace=TRUE)
d_label = c("walk", "bike share", "public transportation", "cab")

mu = sort(rnorm(4,0,0.5),decreasing=TRUE)
sigma = rlnorm(4,0,0.1)
nu = sort(rnorm(4,0,1),decreasing=FALSE)
tau = rlnorm(4,0,0.25)

data.frame(
t = rlnorm(N,mu[d],sigma[d]),
c = rlnorm(N,nu[d],tau[d]),
d = d, d_label = d_label[d])
}

# prepare data
dat <- generate_fake_data(N = 50)

df <- data.frame(val = c(dat$t,dat$c), d = c(dat$d_label,dat$d_label), variable = rep(c("time","cost"),each=nrow(dat)))
ggplot(df,aes(x=val,y=d,col=d)) +
  geom_boxplot() + 
  geom_jitter(alpha=0.3) +
  facet_wrap(~variable) +
  theme_minimal()

# turn data to list
stan_data <- list(N = nrow(dat),t=dat$t,c=dat$c,d=dat$d)

# compile model
model.dm <- cmdstan_model(stan_file="dm.stan")

# perform the mcmc sampling
post <- model.dm$sample(data=stan_data)

Running MCMC with 4 sequential chains...

Chain 1 Iteration:    1 / 2000 [  0%]  (Warmup) 
Chain 1 Iteration:  100 / 2000 [  5%]  (Warmup) 
Chain 1 Iteration:  200 / 2000 [ 10%]  (Warmup) 
Chain 1 Iteration:  300 / 2000 [ 15%]  (Warmup) 
Chain 1 Iteration:  400 / 2000 [ 20%]  (Warmup) 
Chain 1 Iteration:  500 / 2000 [ 25%]  (Warmup) 
Chain 1 Iteration:  600 / 2000 [ 30%]  (Warmup) 
Chain 1 Iteration:  700 / 2000 [ 35%]  (Warmup) 
Chain 1 Iteration:  800 / 2000 [ 40%]  (Warmup) 
Chain 1 Iteration:  900 / 2000 [ 45%]  (Warmup) 
Chain 1 Iteration: 1000 / 2000 [ 50%]  (Warmup) 
Chain 1 Iteration: 1001 / 2000 [ 50%]  (Sampling) 
Chain 1 Iteration: 1100 / 2000 [ 55%]  (Sampling) 
Chain 1 Iteration: 1200 / 2000 [ 60%]  (Sampling) 
Chain 1 Iteration: 1300 / 2000 [ 65%]  (Sampling) 
Chain 1 Iteration: 1400 / 2000 [ 70%]  (Sampling) 
Chain 1 Iteration: 1500 / 2000 [ 75%]  (Sampling) 
Chain 1 Iteration: 1600 / 2000 [ 80%]  (Sampling)

Chain 1 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:

Chain 1 Exception: lognormal_lpdf: Scale parameter[1] is inf, but must be positive finite! (in 'C:/Users/ekol-usa/AppData/Local/Temp/Rtmpw19PU5/model-1c58448a5d5d.stan', line 23, column 2 to column 33)

Chain 1 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,

Chain 1 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

Chain 1

Chain 1 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:

Chain 1 Exception: lognormal_lpdf: Scale parameter[1] is inf, but must be positive finite! (in 'C:/Users/ekol-usa/AppData/Local/Temp/Rtmpw19PU5/model-1c58448a5d5d.stan', line 23, column 2 to column 33)

Chain 1 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,

Chain 1 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

Chain 1

Chain 1 Iteration: 1700 / 2000 [ 85%]  (Sampling) 
Chain 1 Iteration: 1800 / 2000 [ 90%]  (Sampling) 
Chain 1 Iteration: 1900 / 2000 [ 95%]  (Sampling) 
Chain 1 Iteration: 2000 / 2000 [100%]  (Sampling) 
Chain 1 finished in 0.1 seconds.
Chain 2 Iteration:    1 / 2000 [  0%]  (Warmup) 
Chain 2 Iteration:  100 / 2000 [  5%]  (Warmup) 
Chain 2 Iteration:  200 / 2000 [ 10%]  (Warmup) 
Chain 2 Iteration:  300 / 2000 [ 15%]  (Warmup) 
Chain 2 Iteration:  400 / 2000 [ 20%]  (Warmup) 
Chain 2 Iteration:  500 / 2000 [ 25%]  (Warmup) 
Chain 2 Iteration:  600 / 2000 [ 30%]  (Warmup) 
Chain 2 Iteration:  700 / 2000 [ 35%]  (Warmup) 
Chain 2 Iteration:  800 / 2000 [ 40%]  (Warmup) 
Chain 2 Iteration:  900 / 2000 [ 45%]  (Warmup) 
Chain 2 Iteration: 1000 / 2000 [ 50%]  (Warmup) 
Chain 2 Iteration: 1001 / 2000 [ 50%]  (Sampling)

Chain 2 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:

Chain 2 Exception: lognormal_lpdf: Scale parameter[18] is inf, but must be positive finite! (in 'C:/Users/ekol-usa/AppData/Local/Temp/Rtmpw19PU5/model-1c58448a5d5d.stan', line 23, column 2 to column 33)

Chain 2 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,

Chain 2 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

Chain 2

Chain 2 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:

Chain 2 Exception: lognormal_lpdf: Scale parameter[18] is inf, but must be positive finite! (in 'C:/Users/ekol-usa/AppData/Local/Temp/Rtmpw19PU5/model-1c58448a5d5d.stan', line 23, column 2 to column 33)

Chain 2 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,

Chain 2 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

Chain 2

Chain 2 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:

Chain 2 Exception: lognormal_lpdf: Scale parameter[1] is 0, but must be positive finite! (in 'C:/Users/ekol-usa/AppData/Local/Temp/Rtmpw19PU5/model-1c58448a5d5d.stan', line 23, column 2 to column 33)

Chain 2 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,

Chain 2 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

Chain 2

Chain 2 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:

Chain 2 Exception: lognormal_lpdf: Scale parameter[1] is inf, but must be positive finite! (in 'C:/Users/ekol-usa/AppData/Local/Temp/Rtmpw19PU5/model-1c58448a5d5d.stan', line 24, column 2 to column 31)

Chain 2 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,

Chain 2 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

Chain 2

Chain 2 Iteration: 1100 / 2000 [ 55%]  (Sampling) 
Chain 2 Iteration: 1200 / 2000 [ 60%]  (Sampling) 
Chain 2 Iteration: 1300 / 2000 [ 65%]  (Sampling) 
Chain 2 Iteration: 1400 / 2000 [ 70%]  (Sampling) 
Chain 2 Iteration: 1500 / 2000 [ 75%]  (Sampling) 
Chain 2 Iteration: 1600 / 2000 [ 80%]  (Sampling) 
Chain 2 Iteration: 1700 / 2000 [ 85%]  (Sampling) 
Chain 2 Iteration: 1800 / 2000 [ 90%]  (Sampling) 
Chain 2 Iteration: 1900 / 2000 [ 95%]  (Sampling) 
Chain 2 Iteration: 2000 / 2000 [100%]  (Sampling) 
Chain 2 finished in 0.1 seconds.
Chain 3 Iteration:    1 / 2000 [  0%]  (Warmup) 
Chain 3 Iteration:  100 / 2000 [  5%]  (Warmup) 
Chain 3 Iteration:  200 / 2000 [ 10%]  (Warmup) 
Chain 3 Iteration:  300 / 2000 [ 15%]  (Warmup) 
Chain 3 Iteration:  400 / 2000 [ 20%]  (Warmup) 
Chain 3 Iteration:  500 / 2000 [ 25%]  (Warmup) 
Chain 3 Iteration:  600 / 2000 [ 30%]  (Warmup) 
Chain 3 Iteration:  700 / 2000 [ 35%]  (Warmup) 
Chain 3 Iteration:  800 / 2000 [ 40%]  (Warmup) 
Chain 3 Iteration:  900 / 2000 [ 45%]  (Warmup) 
Chain 3 Iteration: 1000 / 2000 [ 50%]  (Warmup) 
Chain 3 Iteration: 1001 / 2000 [ 50%]  (Sampling) 
Chain 3 Iteration: 1100 / 2000 [ 55%]  (Sampling) 
Chain 3 Iteration: 1200 / 2000 [ 60%]  (Sampling) 
Chain 3 Iteration: 1300 / 2000 [ 65%]  (Sampling)

Chain 3 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:

Chain 3 Exception: lognormal_lpdf: Scale parameter[1] is inf, but must be positive finite! (in 'C:/Users/ekol-usa/AppData/Local/Temp/Rtmpw19PU5/model-1c58448a5d5d.stan', line 23, column 2 to column 33)

Chain 3 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,

Chain 3 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

Chain 3

Chain 3 Iteration: 1400 / 2000 [ 70%]  (Sampling) 
Chain 3 Iteration: 1500 / 2000 [ 75%]  (Sampling) 
Chain 3 Iteration: 1600 / 2000 [ 80%]  (Sampling) 
Chain 3 Iteration: 1700 / 2000 [ 85%]  (Sampling) 
Chain 3 Iteration: 1800 / 2000 [ 90%]  (Sampling) 
Chain 3 Iteration: 1900 / 2000 [ 95%]  (Sampling) 
Chain 3 Iteration: 2000 / 2000 [100%]  (Sampling) 
Chain 3 finished in 0.1 seconds.
Chain 4 Iteration:    1 / 2000 [  0%]  (Warmup) 
Chain 4 Iteration:  100 / 2000 [  5%]  (Warmup) 
Chain 4 Iteration:  200 / 2000 [ 10%]  (Warmup) 
Chain 4 Iteration:  300 / 2000 [ 15%]  (Warmup) 
Chain 4 Iteration:  400 / 2000 [ 20%]  (Warmup) 
Chain 4 Iteration:  500 / 2000 [ 25%]  (Warmup) 
Chain 4 Iteration:  600 / 2000 [ 30%]  (Warmup) 
Chain 4 Iteration:  700 / 2000 [ 35%]  (Warmup)

Chain 4 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:

Chain 4 Exception: lognormal_lpdf: Scale parameter[9] is inf, but must be positive finite! (in 'C:/Users/ekol-usa/AppData/Local/Temp/Rtmpw19PU5/model-1c58448a5d5d.stan', line 24, column 2 to column 31)

Chain 4 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,

Chain 4 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

Chain 4

Chain 4 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:

Chain 4 Exception: lognormal_lpdf: Scale parameter[9] is inf, but must be positive finite! (in 'C:/Users/ekol-usa/AppData/Local/Temp/Rtmpw19PU5/model-1c58448a5d5d.stan', line 24, column 2 to column 31)

Chain 4 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,

Chain 4 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

Chain 4

Chain 4 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:

Chain 4 Exception: lognormal_lpdf: Scale parameter[1] is inf, but must be positive finite! (in 'C:/Users/ekol-usa/AppData/Local/Temp/Rtmpw19PU5/model-1c58448a5d5d.stan', line 23, column 2 to column 33)

Chain 4 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,

Chain 4 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

Chain 4

Chain 4 Iteration:  800 / 2000 [ 40%]  (Warmup) 
Chain 4 Iteration:  900 / 2000 [ 45%]  (Warmup) 
Chain 4 Iteration: 1000 / 2000 [ 50%]  (Warmup) 
Chain 4 Iteration: 1001 / 2000 [ 50%]  (Sampling) 
Chain 4 Iteration: 1100 / 2000 [ 55%]  (Sampling) 
Chain 4 Iteration: 1200 / 2000 [ 60%]  (Sampling) 
Chain 4 Iteration: 1300 / 2000 [ 65%]  (Sampling) 
Chain 4 Iteration: 1400 / 2000 [ 70%]  (Sampling) 
Chain 4 Iteration: 1500 / 2000 [ 75%]  (Sampling) 
Chain 4 Iteration: 1600 / 2000 [ 80%]  (Sampling) 
Chain 4 Iteration: 1700 / 2000 [ 85%]  (Sampling) 
Chain 4 Iteration: 1800 / 2000 [ 90%]  (Sampling) 
Chain 4 Iteration: 1900 / 2000 [ 95%]  (Sampling) 
Chain 4 Iteration: 2000 / 2000 [100%]  (Sampling) 
Chain 4 finished in 0.2 seconds.

All 4 chains finished successfully.
Mean chain execution time: 0.1 seconds.
Total execution time: 1.3 seconds.

post$diagnostic_summary()

$num_divergent
[1] 0 0 0 0

$num_max_treedepth
[1] 0 0 0 0

$ebfmi
[1] 0.9753964 1.0315801 1.0048263 1.0552105

# summarise the relevant quantities of interest 
draws <- post$draws()
bayesplot::mcmc_trace(draws,pars = c("util[1]","util[2]","util[3]","util[4]"))

bayesplot::mcmc_intervals(post$draws(),pars = c("util[1]","util[2]","util[3]","util[4]"), prob=0.8)

# Calculate the expected utilities 

df_draws <- posterior::as_draws_df(draws)
df_dm <- data.frame(EU = colMeans(df_draws[,c("util[1]","util[2]","util[3]","util[4]")]),
           d = c("walk", "bike share", "public transportation", "cab"))

Warning: Dropping 'draws_df' class as required metadata was removed.

df_dm$bayesopt <- ""
df_dm$bayesopt[order(df_dm$EU,decreasing=TRUE)[1]] <- "*"
df_dm

               EU                     d bayesopt
util[1] -18.45770                  walk        *
util[2] -32.22089            bike share         
util[3] -39.45292 public transportation         
util[4] -50.98129                   cab