Monte Carlo policy evaluation

source("./Monte_Carlo_policy_evaluation_blackjack.R")

Figure 5.1: Approximate state-value functions for the blackjack policy that sticks only on 20 or 21

plot_fig5.1()

Monte Carlo ES

source("./Monte_Carlo_ES.R")

Figure 5.2: The optimal policy and state-value function for blackjack

plot_fig5.2()

Monte Carlo off-policy

source("./MC_off_policy_blackjack.R")

Figure 5.3: Weighted importance sampling

plot_fig5.3()
## function (x, y, ...) 
## UseMethod("plot")
## <bytecode: 0x0000000019f02d78>
## <environment: namespace:base>

Infinite variance

source("./InfinteVariance.R")

Figure 5.4: Ordinary importance sampling produces surprisingly unstable estimates

plot_fig5.4()