Chapter 9

State aggregation on the 1000-state random walk

source("./state_aggregation_random_walk.R")

Figure 9.1: the gradient Monte Carlo algorithm

plot_fig9.1()

Figure 9.2: Bootstrapping with state aggregation

Asymptotic values of semi-gradient TD

plot_fig9.2.1()

Performance of n-step methods with state-aggregation

plot_fig9.2.2()

Figure 9.5: Fourier basis vs polynomials

plot_fig9.5()

Coarseness of coarse coding

source("./generalization.R")

Figure 9.8: Example of feature width’s effect

plot_fig9.8()