depth first search

In response to a recent Coding Horror piece on Spartan programming I’d like to humbly submit two examples from my own recent work.

First, in C++ we have:

for (int i = 0; i < nstates; i++) {
    for (int a = 0; a < nactions; a++) {
        Q[i][a] += alpha * delta * e[i][a];
        e[i][a] *= gamma * lambda;
    }
}

Next, in Python with Numpy:

self.Q += self.alpha * delta * self.e
self.e *= self.gamma * self.ld

These are identical temporal difference updates in two versions of a Sarsa reinforcement learning agent. Having reasonable syntax and semantics for vector and matrix operations is essential for sparse programming in any kind of numerically intensive setting.

•