Going the other direction is complicated by the fact that service request and response objects, which walk and talk like messages, aren’t actually messages. So populating a call to a service and reading the response requires specifying some code in advance that knows how to translate a service response into a proper message. You can’t use the response object as a topic message type directly since it’s not actually a visible message type and roscore will complain that it can’t find the message definition.

In general, it would be nice if services and messages could be specified dynamically, but I suspect that the performance requirements of serialization require that these be specified in advance and compiled.

More importantly, if this was my answer, would I get a job at Google?

UPDATE: I don’t know the linear time solution, but I feel like if I could grok smoothsort I could figure it out.

Trying anyways: how ignoring the errors may help in learning new skills
B Grzyb, Jaume I University; J Boedecker, M Asada, Osaka University;
A del Pobil, Jaume I University; L Smith, Indiana University

There was another interesting talk on how agents developed shared meanings even though agents had different bodies. You may think this is a sort of strange thing to study, but it turns out that we all have different bodies, and if studies of retinal variation are to be believed, we all likely experience color qualia in different ways, and yet we have a shared understanding of color concepts. The program citation:

The development of shared meaning within different embodiments
J De Greeff, T Belpaeme, University of Plymouth

As a side note, this year’s conference seems to have much better branding/website design than any previous year. I wonder if the organizers decided to hire a designer for the conference. In any event, it’s a welcome change from some of the geocities-esque designs of the past.

As I mention in the project readme, pyrouette is meant as a lightweight Pythonic framework for exploring the implementation of machine learning algorithms. The goal is more pedagogical than research oriented, and the project (whose genesis was my own habit of implementing algorithms as I learned about them) will hopefully compliment more framework heavy and feature complete projects like Pybrain, MDP, and Scikits.learn.

To the extent possible, individual algorithms (in alg/) are as self-contained as possible, depending where necessary on readily available numpy and scipy packages. In many cases, more feature complete versions of these algorithms can be found elsewhere. If you have a similar habit of implementing algorithms in Python and would like to share, let me know!

UPDATE: Some experiments don’t work out. I find it is easier to put individual learning algorithms on gist or in individual repositories rather than trying to develop a learning algorithm framework. Check out my gists or repos on GitHub if you are looking for something that was formerly part of Pyrouette.

When I sat down to start coding I realized I had to implement a multivariate Gaussian pdf and all of a sudden things started looking very familiar. A couple of Google desktop queries led me to the following (written in 2008!). You’ll need this file of old faithful data to run the script, as well as matplotlib for the visualization.

An n-armed bandit is a kind of interactive optimization problem where an agent plays a game that involves “pulling” one of n arms. Just like in a casino, the agent then gets some kind of reward after each pull. The problem the agent has to solve is to figure out what choice of arm is best (e.g. will return the most reward) after repeated trials. This choice is complicated by the fact that the reward for pulling each arm is not fixed, but is instead distributed according to some unknown (but usually fixed) distribution.

The problem may seem either somewhat trivial or too abstract, depending on your particular disposition and knowledge, but in fact this problem provides a natural framework for thinking about what is known as the exploration-exploitation dilemma. At each iteration of the game, an agent can either go with the action that the agent thinks is best, or try to collect new data that may result in the agent discovering an even better solution.

One surprisingly effective and simple strategy is to just be greedy. Unfortunately, just being greedy can cause problems if you end up thinking the wrong arm is optimal early on in your experience playing the game. To address this issue, agents can adopt what’s known as an -greedy strategy where the agent spends some small percentage of moves just randomly exploring other actions. This kind of policy comes up a lot in reinforcement learning (a more general class of problems of which n-armed bandits are one type), but this policy is particularly concise when expressed in code for solving just these kinds of simple bandit problems.

The only thing you would need to specify to run this code is the bandit itself, which would have one method (run) that returns a (possibly stochastic) reward for each action (arm pulled).

Then I remembered the setdefault method and was able to create a more concise function:

Of course if your dictionary is 1-1, you probably don’t want to introduce sets into the reverse version, so I created a function that composes nicely with the previous function:

This comic will now appear at the beginning of every talk I give between now and the end of time.