Like many good ideas, this test has caused a lot of controversy over the years, and as artificial intelligence has moved away from the study of the mind and into concrete application areas, discussions about intelligence tests (Turing and otherwise) have languished somewhat. There are occasionally some exceptions to this general trend. For instance, at this years ICDL in Ann Arbor, Jivko Sinapov and Alexander Stoytchev presented an intelligence test for robots in their award winning paper, “The odd one out task: Toward an intelligence test for robots.”

One property that intelligence tests seem to all share is that they depend critically on human observers building internal models of agents that are indistinguishable from the models human observers build for other humans. Even in constrained situations like the odd one out task, these kinds of assessments usually take the form of specifying a task that human and other agents can both perform. Our interpretation of the functional nature of the performance becomes the basis for assessing whether the agent has “passed” the test.

This seems to lead to a number of interesting problems. For one, there is a many-to-one correspondence between methods and results (many methods may yield the same behavior), so our “attributing” human-like models to agents may in fact be a faulty attribution (the agent could be doing something simple but demonstrating the correct contextual behavior). Of course there’s the opposite problem of recognizing that intelligent behavior may not necessarily look like human behavior, and so even if we fail to attribute human-like models to artificial agents, we may be missing other signs of intelligence. I imagine that this kind of error gives science fiction authors and people searching for other intelligent life in the universe headaches.

There have been a number of alternate proposals for intelligence tests geared towards developmental agents, most recently in a paper by Paul Cohen. In that paper, Cohen summarizes a number of possible replacements for the venerable Turing Test:

1. Robot soccer
2. Never ending language learning
3. A virtual third grader

Unlike these proposals, the new body problem lacks a easily describable goal. This seems to arise naturally in any relatively task-free scenario, and many of Cohen’s examples skirt the issue by incorporating a variety of task or task-like elements in each test. The new body problem simply states a desire to learn about a new body starting from scratch. It’s simple but the criterion for success if still vague. What ought an agent that can solve the new body problem be able to achieve? Certainly we can imagine any number of tasks that we could use to test an agent, but any restraint on the tasks would lead us back into the morass of subjectively attributing human like models of intelligence to agents.

Here’s a possibility that I’m currently considering. What if the end goal of an agent solving the new body problem is to describe itself? We would have to apply subjective kinds of judgments to the descriptions provided (e.g. a low level inventory of raw sensors and motors is not really demonstrative of the kind of development we would like to see). On the other hand, we do not seem to have the same kind of a priori biases in evaluating an agent describing itself than we would evaluating an agent within a particular task domain. In other words, we would have an open ended evaluation for an open ended learning problem, which strikes me as precisely the kind of particularist approach we would ultimately need if we ever wanted to truly test for intelligence.

Viewed from a distance, the idea of a paradigm, paradigm-shift, and the more general and complete scientific revolution all depend in part on never really knowing the mechanism that, deep down, really drives the process. Kuhn resorts to all kinds of tricks to avoid having to really tackle with what originates revolutions, instead describing in great and useful detail the pre and post effect periods, and in many ways describing everything that the black box does without ever getting inside to see how the black box does it.

But I guess this leaves the whole scaffolding subject to a kind of attack based less on science and more on the idea of revolution. It is a loaded term, and one that, for certain theories of creativity and intuition is probably misapplied. We can mount various kinds of hypothetical attacks on shifting scientific commitments by pointing out that, had we an actually theory of creativity mechanising the theory generating aspect of the structure, the larger (theory generation + paradigm shift system) might end up looking less then revolutionary.

I did come across an interesting critique of Popper’s notion of falsifiability. The critique reminded me of this famous quote from George Box:

Essentially, all models are wrong, but some are useful.

The problem with falsifiability is that most theories are actually not just falsifiable but are in fact false. What one then needs is a notion of less false, which is precisely analogous to the verification-lite notion  of more correct.

We all know that women tend to be more expressive about their complaints – you can’t beat ‘em for wailing and gnashing of teeth.

Really? That hasn’t been my experience. And anyway, even by the flimsy standards of economics this particular claim is entirely unsupported.

This serves to reinforce my outsider view of the discipline: you folks spend all your time trying to formulate counterintuitive claims that you can support with limited data and faulty statistics. In the process you routinely boil morality, ethics, and the human condition down to single estimates of utility in conveniently succinct but laughably unsupportable ways. Then, model in hand, you commit the cardinal sin of economics: you confuse your model with reality.

Jeebus. Doesn’t the other 90% of your intellect tell you that your conclusions are totally ridiculous? When I get to such a perverse place in my own thinking I usually ask myself where I went wrong instead of tumbling forward like some goddamn moron.

UPDATE: Because economists might read this let me be a bit more explicit for the slow folks in the room. I often write so as to simulate precisely what I’m writing about, so my universal statement “economists are terrible people” is intended to evoke the precisely kind of distaste that the quoted comment above would. It’s a subtle point that individuals with decent reading skills (who must not be economists [I did it again!]) would understand.

And if people of the future happen upon this desolate and unremarkable wasteland, what then? Well, some of them get sick and die. We have  to trust that humanity’s ability to formulate causal models from that kind of data remains intact, and so the rest will relearn the forgotten lessons of the past. In other words, the best way to communicate with the future by doing nothing special and trusting that they will figure it out.

UPDATE: Ana called me out for not reading the linked article (just the pull quote). This is one of those cases where a piece of news is making multiple circuits around the web, and I had assumed (wrongly) that the links led to an article I had read awhile ago, and not the current (and quite interesting) interview.

I also have to revise my own plan somewhat. A Rosetta stone like monument is clearly the best bet for the short term, assuming some language survives. Though computational linguistics is making progress decoding languages with no Rosetta stone analogue, the ability for future generations to interpret signs increases dramatically if one of the available languages is known.

The case where no language survives in its current form is more complex. Here’s were my plan makes a bit more sense. You have to weigh the probability of discovery against the probable protocols future humans (or others) might employ should the site be discovered. I was imagining an ideal scenario where the site could avoid detection for a million years.

But what if it is discovered? I think the best result is to have some kind of subtle but lasting monument, and let the experience of exploring a radioactive hot zone (and the inevitable bad result) lead future explorers to the correct conclusion about the meaning of the monument. Basically, if people find the repository, we want to make sure they explore it, and by exploring it, learn the nature of the danger. This avoids the terrible case where people settle the area without any knowledge of the danger lurking underneath.

I’m still compiling my notes from both conferences, and hope to distill some of those ideas into entries, but I thought I’d start with an intriguing idea from Professor Barto, that “motivation is the gradient of the value function.” One approach to reinforcement learning, where an agent tries to act so as to maximize reward over time, is to compute value functions, which assign values to each state of the world that are intended to reflect estimates of future rewards agents can hope to achieve from those states acting as they are. Though I think this description is accurate, it is certainly horribly concise, so I’d recommend the book if you are intrigued enough to learn more.

Anyway, if an agent has a value function that represents the best an agent can expect in terms of future rewards for any state, then an agent has enough information to act optimally, since it simply needs to look to the next state with highest value. The gradient, or change in value from state to state, drives the choice of agent actions, and so can be considered a kind of motivation. We should probably complicate this further by noting that agents have to learn value functions, and that not all value functions are created equal. In fact, there’s a unique value function, the optimal value function, that represents the best an agent can do from any particular state. If the agent is gifted with this value function, then gradient as motivation makes sense. If not, then we have to consider the need to change the value function to better approximate the optimal, alongside the need to follow the value function in some greedy way.

[Aside: Thinking off the cuff, we can view the need to properly approximate the optimal value function as part of a kind of meta-value function, which doesn't just consider the values of world states relative to the reward function, but also considers the value of proper value function estimation. And so on down the rabbit hole...]

But all this complexity seems to avoid the tricky issue of motivation. For one thing, we, as the specifiers of the algorithm, are setting up the agent to follow value function gradients (or exploratory gradients) as a consequence of the way the problem is set up. In some sense, this is unappealing since it leaves aside any explanation for why an agent should follow value functions in the first place. Put another way, value function gradients as motivation for behavior only make sense in the context of a reward function that indicates good outcomes (if not the method of achieving those outcomes). So this just begs the question, where do rewards come from? This is a question that reinforcement learning conveniently avoids answering by assuming from the outset (at least in theory) that rewards are given.

“Where do rewards come from?” was, not surprisingly, the title of Andrew Barto’s workshop presentation, so I may very well just be recapitulating his own line of reasoning on the matter. His talk summarized a very interesting piece of work that looked at how evolution can act on reward functions that result in learning agents with better fitness. The presentation made a point about reward functions that I’ve thought of independently, but which psychologists have already enunciated in various forums. The point is this: reward signals are nothing special to the world, even though they are special to the agent. The universe does not care that you go hungry; you care that you go hungry. Drawing rewards as a distinct signal in standard reinforcement learning diagrams does not make sense. Rewards are just normal state signals with a special (internal and evolution-mediated) interpretation by the agent.

If you understand everything you are doing, you are a clerk. Nothing wrong with that. Clerks run most things.

[ Aside: I should note that I'm not protectionist with regards to titles like "intellectual." You can clerk by day and intellectualize by night, or on weekends, or when fishing. It's not the kind of club that has membership dues. If you struggle to understand new things, even old things that are new to you, than you are in. If you don't you're not. ]

Anyway, I was pressed into service to explain the meaning behind quotes from the other day.

[ Aside: Here I should pause to mention that my wife sometimes forces me to explain things she already understands, or that she doesn't have time to read herself. ]

I thought I did an okay job of it, so I thought I’d share with you what I understand so far of Karl Popper’s The Logic of Scientific Discovery. The first thing you should know about this book is that it basically is the foundation of our modern understanding of science. If you ever hear about falsifiability as a criterion for scientific theory, you have Popper to thank. The ideas of TLSD are so pervasive in how scientists now view their own work, that one wonders if Popper hasn’t done the unthinkable and settled a philosophical question for good. Since I’m not plugged into the larger philosophy of science community I don’t really know if that is a correct characterization of TLSD or not, but it certainly is an influential text.

So what was Karl Popper’s project? His goal was to characterize science. To do this he tried to identify the formal logic (or rules) that govern the scientific process. Anything that follows the logic of science can rightly be called science, and anything that does not cannot. The logic of science, once identified, characterizes science. This probably seems like a daunting task, but TLSD is written in a way that when read it is at once both obvious and airtight. Part of the trick on Popper’s part is to conveniently avoid some of the harder questions, like where scientific theories come from (e.g. creativity?). In doing so, he is able to assume that there are things called scientific theories, and explore what properties we expect them to have.

One of the first schools of thought that Popper has to deal with when exploring the logic of science, is the idea that science, rather than being entirely logical, contains a psychological or interpretive component (beyond those that Popper has already conceded). If this is true it is problematic for the project, because then any logic of science would also have to include a logic of psychology, thus enlarging the original problem and removing all hope of tractability. Popper sidesteps this problem by noting, as in the first quote, that while observations and perceptions are psychological, observability is not. Observability is a property of the world. The trick here, is that when describing the logic of science, and just the logic, observability is enough. Observability is one (conveniently non-psychological) property of basic statements about the world needed to falsify theories.

[ Aside: I'm being somewhat vague about basic statements. Popper is very careful defining the concept, and to do so with any rigour requires the many pages Popper spends on the topic. For our purposes,  you can think of basic statements as things that are observable, and that if observed, may falsify a theory. Interestingly, there is a kind of dual relationship between basic statements and theories, since theories have to be constructed in order to have falsifying basic statements. There's no circularity in Popper's definition, however, since basic statements exist independent of theories. Theories just partition an already existing set of basic statements into falsifying and non-falsifying ones. ]

By setting up this framework of basic statements and the partitioning power of theories, Popper is then able to argue that theories are an essential part of science, and in particular, just collecting facts about the world is not enough. That is the meaning I draw from the second quote. It is here, however, where I think Popper gets away with too much. He ignores the interesting question of where theories come from. They certainly don’t spring fully formed from the mind. It seems entirely plausible that theories form through some process running in our heads as we are “collecting and arranging our experiences.” But once we have a theory, we are definitely able to explore the world with a purpose.

Observations and perceptions may be psychological, but observability is not.

The Logic of Scientific Discovery, pg. 103

Thus the real situation is quite different from the one visualized by the naïve empiricist, or a believer in inductive logic. He thinks that we begin by collecting and arranging our experiences, and so ascend the ladder of science. Or, to use the more formal mode of speech, that if we wish to build up a science, we have first to collect protocol sentences. But if I am ordered: ‘Record what you are now experiencing’ I shall hardly know how to obey this ambiguous order. Am I to report that I am writing; that I hear a bell ringing; a newsboy shouting; a loudspeaker droning; or am I to report, perhaps, that these noises irritate me? And even if the order could be obeyed: however rich a collection of statements might be assembled in this way, it could never add up to a science. A science needs points of view, and theoretical problems.

Ibid., pg. 106

I will note that one question Popper does not seem to consider is: “What makes a particular scientific pursuit interesting?” I’ve been struggling with this question in my own research, as I sift through a number of silos of work in machine learning and robotics, looking for both the big picture and the motivations behind each community effort. I do think it is possible to pose scientific questions that meet all the criteria of demarcation that Popper spells out, which fail as scientific questions simply because nobody else cares. Indeed, I think such “trivial” science actually comprises the near totality of posable scientific hypotheses. We don’t notice this because of a combination of our own bias and the natural selection bias that any peer reviewed scientific community uses as an organizing social and meritocratic engine.

People who now think Obama is insufficiently concerned with the Iranian people used to think we should drop bombs on them.

I’m not sure that one party in the dispute is particularly better from the point of view of American interests. From what I’ve read the opposition party is somewhat more moderate (whatever that even means in the context of a theocratic dictatorship). The fact that there are protests at all, and relatively non-violent protests (compared to the size of the demonstrations) as far as I can tell, seems like a good thing for Iranian liberals (whatever liberal means in Iran).

My point in making these observations is that all wonkish terms that we would normally use to describe the ebbs and flows of western style democracies don’t seem to fit this scenario, and reading the news analysis on this issue is more a study in failures of translation and the corresponding deconstruction of political language than an informative view into what is really going on.

Of course, given my lingering discomfort with religion, I’m sort of the last person who could hope to understand any side of the Iranian mindset.

ASIDE: Is it ironic that Iranians are relying on methods of communication that are a product of American entrepreneurial and technological ingenuity? Nevermind. This question sounds too much like “America is the best country on earth” and not enough like what I was trying to ask, which is whether we can conclude from Iranians’ use of things like Twitter whether they have a pro-western stance or not.