depth first search

Today is full of minor little administrative details that need attention and always seem to correlate with 1) the beginning of the classes and 2) returning to Austin after having been away all summer. The end result of all of this is that my mind is all over the place right now, and not nearly in the necessary shape to work on revising my proposal. So instead I’ll leave you with one more comically vague question to ponder.

What is knowledge?

My wife and I were discussing something similar (specifically, whether knowledge could by copied or transferred) while driving home from Ann Arbor. I found out that navigating Dallas traffic was not conducive to having a good technical discussion. In any event,  to discuss how knowledge can be transferred, we would seem to require some basic understanding of what knowledge actually is.

As before, it’s probably a good idea to consider a specific example from a particular field of study. In reinforcement learning, the goal is to have agents learn policies, or ways of choosing how to act, that are somehow “optimal.” One way of doing this is to learn something called a value function, which roughly captures the intuitive notion of how being in particular states would benefit an agent over time. To someone studying reinforcement learning then, value functions reflect a certain kind of very important knowledge about how to act in the world. In addition, people have studied how to transfer the kinds of knowledge that are wrapped up in value functions to new agents in new domains. This area is known as transfer learning.

There is even a belief out there that all of artificial intelligence falls under the rubric of learning value functions. This is known as “the reward hypothesis” or “the value function hypothesis.” You can find a reasonably concise statement of that view here. I don’t really believe in any form of the reward hypothesis. Human intelligence does not seem confined by the narrow constraints that this particular view would necessarily imply. For instance, language acquisition and use almost certainly cannot be satisfactorily explained via some form of adaptive optimal control. If you don’t believe me maybe you’ll believe Noam Chomsky.

•