Causal entropy maximization and intelligence

CEM

Taken from Causal Entropic Forces

Recently I was referred to a paper titled Causal Entropic Forces published in Physical Review Letters that attempts to link intelligence and entropy maximization. You can find reviews of this paper here and here. The paper starts with

Recent advances in fields ranging from cosmology to computer science have hinted at a possible deep connection between intelligence and entropy maximization….In this Letter, we explicitly propose a first step toward such a relationship in the form of a causal generalization of entropic forces that we show can spontaneously induce remarkably sophisticated behaviors associated with the human ‘‘cognitive niche,’’ including tool use and social cooperation, in simple physical systems.

The authors then go on to define a causal path version of entropy. Briefly, this is a generalization from standard entropy, a measure of how many states a system can be in at a specific point in time, to causal path entropy, a measure of how many paths that system can follow during a given time horizon. In technical language, microstates are mapped to paths in configuration space, and macrostates are mapped to configuration space volumes:

In particular, we can promote microstates from instantaneous configurations to fixed-duration paths through configuration space while still partitioning such microstates into macrostates according to the initial coordinate of each path

In other words, an initial coordinate establishes a volume in configuration space which represents possible future histories starting at that point. This is the macrostate (depicted as a cone in the image above)

Having defined this version of entropy, the authors then add the condition of entropy maximization to their model; this is what they call causal entropic forcing. For this to have a net effect, some macrostates have volumes which are partially blocked off for physical reasons. Consequently these macrostates have less available future paths, and less causal path entropy. The result is that different macrostates with different entropies can be differentially favored by condition of causal entropy maximization:

there is an environmentally imposed excluded path-space volume that breaks translational symmetry, resulting in a causal entropic force F directed away from the excluded volume.

Note that, contrary to actual thermodynamical systems that naturally exhibit entropy maximization for statistical reasons, causal entropic forcing is not physical, it is a thermodynamics inspired premise the authors add to their model as a “what if” condition, to see what behaviour results. So, what happens when systems are subject to causal entropic forcing?

 we simulated its effect on the evolution of the causal macrostates of a variety of simple mechanical systems: (i) a particle in a box, (ii) a cart and pole system, (iii) a tool use puzzle, and (iv) a social cooperation puzzle…The latter two systems were selected because they isolate major behavioral capabilities associated with the human ‘‘cognitive niche’’

Before you get excited, the “tool use puzzle” and “social cooperation puzzle” are not what one would imagine. They are simple “toy simulations” that can be interpreted as tool use and social cooperation. In any case, the result was surprising. When running these simulation the authors observed adaptive behaviour that was remarkably sophisticated given the simplicity of the physics model it emerged from. What’s more, not only was the behaviour adaptive, but it exhibited a degree of generality; the same basic model was applied to both examples without specific tuning.

The remarkable spontaneous emergence of these sophisticated behaviors from such a simple physical process suggests that causal entropic forces might be used as the basis for a general—and potentially universal—thermodynamic model for adaptive behavior.

How does this fit in with intelligence?

I see two different ways one can think about this new approach. One, as an independent definition of intelligence from very simple physical principles. Two, in terms of existing definitions of intelligence, seeing where it fits in and if it can be shown to be equivalent or recovered partially.

Defining intelligence as causal entropy maximization (CEM) is a very appealing as it only requires a few very basic physical principles to work. In this sense it is a very powerful concept. But as all definitions it is neither right nor wrong, its merit rests on how useful it is. The question is thus how well does this version of intelligence capture our intutions about the concept, and how well it fits with existing phenomena that we currently classify as intelligent[1]. Ill consider a simple example to suggest that intelligence defined this way cannot be the entire picture.

That example is unsurprisingly life, the cradle of intelligence. The concept that directly collides with intelligence defined as CEM is negentropy. Organisms behave adaptively to keep their biological systems within the narrow space of parameters that is compatible with life. We would call this adaptive behaviour intelligent, and yet its effect is precisely that of reducing entropy. Indeed, maximizing causal entropy for a living being means one thing, death.

One could argue that the system is not just the living organism, but the living organism plus its environment, and that in that case the entropy perhaps would be maximized[2]. This could resolve the apparent incompatibility, but CEM still seems unsatisfying. How can a good definition of intelligence leave out an essential aspect of intelligent life: the entropy minimization that all living beings must carry out. Is this local entropy minimization implicit in the overall causal entropy maximization?

CEM, intelligence and goals

Although there is no single correct existing definition of intelligence, it can be said that current working definitions share certain common features. Citing [3]

If we scan through the definitions pulling out commonly occurring features we find that intelligence is:

• A property that an individual agent has as it interacts with its environment or environments.

• Is related to the agent’s ability to succeed or profit with respect to some goal or objective.

• Depends on how able to agent is to adapt to different objectives and environments.

In particular, intelligence is related to the ability to achieve goals. One of the appealing characteristics of CEM as defining intelligence is that it does not need to define goals explicitly. In the simulations carried out by the authors the resulting behaviour seemed to be directed at achieving some goal that was not specified by the experimenters. It could be said that the goals emerged spontaneously from CEM.  But it remains to be seen whether this goal directed behaviour results automatically in real complex environments. For the example of life I mentioned above, it looks to be just the opposite.

So in general, how does CEM fit in with existing frameworks where intelligence is the ability to achieve goals in a wide range of environments[3]? Again, I see two possibilities:

  • CEM is a very general heuristic[4] that aligns with standard intelligence when there is uncertainty in the utility of different courses of action
  • CEM can be shown to be equivalent if there exists an encoding that represents specific goals via blocked off regions in configuration space (macrostates)

The idea behind the first possibility is very simple. If an agent is faced with many possibilities where it is unclear which one will lead to achieving its goals, then maximizing expected utility would seek to follow courses of action that allow it to react adaptively and flexibly when more information becomes available. This heuristic is just a version of keep your options open.

The second idea is just a matter of realizing that CEM’s resulting behaviour depends on how you count possible paths to determine the causal entropy of a macrostate. If one were to rule out paths that result in low utility given certain goals, then CEM could turn out to be equivalent to existing approaches to intelligence. Is it possible to recover intelligent goal directed behaviour as an instance of CEM given the right configuration space restrictions?


References

[1] Our intuitions about intelligence exist prior to any technical definition. For example, we would agree that a monkey is more intelligent than a rock, and that a person is more intelligent than a fly. A definition that does not fit these basic notions would be unsatisfactory.

[2] http://prd.aps.org/abstract/PRD/v76/i4/e043513

[3] http://www.vetta.org/documents/A-Collection-of-Definitions-of-Intelligence.pdf

[4] This seems related to the idea of basic AI drives identified by Omohundro in his paper http://selfawaresystems.files.wordpress.com/2008/01/ai_drives_final.pdf. In particular 6. AIs will want to acquire resources and use them efficiently. Availability of resources translates to the ability to follow more paths in configuration space, paths that would be unavailable otherwise.

14 thoughts on “Causal entropy maximization and intelligence

  1. …”Indeed, maximizing causal entropy for a living being means one thing, death.”…
    Is that really true?
    Looking at the simplest example in the paper, the particle in a box, shows, i.e., that the particle is drawn into the center of the box under causal entropic forces instead of exploring the whole box under thermodynamic entropy maximization. Or the balancing of a stick on a cart example could be interpreted as tuning certain parameters to a certain (unstable) value (suitable for life).

    1. Thanks for your comment, Daniel.

      Yes you can interpret the stick balancing as tuning parameters for life, but the analogy is not enough to see what actually happen. The problem with the examples is that they are not complex enough to replicate real situations, only for interpretations. My view is that there are more paths traversing areas of death than areas of life, simply because there are more ways of being dead than alive. If there are more such paths, it seems to me that this forcing would push towards these areas. But I may be wrong, as this is also an intuition.

    1. Thank you for those links, I will be looking at them.

      (PS I have not had time to approve your comments until now, my apologies for the delay)

  2. Hey i wanted to say something about this. Probably you would think i´m a bit crazy but well, doesn´t matter. Is about eploration/explotation and it relation to psychosis like states and to socio-economic policies. The main idea in this articles about psychosis is that there is an optimism bias or augmented precision of sensory information (what means that there is a reduced precisión of prior beliefs). And that this leads to false inferences. I think this is an extremely conductual explanation, because they are assuming that the psychotic is wanting to infere something that he could not be wanting to infer. I see it more like a tendency towards exploration, because of the reduced precisión of prior beliefs he will reject the null hipótesis whit less evidence and he will infer other thing (waht they call a false inference or hallucination). And he will infer something that don´t lead towards an outcome with the same precisión that if he were taking into account (or giving more precisión) to the prior beliefs. That means to me that he is doing that more for the pleasure of doing it (exploration or intrinsic motivation) tan because he thinks he will reach a concrete outcome that maximizes the expected utility.

    Related to this i wanted to mention this article about exploration, explotation and entropy. It says that in a more restricted ambient (or where the expected utility of different actions is well defined and differentiated) the best strategies (or the fitting strategies) would be explotation, but in a less restricted, there would be much more ways of fitting, so the more explorative behaviors would be adaptatively to. What i think after seeing this is something i had tought a lot of times, that one of the biggest problems for crazy people is to live in a society where you have to use explotation strategies for reaching near the optimal and more adapted way of living.

    http://zimmer.csufresno.edu/~shliu/pub/Bioma06-Mernik.pdf

  3. I would like to say that in fact the difference beteen intrinsic and extrinsic motivation (or between exploration and explotation) is no t that you do it for the pleasure or not: Actually all we do is for the pleasure of doing it. The difference is that in explotation the pleasure is in controlling the outcomes in a way more inmediately. And in exploration the pleasure. is in knowing new or other links between action and perception.

    The thing is that when the slection impose hard restrictions, the more pleasurable behaviors would be the exploiting ones. And that there are some biological and psychosocial constraints that led some people to prefer explorative behaviors. Having noticed that the exploration is not an optimal behavior only when there is high selection pressure (or low entropy), that is not so crazy to think that we are doing something wrong when the adpative behavior in our society is an high restricted one (or you exploit your enviroment for having money for food and home and…or…). Even from an utilitarian view it is good to have people who tend to explórate (but the fact is that in our society this is a very bad strategy…).

  4. I have read the presentation you have linked to by Merni et al. and I have to say it makes a lot of sense. Entropy in the gene pool is a tunable parameter that can be used as an operational measure of how much the environment is expected to change, such that exploration becomes a better investment. Good stuff, thanks

  5. Thanks for the answers (but still i don’t know how my comments were erased, i suppose you have some automatic mechanism or else). I see you are including in the enviroment the agents too, this is a goood viewpoint i think. More entropy in the gene pool would mean a less restricted selection, so exploration would become a better investment.

    What i wanted to say is that explotation is a tendency toward improving some known model, sampling more inputs-action conjoint distributions in the same way, and exploration towards generating new models i think (sampling perceptions and actions whose conjoint distributions are not well known). About the questión of the null hypothesis, i think rejecting it means that you assume that the sensory input does not fit any known model (because assumig the other does not improve your model) and you try to build a new model with other conjoint distributions.

    This comes for the discussion in “a bordo del otto neurath” (the entry “un mundo sin proyecto”) if you don´t remember. There i mentioned the article “the computational anatomy of psychosis” (that i didn´t know at all and i find after thinking about your slides about rationality and connecting with other concepts, there i treated too in a very informal way this entry and the other you have about art, novelty and dopamine). Sorry for my english i´m trying to improve.

    1. Thanks for your comment, Sergio, I have briefly looked at your blog, and I find your kart driving examples very cool.

      I am not disputing that CEM can be a powerful guiding heuristic or principle, I have stated outright that I believe. What I do dispute is rather the notion that intelligence can be _reduced_ to CEM and altogether replace it. Note the following comment and how it seems quite relevant to your examples:

      “CEM can be shown to be equivalent if there exists an encoding that represents specific goals via blocked off regions in configuration space (macrostates)”

      Note also how you end up reintroducing the idea of goals in “Introducing motivations”, which is a crucial ingredient in standard definitions of intelligence. It just makes perfect sense for you to do this.

      Regards,

      David

  6. Very late to the discussion here, but in case there’s still interest:

    I think what makes “maximizing causal entropy for a living being means one thing, death” untrue is that if we’re focusing only on the alive/dead state it’s possible to go from alive to dead at any time, but not possible (okay.. infinatessimally probable) to go from dead to alive. So in most cases there are more future histories possible by delaying death, because you still have the ability to go from alive to dead whenever. Now, that’s not to say there aren’t some edge cases where there are more possible future histories in which one dies earlier. But I think the original comment may have been conflating vanilla “entropy” with the “causal path entropy” of Wissner-Gross.

    1. Thanks for your comment David B.

      I will reply with a previous comment I made in response to Daniel:

      “My view is that there are more paths traversing areas of death than areas of life, simply because there are more ways of being dead than alive. If there are more such paths, it seems to me that this forcing would push towards these areas. But I may be wrong, as this is also an intuition.”

      In other words, I am not confusing entropy with causal entropy. I am saying: because more paths go through death, there is more causal entropy for actions that go in that direction. I suspect our disagreement may depend on the time horizon for which you count paths, and I think we’d have to run experiments to go beyond our intuitions.

Leave a Reply

Your email address will not be published. Required fields are marked *