## Simplicity and meta-theoretic induction

During my discussion of induction and occams razor, I have said

The key is that our intuition, in fact our intelligence in general, has a built-in simplicity bias. We strongly favor the odd number theory because it is the simplest theory that fits the facts. Hence induction, including our everyday intuitions and the scientific method, is founded upon Occam’s razor as a way to discriminate between equally supported theories.

Note that I have described Occam as a simplicity bias. I deliberately chose this word in order to convey that simplicity is a guideline that we use prior and independent of experience. In the language of probability, Occam takes the form of a prior probability distribution that favors simpler theories before any updating has occured, and is unaffected by posterior evidence.

This state of affairs does not seem satisfactory; as described above, Occam seems like an unjustified and arbitrary principle, in effect, an unsupported bias. Surely, there should be some way to anchor this widely applicable principle on something other than arbitrary choice[1]. The obvious course would be to say something like this:

Simpler theories are more likely to be true because they have been so in the past

thus grounding the principle on experience, just like any other case of inductive inference and scientific knowledge. But let’s go back to the source of the problem that this principle is trying to fix. We have two theories, S and C, which make identical, correct predictions with respect to all observations made to date. These two theories only differ in their future predictions.

And yet, in practice, we consider the predictions made by S much more likely than those made by C. Because by definition these two theories share likelihoods for observed evidence, it is only through their priors that we can assign them different probabilities. Here’s where the simplicity principle comes in. We favour theory S because it is simple, hence granting it a greater prior probability and consequently a greater posterior despite its shared likelihood with C. When asking ourselves how we justify the simplcity principle, we answer

Because simple theories have been true in the past.

So the simplicity principle acts like a meta-theory and can accrue probability through experience just like any other theory. Until now, everything seems to work, but here’s the problem. Let’s say we have two guiding principles:

Occam: Simpler theories are more likely to be true

Occam(t): Simpler theories are more likely to be true until time t

Whatever the mechanism by which the simplicity meta-theory accumulates posterior probability, so shall its peculiar brother, and in the same exact amount. When going back to our two theories, S and C, Occam will favour S while Occam(t) will favour C. Because both Occam and Occam(t) are supported by the same amount of evidence, equal priors will be assigned to S and C. The only way out of this is for Occam and Occam(t) to have different priors themselves. But this leaves us back where we started!

So in conclusion, if we try to solve the problem with

Simpler theories are more likely to be true because they have been so in the past

we are just recasting the original problem at the meta level, we end up begging the question[2] or in an infinite regress. Which should not really come as a surprise, there is no way to justify knowledge absolutely, there must be unjustified assumptions somewhere. In my point of view, Occam is such a bedrock

The way I see it, the foundations of scientific knowledge are the postulates of probability theory (as derived for example by Bernardo-Smith or Cox) together with Occam’s razor.

Please see my next post where I formalize meta-theoretic induction

[1] In essence, this is the problem of induction, which dates back to Hume who first posed it in its original form.

[2] The future is like the past because in the past, the future was like the past

## Time to AI, anyone’s guess?

When somebody asks me how long until AI is developed, I say between two and four decades. The question as posed refers to artificial intelligence that is competent at a variety of tasks that humans can carry out. Competent means not obviously distinguishable from a human’s performance.  So even though the question is not technically precise, it  is reasonably well defined, because human standards of performance are implicit in the question.

Now, getting back to the question, my usual answer as I said is two to four decades. But this brings me to this post I read recently, which tries to shed light on the matter by surveying and analyzing a large number of AI predictions, both from experts and non-experts.  The first conclusion, which I quote from the summary is

Over a third of predictors claim AI will happen 16-25 years in the future.

But I find the following two results of special significance

There is little difference between experts and non-experts (some possible reasons for this can be found here).
There is little difference between current predictions, and those known to have been wrong previously.

This paints a pretty bleak picture as to our ability to predict the development of AI. If experts can’t be distiguished from non-experts, it means that for the purposes of predicting AI, there really exist no experts at all, and uninformed guesses are as good as anything.

Along the same lines goes the second point I quoted. If current predictions are similar to previous ones that we know were flawed (because they were completely wrong), this suggests current ones could also be wrong. More specifically, if predictions made at different times, with significantly different contexts (ie scientific and technological progress) yielded the same predictions, it suggests that the predictions are independent of concrete scientific and technological considerations; they are based on non-specific reasoning, or even worse, unrelated factors (eg psychological) that are common to both periods.

Lastly, there is another point to be made about expert predictions. These experts are by definition active in the field of AI research, and have a stake (eg funding) in what is believed will happen. Plus, if they’re working towards AI, presumably they want their predictions to be true, it is their goal as researchers.

Where does this leave “two to four decades”? It’s not an expert prediction, although my intuition for this number is certainly based on expert opinions I’ve come accross over the years. In theory, reconsidering with the extra information should change my mind. Rationality dictates that one should widen the prediction interval in order to reflect the extra uncertainty that this new information introduces. The widening could be asymmetrical, because ignorance about how to achieve something points to that something being difficult rather than easy, so one could say something like “15 to 50 years”.

Perhaps this meta level correction is not incompatible with my original, object level intuitions: Advances in machine learning, neuroscience and computing power together with growth trends in funding suggest to me that arrival of AI in 30 years is quite likely, and that in 40 years it is beginning to approach certainty. But one has to admit, these are just intuitions and handwaving, we simply do not seem to have the tools to make predictions in a technical way.

Prediction is very difficult, especially if it’s about the future.
– Niels Bohr

## “Human level AI” is a misleading concept

I’ve used and will probably continue to use the term human level AI to refer to a level of machine intelligence that is competent enough to carry out tasks that humans carry out. In particular, capable of carrying out those tasks that we consider evidence for intelligence when done by a human. But I’ve just realised that it’s a misleading term. Here’s why.

Let’s try to make the notion of level more precise. We have the universal intelligence measure, as well as AIQ, as precise mathematical constructs for this idea. If you recall, these two constructs are based around the idea of summing the number of environments at which an agent succeeds in as an indicator of intelligence. This sum is weighted by simplicity. Agents that are competent in a higher number of environments are said to be more intelligent, and the simplicity bias favors those agents that do so by virtue of a general capacity, rather than a sum of narrow specialisations.

Anyway, the point is that beyond the simplicity bias, such a notion of level does not require  success at any given environment, but rather it is the sum that counts, not which environments take part in that sum. Hence, two agents of equal intelligence could be competent in a very different set of environments.

Therefore, a human level AI could correspond to any agent whose sum of environments is equal to that of a human. But these environments could be widely different. This means that a human level AI could easily be incompetent at tasks that humans carry out. And this is precisely the original, intuitive meaning of human level AI. Human level AI is therefore misleading, because intuitively it means one thing, but precisely means something different.

What human level AI really means is artificial human intelligence, not just the level of intelligence, but also a specification of which environments the intelligence must succeed at. It’s the usual anthropocentric bias that incorrectly discards the generality of the level specification.

Hence we should really say artificial human intelligence, although I’d understand accusations of nitpicking!

## Morals and the machine

It’s the title of an article on The Economist regarding machines and morality, the sub-heading reads

As robots grow more autonomous, society needs to develop rules to manage them

Seems pretty reasonable, except for the use of the term ‘rule’, which I’ll get to in a minute. After framing the problem, the author proposes the following agenda

First, laws are needed to determine whether the designer, the programmer, the manufacturer or the operator is at fault if an autonomous drone strike goes wrong or a driverless car has an accident…

Second, where ethical systems are embedded into robots, the judgments they make need to be ones that seem right to most people….

Last, and most important, more collaboration is required between engineers, ethicists, lawyers and policymakers, all of whom would draw up very different types of rules if they were left to their own devices.

Starting with the last point, like any legislation it should be something that concerns all of society, and it is well noted that it’s a very cross-disciplinary matter, hence experts from different fields will be required.

The first point is also important, but it makes an assumption that dodges a central problem. The assumption is that responsibility always lies outside of the “robot”,  and it’s a matter of deciding which of the “creators” is to blame and how. But will there be a moment when the moral agency is transferred away from the creators to the robot? In the short term the answer is clearly no, except for science fiction. But if we’re speaking of increased autonomy..

Lastly, I find the most interesting and difficult point to be the second one. It’s one of those cases where a very deep problem can be stated in a deceptively small number of words.

..the judgments they make need to be ones that seem right to most people..

In fact, the trouble comes from one word, right, which carries a huge amount of hidden complexity. We intuitively know what is meant here, but just try formalizing this into something explicit that can be programmed. Good luck. That’s why I remarked about the use of the word ‘rules’ above, I’m very skeptical that right can be translated into programmable rules.

Rather it seems more likely that any sophisticated morality that a robot follows will include an element of learning, similarly[1] to how children learn what’s right and what’s wrong. The problem with this approach is noted in the article on machine ethics in wikipedia, that sometimes learned representations (i.e. how the knowledge is encoded in the robot) are hard or impossible to understand, inspect, debug and correct.

But it’s not impossible, it would be pretty ironic if formalizing human morality and the meaning of right finally came about by looking into the “brain” of something that wasn’t human!

[1] There is a danger of anthropomorphizing artificial intelligences and making hidden and unwarranted assumptions that may not be applicable to machines. But my point here is on using learning rather than programming, not on the specifics.