A New Utopian Activism is Born

Russel writes here as a politician, not an "expert" on Artificial Intelligence. In fact, one may ask if there is any such thing as "Artificial Intelligence," the entity that needs to be controlled.

Page 173 summarizes Russel's Utopia:

1. The machine's only objective is to maximize the realization of human preferences
2. The machine is initially uncertain about what those preferences are
3. The ultimate source of information about human preferences is human behavior

The assumption that underlies the book is that "AI" can be controlled to create a society that follows the rules above. The rules don't arise from "AI" itself. They are straightforward statements of political agenda.

Russel spends the rest of the book showing that all the above rules are not feasible in practice (let alone desirable). He misses the real issue with Rule #1, which calls for the optimization of an ordinal entity. This fallacy is so common it passes without notice. However, the mistake is hard to forgive for someone posing as an expert in logic.

The problem is so common I need to make a brief digression to illustrate the problem.

Suppose my friend and I fill in a questionnaire after we leave a restaurant. The form is full of questions of form (0) Disagree (1) Slightly Agree (3) Agree slightly (4) agree. So we are being asked, for example, if the meal was satisfying, what we expected, or if we were satisfied with the service.

Some computer program will "do the math" and conclude, for example, that "average" satisfaction with the service is 4.5. This is nonsense. The categories are not numbers.

For example, the "math" will involve adding my answers to my friend's answers. Does it make sense to "average" the happiness of two people? This would be made more obvious if the categories were A,B, C,D and not numbers. Nobody would be tempted to "average" A's and C's.

Averaging ratings is ubiquitous on the web. For example, here is the way Amazon treats reviews of this particular book:

What can this possibly mean?

First off, you can say something. For example, 71% rated the book with 5 stars. That's about it. All by itself, it doesn't say much that is useful. I always check the 1-stars.

Does this mean they liked the book better than the 18% that rated the book with only 4 stars? To answer this, you need to get inside the heads of the reviewers. Do 3-star reviewers like the book three times as much as the 1-star reviewers? Do the 5-star reviewers like the book five times as much as the one-star reviewers?

Wouldn't checking why the 1-star reviewers hated the book make sense? Often they are not happy with the binding or late delivery! Does it make sense to check the 5-star crowd to see if they actually read the book or considered the criticisms of the 1-star reviewers? Did the 5-star reviewers actually understand the book? 

This is actually a dense and difficult book. Have all the reviewers read it?

You can ask the same thing about a hypothetical survey of the human race to determine how "happy" we are, on average or on average. Such a survey would not produce meaningful results because you can't do math with ordinal numbers, which express a sequence (order) but say nothing about the degree of whatever is being talked about. For example, the runner coming in first did not run twice as fast as the runner coming in second.

By the way, the same problem arises in Sam Harris's famous "moral landscape." which creates a multi-dimensional fallacy out of ordinal rankings and assumes against all reason that there is an optimum way to achieve the "best" outcome for humanity or just one person. Again we have an example of a common fallacy: Appeal to Irrelevant Authority. Harris knows nothing about actual optimization.

Russel's agenda is dead on arrival. Astonishingly, he spends the rest of the book showing why the idea is unworkable, yet reaches the conclusion that somehow it will work out, and he will continue to be invited to political events where he can headline as an expert on the subject. In his impressive list of references, Russel fails to present a single voice of support for his core idea, but his qualifications on the book jacket list all the political entities he belongs to.

There are even more fundamental problems with the book:

• Introducing "AI" totally misses the point, which is "control" of the systems we create, with or without hardware;
• The concept of "AI" is a smoke screen to cover the fact that the issues of control, even when confined to "computer systems," have nothing to do with the specific technology of "AI."
• The "Russel Agenda" is a Utopian "solution" to a made-up problem. Russel specifically advocates for an international panel like the IPCC to address it. We know how that is going: Utopian advocacy replaces Science.

I will have more to say about these issues.