(2017-11-25) Hanson Why Be Contrarian

Robin Hanson: Why Be Contrarian? While I’m a contrarian in many ways, it think it fair to call my ex-co-blogger Eliezer Yudkowsky even more contrarian than I. And he has just published a book, Inadequate Equilibria, defending his contrarian stance, against what he calls “modesty”, illustrated in these three quotes:

I should expect a priori to be below average at half of things, and be 50% likely to be of below average talent overall; … to be mistaken about issues on which there is expert disagreement about half of the time. …

On most issues, the average opinion of humanity will be a better and less biased guide to the truth than my own judgment. …

We all ought to [avoid disagreeing with] each other as a matter of course. … You can’t trust the reasoning you use to think you’re more meta-rational than average.

In contrast, Yudkowsky claims that his book readers can realistically hope to become successfully contrarian in these 3 ways:

• 0-2 lifetime instances of answering “Yes” to “Can I substantially improve on my civilization’s current knowledge if I put years into the attempt?” …
• Once per year or thereabouts, an answer of “Yes” to “Can I generate a synthesis of existing correct contrarianism which will beat my current civilization’s next-best alternative, for just myself. …
• Many cases of trying to pick a previously existing side in a running dispute between experts, if you think that you can follow the object-level arguments reasonably well and there are strong meta-level cues that you can identify. … [This] is where you get the fuel for many small day-to-day decisions, and much of your ability to do larger things.

most of the book focuses on claim #2, that “for just myself” one might annually improve on the recommendation of our best official experts.

The main reason to accept #2 is that there exist what we economists call “agency costs” and other “market failures” that result in “inefficient equilibria” (which can also be called “inadequate”).

Yudkowsky gives some dramatic personal examples, but simpler examples can also make the point.

official medical advisors tend to advise medical treatment too often relative to doing nothing, official education experts tend to advise education too often as a career strategy, official investment advisors suggest active investment too often relative to index funds, and official religion experts advise religion too often relative to non-religion

To explain how inadequate are many of our equilibria, Yudkowsky contrasts them with our most adequate institution: competitive speculative financial markets, where it is kind of crazy to expect your beliefs to be much more accurate than are market prices.

Yudkowsky accepts my summary of the rationality of disagreement, which says that agents who are mutually aware of being meta-rational (i.e., trying to be accurate and getting how disagreement works) should not be able to foresee their disagreements

If you and a trusted peer don’t converge on identical beliefs once you have a full understanding of one another’s positions, at least one of you must be making some kind of mistake.

Yudkowsky says he has applied this result, in the sense that he’s learned to avoid disagreeing with two particular associates that he greatly respects. But he isn’t much inclined to apply this toward the other seven billion humans on Earth; his opinion of their meta-rationality seems low.

Now yes, the meta-rationality of some might be low, that of others might be high

Alas, Yudkowsky doesn’t offer empirical evidence that these possible clues of meta-rationality are in fact actually clues in practice, that some correctly apply these clues much more reliably than others

For many people, yes, an attempt to identify contrarian experts ends with them trusting faith healers over traditional medicine. But it’s still in the range of things that amateurs can do with a reasonable effort, if they’ve picked up on unusually good epistemology from one source or another.

Even so, Yudkowsky endorses anti-modesty for his book readers, who he sees as better than average, and also too underconfident on average

If you’re trying to do something unusually well (a common enough goal for ambitious scientists, entrepreneurs, and effective altruists), then this will often mean that you need to seek out the most neglected problems

modesty is especially detrimental for that kind of work, because it discourages acting on private information, making less-than-certain bets, and breaking new ground.

This seems to me to be a good reason to take a big anti-modest stance. If you are serious about trying hard to make a big advance somewhere, then you must get into the habit of questioning the usual accounts, and thinking through arguments for yourself in detail.

Perhaps you could do even better if you limited this habit to the topic areas near where you have a chance of making a big advance. But maybe that sort of mental separation is just too hard.

So far this discussion of disagreement and meta-rationality has drawn nothing from the previous discussion of inefficient institutions in a broken world

Yudkowsky doesn’t directly make a connection, but I can make some guesses. One possible connection applies if official experts tend to deny that they sit in inadequate equilibria, or that their claims and advice are compromised by such inadequacy.

note that if the reason you can beat on our best experts is that you can act directly, while they must win via social institutions, then this shouldn’t help much when you must also act via social institutions

Putting this all together my best guess is that Yudkowsky sees himself, his associates, and his best readers as only moderately smarter and more knowledgeable than others; what really distinguishes them is that they really care much more about the world and truth.

Those others are mainly trying to win the usual status games, while he and his associates are after truth.

Alas this is a familiar story from a great many sides in a great many disputes. Each says they are right because the others are less sincere and more selfish.

Eliezer Yudkowsky’s new book Inadequate Equilibria is really two disconnected books, one (larger) book that does an excellent job of explaining how individuals acting directly can often improve on the best advice of experts embedded in broken institutions, and another (smaller) book that largely fails to explain why one can realistically hope to consistently pick the correct side among disputing experts. I highly recommend the first book, even if one has to sometimes skim through the second book to get to it.