Automation & philosophy

John Ohno on 2018-10-10

Automation tools don’t obviate abstract discussions about how best to think about decisions (like ‘should translations be precise or should they be accurate’ or ‘what is the good’) but make them more important.

AI is a force-multiplier for philosophical positions. The programmer encodes their understanding of the world (or a program extracts a statistical model of the worldview of the guy who decided what goes in the data set), then the running program mass-produces the decisions made inevitable by that world-model. Since it’s harder for us to backtrack large-scale automated decisions, it’s very important to be critical about them before any automation occurs.

The paperclip-maximizer model gets a lot of hassle because making paperclips is obviously stupid. But if you replace ‘paperclips are good’ with ‘communication is good’ you get Twitter refusing to ban nazis. If you give it ‘money is good’ you get global neoliberal capitalism. People are really good at making generalizations — at substituting their actual goals with nearly-unrelated but easily-measurable proxy goals — but they’re not great at identifying when their proxy goals have diverged from their real goals, particularly when their measurements are changing rapidly. (Just look at Jim Jones’ trajectory: he allowed himself to become paranoid and myopic, and the end game was mass suicide — something that was neither inevitable nor anybody’s desired result.)

As a human being, you can make a mistake, recognize it in a vague and intuitive way, and change your behavior. When you replace yourself with a small shell script, that mistake can be made millions of times while you sleep. When you use statistical learning, you poorly-reproduce the System-I behavior of thousands of people who are worse than you are at the thing you’re reproducing & haven’t spent any time thinking about the subject. (It’s probably a bad idea, & should be applied only very carefully.)

As more and more ‘doing’ is performed by machines, careful philosophy becomes more important, because all ‘doing’ is a crystallization of philosophy.