For supervised learning systems, serious concerns about “extortion” are essentially equivalent to concerns about simulations. These problems can’t really be resolved by better decision theory, they can only be resolved by pursuing unsupervised objectives or addressing concerns about simulations. Supervised learners As examples, consider either: a reinforcement learning system, trained to maximize the discounted sum […]
