Corrigibility may be undesirable
From Issawiki (static archive)
- a point originally made by Wei Dai is that if an AI is corrigible to its human operators, then it may have to forgo certain kinds of trades with other AIs such as merging utility functions.