Short-term preferences-on-reflection

From Issawiki
Revision as of 23:00, 26 August 2020 by Issa (talk | contribs) (Created page with "i need to summarize this at some point. https://www.greaterwrong.com/posts/nRAMpjnb6Z4Qv3imF/the-strategy-stealing-assumption#comment-ERmqHuN3ZWziS4SGH is this right? "Detai...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

i need to summarize this at some point.

https://www.greaterwrong.com/posts/nRAMpjnb6Z4Qv3imF/the-strategy-stealing-assumption#comment-ERmqHuN3ZWziS4SGH

is this right? "Details: “Short-term preferences” does not mean “preferences over what can be achieved in the short-term” but “things that I would still endorse in the short-term”: An AI system that is optimizing for short-term preferences could still do long-term planning and act accordingly, but it does so because that’s what the human wants it to do at that moment. If we changed our mind, the AI would also change action. It would not take actions that we oppose even if 10 years later we would think it would have been good in hindsight. The logic here is that the user always knows better than the AI." [1]