Unintended consequences of AI safety advocacy argument against AI safety
I first heard this argument made by Michael Nielsen, that AI safety advocacy has increased the amount of work done on AI capabilities without actually increasing the safety of AI systems. In other words, it's a kind of differential progress argument, where advocacy about safety has the unintentional consequence of speeding up capabilities relative to safety.[1][2]
Buck also said in some EA forum post about how some people think MIRI screwed up by posting weird things on the internet in the early days.
there was also that Dario quote on Facebook.
In meme form: https://www.facebook.com/ozzie.gooen/posts/10165231537695363
See also
References
- ↑ https://twitter.com/michael_nielsen/status/1350549515762167808 "Curious: have any interventions yet clearly reduced AI risk?" "Afaict talking a lot about AI risk has clearly increased it quite a bit (many of the most talented people I know working on actual AI were influenced to by Bostrom.)" "Many of the most talented people I know working on building AGI seem to have gotten interested in part due to AI safety arguments. This seems likely (a) to have meaningfully accelerated progress toward AGI; but AFAICT (b) has done little to make such systems safer."
- ↑ https://80000hours.org/podcast/episodes/alexander-berger-improving-global-health-wellbeing-clear-direct-ways/ "So, Michael Nielsen is a quantum physicist who is popular on Twitter and is a friend of mine. I think he’s one of the smartest people I know. He has tweeted about how he thinks one of the biggest impacts of EA concerns with AI x-risk was to cause the creation of DeepMind and OpenAI, and to accelerate overall AI progress. I’m not saying that he’s necessarily right, and I’m not saying that that is clearly bad from an existential risk perspective, I’m just saying that strikes me as a way in which well-meaning increasing salience and awareness of risks could have turned out to be harmful in a way that has not been… I haven’t seen that get a lot of grappling or attention from the EA community."
