Difference between revisions of "Future planning"

From Issawiki
Jump to: navigation, search
 
(23 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
things to talk about:
 
−
 
 
* the most decision-relevant questions for me right now (everything else should feed into one of these questions):
 
* the most decision-relevant questions for me right now (everything else should feed into one of these questions):
 
** AI safety vs something else? right now AI safety seems like the best candidate for the biggest/soonest change, but i want to investigate some other things.
 
** AI safety vs something else? right now AI safety seems like the best candidate for the biggest/soonest change, but i want to investigate some other things.
−
** if AI safety, then what technical agenda seems best? this matters for (1) deciding what to do technical research on, if at all; (2) what technical research to follow/promote/give money to.
+
** if AI safety, then what technical agenda seems best? this matters for (1) deciding what to do technical research on, if at all; (2) what technical research to pay attention to/support.
−
** if AI safety, then what will the end of the world look like? (this matters for prepping)
+
** if AI safety, then what will the end of the world look like (basically this is takeoff speeds + what the most likely failure mode looks like)? (this matters for [[AI prepping|prepping]])
 
** how likely is the end of the world?
 
** how likely is the end of the world?
 
** when will AGI come?
 
** when will AGI come?
−
* how doomed ML safety approaches are e.g. see discussion [https://www.greaterwrong.com/posts/suxvE2ddnYMPJN9HD/realism-about-rationality#comment-32dCL2u6p8L8td9BA here] -- [[How doomed are ML safety approaches?]]
+
 
−
** there's the sort of opposite question of, how doomed is MIRI's approach? i.e. if there turns out to be no [[simple core algorithm for agency]], or if understanding agency better doesn't help us build an AGI, then we might not be in a better place wrt aligning AI.
+
Example scenarios:
−
* can MIRI-type research be done in time to help with AGI? see [https://www.greaterwrong.com/posts/suxvE2ddnYMPJN9HD/realism-about-rationality#comment-Dk5LmWMEL55ufkTB5 this comment]
+
 
−
* prior on difficulty of alignment, and ideas like "if ML-based safety were to have any shot at working, wouldn't we just go all the way and expect the default (no EA intervention) approach to AGI to just produce basically ok outcomes?"
+
{| class="wikitable"
−
* list of things people disagree about:
+
! Name of scenario !! Next big thing? !! AI safety technical agenda !! What will end of the world look like? !! How likely is the world to end? !! When will the world end/reach a singularity/cure aging? || What I should do
−
** probability of doom
+
|-
−
** civilizational adequacy
+
| Eliezerian end of world || AGI || MIRI? || FOOM/AI takeover || Highly likely? || 10-20 years? || Don't have kids. Probably don't even bother with wife search. Focus on AI safety research, either field building or figuring out how to contribute to MIRI-like technical work.
−
** probability of doom without any special EA intervention
+
|-
−
** shape of takeoff
+
| Eliezerian end of world, but longer timelines || AGI || MIRI? || FOOM/AI takeover || Highly likely? || 40 years ||
−
*** [[Will there be significant changes to the world prior to some critical AI capability threshold being reached?]]
+
|-
−
** what precursors/narrow systems we will see prior to AGI
+
| Eliezerian end of world, but even longer timelines || AGI || MIRI? || FOOM/AI takeover || Highly likely? || 60 years ||
−
** AI timelines
+
|-
−
** what the first AGI will look like
+
| Age of em || WBE ||
−
** how big of a problem collusion between subsystems of an AI will be
+
|-
−
** how likely optimization daemons are or what they will look like
+
| Paul's end of world || AGI || ? || || || 20 years? || [[AI prepping]]???
−
** whether there is a basic of attraction for corrigibility
+
|-
−
** whether an AGI will look like a utility maximizer?
+
| CAIS
−
** something-like-realism-about-rationality
+
|-
−
** whether MIRI-type work can be done in time
+
| GPT-3-like limited AI causes big changes in the world
−
** whether ML-based approaches are doomed
+
|-
−
** whether "weird recursions" are a good idea
+
| Great Stagnation (world stays basically the same for the next 100 years) ||
−
** whether we can correct mistakes when deploying AI systems as they come up (i.e. how catastrophic the initial problems will be)
+
|-
−
** how many/how "lumpy" insights are for creating an AGI
+
| Super long AI timelines (AI won't come for 100 years) ||
−
*** "the degree of complexity of useful combination, and the degree to which a simple general architecture search and generation process can find such useful combinations for particular tasks" [https://www.facebook.com/yudkowsky/posts/10155848910529228?comment_id=10155848951264228&reply_comment_id=10155849315174228]
+
|}
−
** how much sharing/trading there will be between different AI companies (eliezer vs [[Robin Hanson]]) -- this one is downstream of lumpiness of insights, because hanson expects that if there are very few insights needed to get to AGI, then there won't be any need for sharing (so in that case even hanson would agree with eliezer).
 
−
** how important it is to get the right architecture e.g. "That is what I meant by suggesting that architecture isn’t the key to AGI." [https://www.greaterwrong.com/posts/D3NspiH2nhKA6B2PE/what-evidence-is-alphago-zero-re-agi-complexity]. There is [[Dario Amodei]]'s comment [https://www.facebook.com/yudkowsky/posts/10155848910529228?comment_id=10155849004324228&reply_comment_id=10155849068769228 here] which is the opposite view.
 
−
** is it possible to turn a small lead in AGI development into a big lead?
 

Latest revision as of 18:53, 7 September 2020

  • the most decision-relevant questions for me right now (everything else should feed into one of these questions):
    • AI safety vs something else? right now AI safety seems like the best candidate for the biggest/soonest change, but i want to investigate some other things.
    • if AI safety, then what technical agenda seems best? this matters for (1) deciding what to do technical research on, if at all; (2) what technical research to pay attention to/support.
    • if AI safety, then what will the end of the world look like (basically this is takeoff speeds + what the most likely failure mode looks like)? (this matters for prepping)
    • how likely is the end of the world?
    • when will AGI come?

Example scenarios:

Name of scenario Next big thing? AI safety technical agenda What will end of the world look like? How likely is the world to end? When will the world end/reach a singularity/cure aging? What I should do
Eliezerian end of world AGI MIRI? FOOM/AI takeover Highly likely? 10-20 years? Don't have kids. Probably don't even bother with wife search. Focus on AI safety research, either field building or figuring out how to contribute to MIRI-like technical work.
Eliezerian end of world, but longer timelines AGI MIRI? FOOM/AI takeover Highly likely? 40 years
Eliezerian end of world, but even longer timelines AGI MIRI? FOOM/AI takeover Highly likely? 60 years
Age of em WBE
Paul's end of world AGI ? 20 years? AI prepping???
CAIS
GPT-3-like limited AI causes big changes in the world
Great Stagnation (world stays basically the same for the next 100 years)
Super long AI timelines (AI won't come for 100 years)
Retrieved from "https://wiki.issarice.com/index.php?title=Future_planning&oldid=1601"