Page
ReadView sourceView history

Soft-hard takeoff

From Issawiki (static archive)

("Soft-hard takeoff" is a pretty horrible name; maybe something like "continuous takeoff + FOOM/locality" is better)

a scenario i've been wondering about recently is where there is a stereotypical "soft takeoff" until around the point where the AI has somewhat-infra-human level general intelligence, and then once it crosses some threshold, a stereotypical "hard takeoff" happens.

the only places where i've found any discussion of anything similar to this:

If we see something like “fast takeoff”, it is likely to occur in a world that is already far up the slope of a slow takeoff trajectory; if so, then many (though not all) of the key strategic considerations resemble those you’ve discussed in the context of slow-takeoff models.
The continued popularity of scenarios that posit fast takeoff with weak precursors is, I think, the result of a failure to update on the actual trajectory of AI development, or a failure of imagination in considering how intermediate levels of AI technology could be exploited."""
"33. Weak AGI" in https://aiimpacts.org/relevant-pre-agi-possibilities/

I thought this post about GPT-2 was pretty interesting: "But even this would be an important discovery – the discovery that huge swaths of what we consider most essential about language can be done “non-linguistically.” For every easy test that children pass and GPT-2 fails, there are hard tests GPT-2 passes which the scholars of 2001 would have thought far beyond the reach of any near-future machine. If this is the conclusion we’re drawing, it would imply a kind of paranoia about true linguistic ability, an insistence that one can do so much of it so well, can learn to write like spookily like Nabokov (or like me) given 12 books and 6 hours to chew on them … and yet still not be “the real thing,” not even a little bit. It would imply that there are language-like behaviors out there in logical space which aren’t language and which are nonetheless so much like it, non-trivially, beautifully, spine-chillingly like it." -- if many other things that humans can do are like language in this sense, we could start seeing crazy things happening in the world even though the AI "doesn't really understand" anything.

"Continuous takeoff is a statement about what happens before we reach the point where a fast takeoff is supposed to happen, and is perfectly consistent with the claim that given the stated preconditions for fast takeoff, fast takeoff will happen. It’s a statement that serious problems, possibly serious enough to pose an existential threat, will show up before the window where we expect fast takeoff scenarios to occur." [3] -- i guess this is the same sort of thing as what i am describing here. if so, then i agree that talking about "soft" vs "hard" takeoff is pretty misleading, and we should be talking about "shape prior to takeoff" (business as usual vs crazy shit happens), and then separately about the speed of takeoff (where it doesn't seem like paul and MIRI even disagree?). Also if this is the case, i don't see how anything eliezer writes actually contradicts what paul writes. like, i think eliezer is consistently only talking about the speed of takeoff, not the shape of progress prior to takeoff?

so what is paul's argument that once we reach a "takeoff level capability", we won't get a unipolar/decisive strategic advantage outcome? i think it's something like: because the runner-ups will have almost-as-capable AI, they can fight back somewhat effectively, and in a short time they will reach the current winner's capability (even while the current winner will be even ahead by then). it has to be something like this, since if we use the counterfactual in [4] where a seed-level AI is suddenly dropped into today's world, i think even paul would say there would be a sudden unipolar outcome. See counterfactual of dropping a seed AI into a world without other capable AI.

https://www.facebook.com/yudkowsky/posts/10154597918309228

implications

let's look at paul's "Why does this matter?" section in his takeoff speeds post.

See also

  • ↑ https://www.greaterwrong.com/posts/aFaKhG86tTrKvtAnT/against-gdp-as-a-metric-for-timelines-and-takeoff-speeds