Soft-hard takeoff

a scenario i've been wondering about recently is where there is a stereotypical "soft takeoff" until around the point where the AI has somewhat-infra-human level general intelligence, and then once it crosses some threshold, a stereotypical "hard takeoff" happens.

the only places where i've found any discussion of anything similar to this:

• eric drexler says: """What I find extremely implausible are scenarios in which humanity confronts high-level AI without the prior emergence of potentially-strategically-decisive AI — that is, AI capabilities that are potentially decisive when employed by some group of ingenious, well-resourced human actors.

If we see something like “fast takeoff”, it is likely to occur in a world that is already far up the slope of a slow takeoff trajectory; if so, then many (though not all) of the key strategic considerations resemble those you’ve discussed in the context of slow-takeoff models.

The continued popularity of scenarios that posit fast takeoff with weak precursors is, I think, the result of a failure to update on the actual trajectory of AI development, or a failure of imagination in considering how intermediate levels of AI technology could be exploited."""

• In this gist, buck asks "It seems like there might be various ways that the world could be radically transformed by narrow AI before it's transformed by AGI; why don't people talk about that? (Maybe CAIS is people talking about this?)"

I thought this post about GPT-2 was pretty interesting: "But even this would be an important discovery – the discovery that huge swaths of what we consider most essential about language can be done “non-linguistically.” For every easy test that children pass and GPT-2 fails, there are hard tests GPT-2 passes which the scholars of 2001 would have thought far beyond the reach of any near-future machine. If this is the conclusion we’re drawing, it would imply a kind of paranoia about true linguistic ability, an insistence that one can do so much of it so well, can learn to write like spookily like Nabokov (or like me) given 12 books and 6 hours to chew on them … and yet still not be “the real thing,” not even a little bit. It would imply that there are language-like behaviors out there in logical space which aren’t language and which are nonetheless so much like it, non-trivially, beautifully, spine-chillingly like it." -- if many other things that humans can do are like language in this sense, we could start seeing crazy things happening in the world even though the AI "doesn't really understand" anything.

implications

let's look at paul's "Why does this matter?" section in his takeoff speeds post.

• "it will become quite obvious that AI is going to transform the world well before we kill ourselves" -- yes, this is true for soft-hard
• "we will have some time to experiment with different approaches to safety" -- not necessarily? AGI could still be very different from the narrow kinds of AI, so that the same safety tricks won't work. or AGI is just much harder to align (because it understands things or whatever).
• "policy-makers will have time to understand and respond to AI" -- i guess this is true
• "whoever develops AGI first has a massive advantage over the rest of the world and hence great freedom in choosing what to do with their invention" -- i think this is true
• "in slow takeoff scenarios, other actors will already have nearly-as-good-AGI, and a group that tries to use AGI in a very restricted or handicapped way won’t be able to take any pivotal action" -- not necessarily true; in soft-hard takeoff, AGI could still be much better than the narrow predecessors.