Narrow window argument against continuous takeoff - Revision history

Issa at 16:56, 24 June 2020

2020-06-24T16:56:56Z

Issa at 16:41, 24 June 2020

2020-06-24T16:41:09Z

Issa at 23:36, 23 June 2020

2020-06-23T23:36:46Z

Issa at 23:36, 23 June 2020

2020-06-23T23:36:30Z

Issa at 19:12, 27 May 2020

2020-05-27T19:12:21Z

Issa at 05:08, 22 May 2020

2020-05-22T05:08:10Z

Issa at 03:50, 22 May 2020

2020-05-22T03:50:41Z

Issa at 03:49, 22 May 2020

2020-05-22T03:49:40Z

Issa: Created page with " "When you fold a whole chain of differential equations in on itself like this, it should either peter out rapidly as improvements fail to yield further improvements, or else..."

2020-05-22T03:44:55Z

Created page with " "When you fold a whole chain of differential equations in on itself like this, it should either peter out rapidly as improvements fail to yield further improvements, or else..."

New page

"When you fold a whole chain of differential equations in on itself like this, it should either peter out rapidly as improvements fail to yield further improvements, or else go FOOM. An ''exactly right law of diminishing returns'' that lets the system fly through the ''soft takeoff keyhole'' is unlikely - ''far'' more unlikely than seeing such behavior in a system with a roughly-constant underlying optimizer, like evolution improving brains, or human brains improving technology. Our present life is no good indicator of things to come."<ref>https://www.lesswrong.com/posts/tjH8XPxAnr6JRbh7k/hard-takeoff</ref>

narrow window argument against soft takeoff: eliezer says at some points that you need the parameter k that controls the growth to be in a really narrow range for it to NOT go into either a FOOM or petering out. in contrast, i think buck/paul has said something like, if you try to model it with math, you typically get a soft takeoff? what's going on here?

==References==

<references/>

[[Category:AI safety]]

← Older revision		Revision as of 16:56, 24 June 2020
Line 20:		Line 20:

	[https://www.greaterwrong.com/posts/66FKFkWAugS8diydF/modelling-continuous-progress This post] can be read as an argument against a narrow window, since it introduces a new parameter and shows that we get continuous takeoff for a large range of that parameter.		[https://www.greaterwrong.com/posts/66FKFkWAugS8diydF/modelling-continuous-progress This post] can be read as an argument against a narrow window, since it introduces a new parameter and shows that we get continuous takeoff for a large range of that parameter.
		+
		+	==See also==
		+
		+	* [[Progress in self-improvement]]

	==References==		==References==

@@ Line 18: / Line 18: @@
 <p>Since Deepmind picked a particular challenge time in advance, rather than challenging at a point where their AI seemed just barely good enough, it was improbable that they'd make *exactly* enough progress to give Sedol a nearly even fight.
 <p>AI is either overwhelmingly stupider or overwhelmingly smarter than you. The more other AI progress and the greater the hardware overhang, the less time you spend in the narrow space between these regions. There was a time when AIs were roughly as good as the best human Go-players, and it was a week in late January.</blockquote>
 ==References==

@@ Line 12: / Line 12: @@
 counter arguement? https://ea.greaterwrong.com/posts/tDk57GhrdK54TWzPY/i-m-buck-shlegeris-i-do-research-and-outreach-at-miri-ama/comment/R6HdmWSx9WfM3Dciu
-This part might also be relevant:
+This part might also be relevant:<ref>https://www.facebook.com/yudkowsky/posts/10154018209759228</ref>
 <blockquote><p>Human-equivalent competence is a small and undistinguished region in possibility-space.

← Older revision		Revision as of 23:36, 23 June 2020
Line 11:		Line 11:

	counter arguement? https://ea.greaterwrong.com/posts/tDk57GhrdK54TWzPY/i-m-buck-shlegeris-i-do-research-and-outreach-at-miri-ama/comment/R6HdmWSx9WfM3Dciu		counter arguement? https://ea.greaterwrong.com/posts/tDk57GhrdK54TWzPY/i-m-buck-shlegeris-i-do-research-and-outreach-at-miri-ama/comment/R6HdmWSx9WfM3Dciu
		+
		+	This part might also be relevant:
		+
		+	<blockquote><p>Human-equivalent competence is a small and undistinguished region in possibility-space.
		+	<p>As I tweeted early on when the first game still seemed in doubt, "Thing that would surprise me most about #alphago vs. #sedol: for either player to win by three games instead of four or five."
		+	<p>Since Deepmind picked a particular challenge time in advance, rather than challenging at a point where their AI seemed just barely good enough, it was improbable that they'd make exactly enough progress to give Sedol a nearly even fight.
		+	<p>AI is either overwhelmingly stupider or overwhelmingly smarter than you. The more other AI progress and the greater the hardware overhang, the less time you spend in the narrow space between these regions. There was a time when AIs were roughly as good as the best human Go-players, and it was a week in late January.</blockquote>

	==References==		==References==

← Older revision		Revision as of 19:12, 27 May 2020
Line 9:		Line 9:

	narrow window argument against soft takeoff: eliezer says at some points that you need the parameter k that controls the growth to be in a really narrow range for it to NOT go into either a FOOM or petering out. in contrast, i think buck/paul has said something like, if you try to model it with math, you typically get a soft takeoff? what's going on here?		narrow window argument against soft takeoff: eliezer says at some points that you need the parameter k that controls the growth to be in a really narrow range for it to NOT go into either a FOOM or petering out. in contrast, i think buck/paul has said something like, if you try to model it with math, you typically get a soft takeoff? what's going on here?
		+
		+	counter arguement? https://ea.greaterwrong.com/posts/tDk57GhrdK54TWzPY/i-m-buck-shlegeris-i-do-research-and-outreach-at-miri-ama/comment/R6HdmWSx9WfM3Dciu

	==References==		==References==

@@ Line 5: / Line 5: @@
 <blockquote><p>A secret of a lot of the futurism I'm willing to try and put any weight on, is that it involves the startling, amazing, counterintuitive prediction that something ends up in the not-human space instead of the human space - humans think their keyholes are the whole universe, because it's all they have experience with. So if you say, "It's in the (much larger) not-human space" it sounds like an amazing futuristic prediction and people will be shocked, and try to dispute it. But livable temperatures are rare in the universe - most of it's either much colder or much hotter. A place like Earth is an anomaly, though it's the only place beings like us can live; the interior of a star is much denser than the materials of the world we know, and the rest of the universe is much closer to vacuum.</p>
 <p>So really, the whole hard takeoff analysis of "flatline or FOOM" just ends up saying, "the AI will not hit the human timescale keyhole." From our perspective, an AI will either be so slow as to be bottlenecked, or so fast as to be FOOM. When you look at it that way, it's not so radical a prediction, is it?<ref>https://www.lesswrong.com/posts/tjH8XPxAnr6JRbh7k/hard-takeoff?commentId=PDeciAspyhH5eHWhc</ref></p></blockquote>
 narrow window argument against soft takeoff: eliezer says at some points that you need the parameter k that controls the growth to be in a really narrow range for it to NOT go into either a FOOM or petering out. in contrast, i think buck/paul has said something like, if you try to model it with math, you typically get a soft takeoff? what's going on here?

← Older revision		Revision as of 03:50, 22 May 2020
Line 1:		Line 1:
		+	'''Narrow window argument against continuous takeoff''' (also '''continuous takeoff keyhole''') says ...

	"When you fold a whole chain of differential equations in on itself like this, it should either peter out rapidly as improvements fail to yield further improvements, or else go FOOM. An ''exactly right law of diminishing returns'' that lets the system fly through the ''soft takeoff keyhole'' is unlikely - ''far'' more unlikely than seeing such behavior in a system with a roughly-constant underlying optimizer, like evolution improving brains, or human brains improving technology. Our present life is no good indicator of things to come."<ref>https://www.lesswrong.com/posts/tjH8XPxAnr6JRbh7k/hard-takeoff</ref>		"When you fold a whole chain of differential equations in on itself like this, it should either peter out rapidly as improvements fail to yield further improvements, or else go FOOM. An ''exactly right law of diminishing returns'' that lets the system fly through the ''soft takeoff keyhole'' is unlikely - ''far'' more unlikely than seeing such behavior in a system with a roughly-constant underlying optimizer, like evolution improving brains, or human brains improving technology. Our present life is no good indicator of things to come."<ref>https://www.lesswrong.com/posts/tjH8XPxAnr6JRbh7k/hard-takeoff</ref>

← Older revision		Revision as of 03:49, 22 May 2020
Line 1:		Line 1:

	"When you fold a whole chain of differential equations in on itself like this, it should either peter out rapidly as improvements fail to yield further improvements, or else go FOOM. An ''exactly right law of diminishing returns'' that lets the system fly through the ''soft takeoff keyhole'' is unlikely - ''far'' more unlikely than seeing such behavior in a system with a roughly-constant underlying optimizer, like evolution improving brains, or human brains improving technology. Our present life is no good indicator of things to come."<ref>https://www.lesswrong.com/posts/tjH8XPxAnr6JRbh7k/hard-takeoff</ref>		"When you fold a whole chain of differential equations in on itself like this, it should either peter out rapidly as improvements fail to yield further improvements, or else go FOOM. An ''exactly right law of diminishing returns'' that lets the system fly through the ''soft takeoff keyhole'' is unlikely - ''far'' more unlikely than seeing such behavior in a system with a roughly-constant underlying optimizer, like evolution improving brains, or human brains improving technology. Our present life is no good indicator of things to come."<ref>https://www.lesswrong.com/posts/tjH8XPxAnr6JRbh7k/hard-takeoff</ref>
		+
		+	<blockquote><p>A secret of a lot of the futurism I'm willing to try and put any weight on, is that it involves the startling, amazing, counterintuitive prediction that something ends up in the not-human space instead of the human space - humans think their keyholes are the whole universe, because it's all they have experience with. So if you say, "It's in the (much larger) not-human space" it sounds like an amazing futuristic prediction and people will be shocked, and try to dispute it. But livable temperatures are rare in the universe - most of it's either much colder or much hotter. A place like Earth is an anomaly, though it's the only place beings like us can live; the interior of a star is much denser than the materials of the world we know, and the rest of the universe is much closer to vacuum.</p>
		+	<p>So really, the whole hard takeoff analysis of "flatline or FOOM" just ends up saying, "the AI will not hit the human timescale keyhole." From our perspective, an AI will either be so slow as to be bottlenecked, or so fast as to be FOOM. When you look at it that way, it's not so radical a prediction, is it?<ref>https://www.lesswrong.com/posts/tjH8XPxAnr6JRbh7k/hard-takeoff?commentId=PDeciAspyhH5eHWhc</ref></p></blockquote>

	narrow window argument against soft takeoff: eliezer says at some points that you need the parameter k that controls the growth to be in a really narrow range for it to NOT go into either a FOOM or petering out. in contrast, i think buck/paul has said something like, if you try to model it with math, you typically get a soft takeoff? what's going on here?		narrow window argument against soft takeoff: eliezer says at some points that you need the parameter k that controls the growth to be in a really narrow range for it to NOT go into either a FOOM or petering out. in contrast, i think buck/paul has said something like, if you try to model it with math, you typically get a soft takeoff? what's going on here?