Missing gear for intelligence - Revision history

Issa: /* Evidence */

2020-06-30T21:42:40Z

‎Evidence

Issa at 16:35, 24 June 2020

2020-06-24T16:35:21Z

Issa: /* Does Eliezer Yudkowsky believe in a missing gear? */

2020-06-24T07:02:56Z

‎Does Eliezer Yudkowsky believe in a missing gear?

Issa: /* Does Eliezer Yudkowsky believe in a missing gear? */

2020-06-17T07:11:14Z

‎Does Eliezer Yudkowsky believe in a missing gear?

Issa: /* Does Eliezer Yudkowsky believe in a missing gear? */

2020-06-12T21:09:31Z

‎Does Eliezer Yudkowsky believe in a missing gear?

Issa: /* Does Eliezer Yudkowsky believe in a missing gear? */

2020-06-10T03:38:42Z

‎Does Eliezer Yudkowsky believe in a missing gear?

Issa: /* Evidence */

2020-06-09T23:55:55Z

‎Evidence

Issa: /* Evidence */

2020-06-09T23:54:41Z

‎Evidence

Issa at 23:49, 9 June 2020

2020-06-09T23:49:49Z

Issa at 23:49, 9 June 2020

2020-06-09T23:49:41Z

@@ Line 32: / Line 32: @@
 * Variation in scientific ability among humans suggests minor tweaks can lead to big improvements in ability: [[Wei Dai]]: "The biological von Neumann's brain must have been architecturally very similar to a typical university professor's. Nor could it have contained independent improvements to lots of different modules. Given this, I speculate that obtaining the analogous improvement in AI intelligence may only require a few design tweaks, which a relatively small project could find first by being luckier or a bit more talented than everyone else." [http://www.overcomingbias.com/2014/07/30855.html#comment-1502991300]
 ** possibly relevant: https://www.gwern.net/Differences
-* Variation in cognitive ability among chimps (what does this imply?)
+* Variation in cognitive ability among chimps: I think this is similar to variation among humans? The fact that [[Kanzi]] was much smarter than other chimps<ref>https://www.lesswrong.com/posts/YicoiQurNBxSp7a65/is-clickbait-destroying-our-general-intelligence?commentId=Cva4XBXsPcwjyFLAa</ref> suggests some kind of fine-tuning or "shallow" software can have some large effects on capability.
 * Comparing scientific ability in chimps vs humans suggests that some sort of "missing gear" was added which made human intelligence much more general. (This can also suggest a [[Secret sauce for intelligence#Evidence|secret sauce]], if the final piece was some big breakthrough.)

← Older revision		Revision as of 16:35, 24 June 2020
Line 43:		Line 43:

	* [[Missing gear vs secret sauce]]		* [[Missing gear vs secret sauce]]
		+
		+	==notes==
		+
		+	* Does AI impacts's [https://aiimpacts.org/likelihood-of-discontinuous-progress-around-the-development-of-agi/#Payoff_thresholds payoff thresholds] section assume that the threshold can be determined in advance? (rather than that just there ''is'' such a threshold, but we don't know its location.)

	==References==		==References==

@@ Line 22: / Line 22: @@
 * "Intelligence is mostly about architecture, or “knowledge” along the lines of knowing to look for causal structure (Bayes-net type stuff) in the environment; this kind of knowledge will usually be expressed procedurally as well as declaratively. Architecture is mostly about deep insights. This point has not yet been addressed (much) on Overcoming Bias, but Bayes nets can be considered as an archetypal example of “architecture” and “deep insight”. Also, ask yourself how lawful intelligence seemed to you before you started reading this blog, how lawful it seems to you now, then extrapolate outward from that." [https://www.greaterwrong.com/posts/z3kYdw54htktqt9Jb/what-i-think-if-not-why]
 * I think https://www.greaterwrong.com/posts/QrhAeKBkm2WsdRYao/searching-for-bayes-structure is also important for understanding Eliezer's view about what progress in AI looks like.
-* Posts like https://www.facebook.com/yudkowsky/posts/10153914357214228 where he uses [[AlphaGo]] as evidence of discontinuities also make it seem like he expects/allows a discontinuity to happen without a huge breakthrough.
+* Posts like https://www.facebook.com/yudkowsky/posts/10153914357214228 where he uses [[AlphaGo]] as evidence of discontinuities also make it seem like he expects/allows a discontinuity to happen without a huge breakthrough. (actually, maybe Eliezer would call AlphaGo a big architectural insight/breakthrough.)
 * "in an evolutionary trajectory, it can't ''literally'' be a "missing gear", the sort of discontinuity that follows from removing a gear that an otherwise functioning machine was built around.  So if you suppose that a final set of changes was enough to produce a sudden huge leap in effective intelligence, it does demand the question of what those changes were.  Something to do with reflection - the brain modeling or controlling itself - would be one obvious candidate.  Or perhaps a change in motivations (more curious individuals, using the brainpower they have in different directions) in which case you ''wouldn't'' expect that discontinuity to appear in the AI's development, but you would expect it to be more effective at earlier stages than humanity's evolutionary history would suggest...  But you could have whole journal issues about that one question, so I'm just going to leave it at that."<ref>https://lw2.issarice.com/posts/tjH8XPxAnr6JRbh7k/hard-takeoff</ref>
 * "Later on, there's an exciting result in a more interesting algorithm that operates on a more general level (I'm not being very specific here, for the same reason I don't talk about my ideas for building really great bioweapons)." [https://www.facebook.com/yudkowsky/posts/10154018209759228]

@@ Line 24: / Line 24: @@
 * Posts like https://www.facebook.com/yudkowsky/posts/10153914357214228 where he uses [[AlphaGo]] as evidence of discontinuities also make it seem like he expects/allows a discontinuity to happen without a huge breakthrough.
 * "in an evolutionary trajectory, it can't ''literally'' be a "missing gear", the sort of discontinuity that follows from removing a gear that an otherwise functioning machine was built around.  So if you suppose that a final set of changes was enough to produce a sudden huge leap in effective intelligence, it does demand the question of what those changes were.  Something to do with reflection - the brain modeling or controlling itself - would be one obvious candidate.  Or perhaps a change in motivations (more curious individuals, using the brainpower they have in different directions) in which case you ''wouldn't'' expect that discontinuity to appear in the AI's development, but you would expect it to be more effective at earlier stages than humanity's evolutionary history would suggest...  But you could have whole journal issues about that one question, so I'm just going to leave it at that."<ref>https://lw2.issarice.com/posts/tjH8XPxAnr6JRbh7k/hard-takeoff</ref>
 ==Evidence==

@@ Line 23: / Line 23: @@
 * I think https://www.greaterwrong.com/posts/QrhAeKBkm2WsdRYao/searching-for-bayes-structure is also important for understanding Eliezer's view about what progress in AI looks like.
 * Posts like https://www.facebook.com/yudkowsky/posts/10153914357214228 where he uses [[AlphaGo]] as evidence of discontinuities also make it seem like he expects/allows a discontinuity to happen without a huge breakthrough.
 ==Evidence==

← Older revision		Revision as of 23:49, 9 June 2020
Line 1:		Line 1:
	'''Missing gear for intelligence''' (also called '''[[one wrong number problem]]''', '''step function''', '''understanding is discontinuous''', '''payoff thresholds argument''') is an argument for a [[Discontinuous takeoff\|discontinuity]] in AI takeoff. Unlike a [[secret sauce for intelligence]], the missing gear argument does not require that the final part of AI development be a huge conceptual breakthrough; instead, the final piece is small but nevertheless results in a discontinuity (see Variants section below for details).		'''Missing gear for intelligence''' (also called '''[[one wrong number problem]]''', '''step function''', '''understanding is discontinuous''', '''payoff thresholds argument''') is an argument for a [[Discontinuous takeoff\|discontinuity]] in AI takeoff. Unlike a [[secret sauce for intelligence]], the missing gear argument does not require that the final part of AI development be a huge conceptual breakthrough; instead, the final piece is small but nevertheless results in a discontinuity (see Variants section below for details).
−
−

	==Variants==		==Variants==

@@ Line 1: / Line 1: @@
-'''Missing gear for intelligence''' (also called '''[[one wrong number problem]]''', '''step function''', '''understanding is discontinuous''', '''payoff thresholds argument''') is an argument for a [[Discontinuous takeoff|discontinuity]] in AI takeoff. Unlike a [[secret sauce for intelligence]], the missing gear argument does not require that the final part of AI development be a huge conceptual breakthrough; instead, the final piece can be minor design tweaks, a payoff threshold in intelligence, or understanding being discontinuous.
+'''Missing gear for intelligence''' (also called '''[[one wrong number problem]]''', '''step function''', '''understanding is discontinuous''', '''payoff thresholds argument''') is an argument for a [[Discontinuous takeoff|discontinuity]] in AI takeoff. Unlike a [[secret sauce for intelligence]], the missing gear argument does not require that the final part of AI development be a huge conceptual breakthrough; instead, the final piece is small but nevertheless results in a discontinuity (see Variants section below for details).
 ==Variants==
@@ Line 10: / Line 12: @@
 * "missing gear" is in the middle in terms of generality because it says the final piece is a "gear", so some structurally distinct thing? -- I think "missing gear" is also somewhat ambiguous about whether the final piece is big vs small
 * "payoff thresholds" is the most general because it makes no assumption about the nature of the final piece, just that there is some imaginary line, and once you cross it something happens
 (Another different but related possibility is that someone team suddenly realizes a particular use case of AI in doing AI research. e.g. maybe lots of people have IQ 35 AIs, but one project realizes you can use them to speed up some particular part of research (before the other projects do) and suddenly they get a huge lead.)