Comparison of AI takeoff scenarios - Revision history

Issa at 00:50, 5 March 2021

2021-03-05T00:50:41Z

Issa at 19:09, 27 February 2021

2021-02-27T19:09:18Z

Issa at 23:17, 13 June 2020

2020-06-13T23:17:44Z

Issa at 23:01, 13 June 2020

2020-06-13T23:01:04Z

Issa at 06:32, 19 March 2020

2020-03-19T06:32:03Z

Issa at 05:56, 24 February 2020

2020-02-24T05:56:24Z

Issa at 05:31, 24 February 2020

2020-02-24T05:31:50Z

Issa at 04:01, 24 February 2020

2020-02-24T04:01:46Z

Issa at 04:01, 24 February 2020

2020-02-24T04:01:20Z

Issa at 03:56, 24 February 2020

2020-02-24T03:56:55Z

@@ Line 7: / Line 7: @@
 | [[Paul]]'s slow takeoff || Yes || Yes<ref group=notes>Paul: "Note: this is ''not'' a post about whether an intelligence explosion will occur. That seems very likely to me. Quantitatively I expect it to go [https://sideways-view.com/2017/10/04/hyperbolic-growth/ along these lines]. So e.g. while I disagree with many of the claims and assumptions in [https://intelligence.org/files/IEM.pdf Intelligence Explosion Microeconomics], I don’t disagree with the central thesis or with most of the arguments." [https://sideways-view.com/2018/02/24/takeoff-speeds/]</ref> || No || No
 |-
-| [[Daniel Kokotajlo]] || Yes || Yes || No || Yes
+| [[Daniel Kokotajlo]] (NOTE: this row needs to be split because Daniel has multiple scenarios in mind) || Yes || Yes || No || Yes
 |-
 | [[Hansonian]] slow takeoff || Yes || No?<ref group=notes>In some places Hanson says things like "You may recall that I did not dispute that an AI based economy would grow faster than does our economy today. The issue is the relative rate of growth of one AI system, across a broad range of tasks, relative to the entire rest of the world at that time." [https://www.facebook.com/yudkowsky/posts/10155848910529228?comment_id=10155848990064228&reply_comment_id=10155849018834228] This sounds more like Paul's takeoff scenario. I'm not clear on how the Paul and Hanson scenarios differ. Also see [https://www.greaterwrong.com/posts/AfGmsjGPXN97kNp57/arguments-about-fast-takeoff#comment-ov4b6S2igwRZxXB8x this comment] by Paul: "I agree that some people talking about slow takeoff mean something stronger (e.g. “no singularity ever”), but I think that’s an unusual position inside our crowd (and even an unusual position amongst thoughtful ML researchers), and it’s not e.g. Robin’s view (who I take as a central example of a slow takeoff proponent)." So who ''are'' these slow takeoff proponents who don't believe in an intelligence explosion/singularity?</ref> || No || No

← Older revision		Revision as of 19:09, 27 February 2021
Line 31:		Line 31:

	<references group=notes/>		<references group=notes/>
		+
		+	[[Category:AI safety]]

← Older revision		Revision as of 23:17, 13 June 2020
Line 21:		Line 21:

	"I'm inclined to agree that some aspects of Robin's forecasts haven't fared well over the last few years (e.g. human object-level knowledge doesn't look like it's going to be very valuable), but I don't think anything is challenging the basic economics." [https://www.facebook.com/yudkowsky/posts/10155848910529228?comment_id=10155849456079228]		"I'm inclined to agree that some aspects of Robin's forecasts haven't fared well over the last few years (e.g. human object-level knowledge doesn't look like it's going to be very valuable), but I don't think anything is challenging the basic economics." [https://www.facebook.com/yudkowsky/posts/10155848910529228?comment_id=10155849456079228]
		+
		+	There are some other comparison axes for different views on takeoff (wall clock time, GDP trend extrapolation [https://www.lesswrong.com/posts/YgNYA6pj2hPSDQiTE/distinguishing-definitions-of-takeoff?commentId=zyPxKXsX8ztEe6cuh]) but I actually don't think this is where the real disagreement is.

	==See also==		==See also==

← Older revision		Revision as of 23:01, 13 June 2020
Line 19:		Line 19:

	How are Robin's and Paul's views different? Does Robin's takeoff scenario just have extra/unnecessary parts (like massive modularity, architecture changes don't produce large advantages)? Or is there an actual difference in predictions?		How are Robin's and Paul's views different? Does Robin's takeoff scenario just have extra/unnecessary parts (like massive modularity, architecture changes don't produce large advantages)? Or is there an actual difference in predictions?
		+
		+	"I'm inclined to agree that some aspects of Robin's forecasts haven't fared well over the last few years (e.g. human object-level knowledge doesn't look like it's going to be very valuable), but I don't think anything is challenging the basic economics." [https://www.facebook.com/yudkowsky/posts/10155848910529228?comment_id=10155849456079228]

	==See also==		==See also==

← Older revision		Revision as of 06:32, 19 March 2020
Line 17:		Line 17:

	This comment suggests more columns for the table: https://www.greaterwrong.com/posts/AfGmsjGPXN97kNp57/arguments-about-fast-takeoff#comment-JEkP5AmXmi4dHHpqo		This comment suggests more columns for the table: https://www.greaterwrong.com/posts/AfGmsjGPXN97kNp57/arguments-about-fast-takeoff#comment-JEkP5AmXmi4dHHpqo
		+
		+	How are Robin's and Paul's views different? Does Robin's takeoff scenario just have extra/unnecessary parts (like massive modularity, architecture changes don't produce large advantages)? Or is there an actual difference in predictions?

	==See also==		==See also==

← Older revision		Revision as of 05:56, 24 February 2020
Line 15:		Line 15:

	DSA without discontinuity is a little weird to think about/unstable situation i think: if you achieve DSA without a large lead in AI capabilities, well then you can probably achieve a discontinuity soon afterwards anyway. Maybe the questions should be "DSA?" and "If DSA, achieved via discontinuity?"		DSA without discontinuity is a little weird to think about/unstable situation i think: if you achieve DSA without a large lead in AI capabilities, well then you can probably achieve a discontinuity soon afterwards anyway. Maybe the questions should be "DSA?" and "If DSA, achieved via discontinuity?"
		+
		+	This comment suggests more columns for the table: https://www.greaterwrong.com/posts/AfGmsjGPXN97kNp57/arguments-about-fast-takeoff#comment-JEkP5AmXmi4dHHpqo

	==See also==		==See also==

@@ Line 9: / Line 9: @@
 | [[Daniel Kokotajlo]] || Yes || Yes || No || Yes
 |-
-| [[Hansonian]] slow takeoff || Yes || No?<ref group=notes>In some places Hanson says things like "You may recall that I did not dispute that an AI based economy would grow faster than does our economy today. The issue is the relative rate of growth of one AI system, across a broad range of tasks, relative to the entire rest of the world at that time." [https://www.facebook.com/yudkowsky/posts/10155848910529228?comment_id=10155848990064228&reply_comment_id=10155849018834228] This sounds more like Paul's takeoff scenario. I'm not clear on how the Paul and Hanson scenarios differ.</ref> || No || No
+| [[Hansonian]] slow takeoff || Yes || No?<ref group=notes>In some places Hanson says things like "You may recall that I did not dispute that an AI based economy would grow faster than does our economy today. The issue is the relative rate of growth of one AI system, across a broad range of tasks, relative to the entire rest of the world at that time." [https://www.facebook.com/yudkowsky/posts/10155848910529228?comment_id=10155848990064228&reply_comment_id=10155849018834228] This sounds more like Paul's takeoff scenario. I'm not clear on how the Paul and Hanson scenarios differ. Also see [https://www.greaterwrong.com/posts/AfGmsjGPXN97kNp57/arguments-about-fast-takeoff#comment-ov4b6S2igwRZxXB8x this comment] by Paul: "I agree that some people talking about slow takeoff mean something stronger (e.g. “no singularity ever”), but I think that’s an unusual position inside our crowd (and even an unusual position amongst thoughtful ML researchers), and it’s not e.g. Robin’s view (who I take as a central example of a slow takeoff proponent)." So who ''are'' these slow takeoff proponents who don't believe in an intelligence explosion/singularity?</ref> || No || No
 |-
 | [[Eric Drexler]]'s CAIS || Yes || No? I think he says something weird, like all of the AI systems together recursively self-improving at the all-AI-services-combined level, without a single agent-like AI that self-improves. || No || No?

@@ Line 14: / Line 14: @@
 |}
-DSA without discontinuity is a little weird to think about/unstable situation i think: if you achieve DSA without a large lead in AI capabilities, well then you can probably achieve a discontinuity soon anyway. Maybe the questions should be "DSA?" and "If DSA, achieved via discontinuity?"
+DSA without discontinuity is a little weird to think about/unstable situation i think: if you achieve DSA without a large lead in AI capabilities, well then you can probably achieve a discontinuity soon afterwards anyway. Maybe the questions should be "DSA?" and "If DSA, achieved via discontinuity?"
 ==See also==

@@ Line 1: / Line 1: @@
 {| class="wikitable"
 |-
-! Scenario !! Significant changes to the world prior to critical AI capability threshold being reached? !! Intelligence explosion? !! Decisive strategic advantage? / Unipolar outcome? (i.e. not distributed) Can a single AI project get massively ahead (either by investing way more effort into building AGI or by converting a small lead into a large lead)?
+! Scenario !! Significant changes to the world prior to critical AI capability threshold being reached? !! Intelligence explosion? !! Discontinuity in AI development? (i.e. one project suddenly has much more advanced AI than everyone else in the world combined) !! Decisive strategic advantage? / Unipolar outcome? (i.e. not distributed) Can a single AI project get massively ahead (either by investing way more effort into building AGI or by converting a small lead into a large lead)?
 |-
-| [[Yudkowskian]] hard takeoff || No || Yes || Yes
+| [[Yudkowskian]] hard takeoff || No || Yes || Yes || Yes
 |-
-| [[Paul]]'s slow takeoff || Yes || Yes<ref group=notes>Paul: "Note: this is ''not'' a post about whether an intelligence explosion will occur. That seems very likely to me. Quantitatively I expect it to go [https://sideways-view.com/2017/10/04/hyperbolic-growth/ along these lines]. So e.g. while I disagree with many of the claims and assumptions in [https://intelligence.org/files/IEM.pdf Intelligence Explosion Microeconomics], I don’t disagree with the central thesis or with most of the arguments." [https://sideways-view.com/2018/02/24/takeoff-speeds/]</ref> || No
+| [[Paul]]'s slow takeoff || Yes || Yes<ref group=notes>Paul: "Note: this is ''not'' a post about whether an intelligence explosion will occur. That seems very likely to me. Quantitatively I expect it to go [https://sideways-view.com/2017/10/04/hyperbolic-growth/ along these lines]. So e.g. while I disagree with many of the claims and assumptions in [https://intelligence.org/files/IEM.pdf Intelligence Explosion Microeconomics], I don’t disagree with the central thesis or with most of the arguments." [https://sideways-view.com/2018/02/24/takeoff-speeds/]</ref> || No || No
 |-
-| [[Daniel Kokotajlo]] || Yes || Yes || Yes
+| [[Daniel Kokotajlo]] || Yes || Yes || No || Yes
 |-
-| [[Hansonian]] slow takeoff || Yes || No?<ref group=notes>In some places Hanson says things like "You may recall that I did not dispute that an AI based economy would grow faster than does our economy today. The issue is the relative rate of growth of one AI system, across a broad range of tasks, relative to the entire rest of the world at that time." [https://www.facebook.com/yudkowsky/posts/10155848910529228?comment_id=10155848990064228&reply_comment_id=10155849018834228] This sounds more like Paul's takeoff scenario. I'm not clear on how the Paul and Hanson scenarios differ.</ref> || No
+| [[Hansonian]] slow takeoff || Yes || No?<ref group=notes>In some places Hanson says things like "You may recall that I did not dispute that an AI based economy would grow faster than does our economy today. The issue is the relative rate of growth of one AI system, across a broad range of tasks, relative to the entire rest of the world at that time." [https://www.facebook.com/yudkowsky/posts/10155848910529228?comment_id=10155848990064228&reply_comment_id=10155849018834228] This sounds more like Paul's takeoff scenario. I'm not clear on how the Paul and Hanson scenarios differ.</ref> || No || No
 |-
-| [[Eric Drexler]]'s CAIS || Yes || No? I think he says something weird, like all of the AI systems together recursively self-improving at the all-AI-services-combined level, without a single agent-like AI that self-improves. || No?
+| [[Eric Drexler]]'s CAIS || Yes || No? I think he says something weird, like all of the AI systems together recursively self-improving at the all-AI-services-combined level, without a single agent-like AI that self-improves. || No || No?
 |}