Goalpost for usefulness of HRAD work - Revision history

Issa at 20:17, 26 June 2020

2020-06-26T20:17:57Z

Issa: /* See also */

2020-06-01T21:11:39Z

‎See also

Issa at 09:30, 27 May 2020

2020-05-27T09:30:38Z

Issa at 09:30, 27 May 2020

2020-05-27T09:30:22Z

Issa at 09:30, 27 May 2020

2020-05-27T09:30:09Z

Issa at 09:28, 27 May 2020

2020-05-27T09:28:34Z

Issa at 07:19, 27 May 2020

2020-05-27T07:19:24Z

Issa at 06:14, 27 May 2020

2020-05-27T06:14:51Z

Issa at 05:16, 27 May 2020

2020-05-27T05:16:16Z

Issa at 00:53, 27 May 2020

2020-05-27T00:53:13Z

@@ Line 22: / Line 22: @@
 * [[List of success criteria for HRAD work]]
 * [[Something like realism about rationality]]
 [[Category:AI safety]]

@@ Line 20: / Line 20: @@
 ==See also==
 * [[Something like realism about rationality]]
 [[Category:AI safety]]

@@ Line 8: / Line 8: @@
 This is an example of what I mean, when I say that the goalpost is set at an easier spot by MIRI: "Techniques you can actually adapt in a safe AI, come the day, will probably have very simple cores — the sort of core concept that takes up three paragraphs, where any reviewer who didn’t spend five years struggling on the problem themselves will think, “Oh I could have thought of that.” Someday there may be a book full of clever and difficult things to say about the simple core — contrast the simplicity of the core concept of causal models, versus the complexity of proving all the clever things Judea Pearl had to say about causal models. But the planetary benefit is mainly from posing understandable problems crisply enough so that people can see they are open, and then from the simpler abstract properties of a found solution — complicated aspects will not carry over to real AIs later." [https://intelligence.org/files/OpenPhil2016Supplement.pdf#page=13]
-In contrast, the kind of goalpost [[Daniel Dewey]] sets in [https://eaforum.issarice.com/posts/SEL9PW8jozrvLnkb4/my-current-thoughts-on-miri-s-highly-reliable-agent-design] seems much harder/restrictive.
+In contrast, the kind of goalpost [[Daniel Dewey]] sets in [https://eaforum.issarice.com/posts/SEL9PW8jozrvLnkb4/my-current-thoughts-on-miri-s-highly-reliable-agent-design] seems much harder/more restrictive.
 ----

← Older revision		Revision as of 09:30, 27 May 2020
Line 9:		Line 9:

	In contrast, the kind of goalpost [[Daniel Dewey]] sets in [https://eaforum.issarice.com/posts/SEL9PW8jozrvLnkb4/my-current-thoughts-on-miri-s-highly-reliable-agent-design] seems much harder/restrictive.		In contrast, the kind of goalpost [[Daniel Dewey]] sets in [https://eaforum.issarice.com/posts/SEL9PW8jozrvLnkb4/my-current-thoughts-on-miri-s-highly-reliable-agent-design] seems much harder/restrictive.
		+
		+	----

	* will early advanced AI systems be understandable in terms of HRAD's formalisms? [https://eaforum.issarice.com/posts/SEL9PW8jozrvLnkb4/my-current-thoughts-on-miri-s-highly-reliable-agent-design#3__What_do_I_think_about_HRAD_]		* will early advanced AI systems be understandable in terms of HRAD's formalisms? [https://eaforum.issarice.com/posts/SEL9PW8jozrvLnkb4/my-current-thoughts-on-miri-s-highly-reliable-agent-design#3__What_do_I_think_about_HRAD_]

@@ Line 6: / Line 6: @@
 It seems like there's a question of what is the relevant goalpost, for deciding whether HRAD work is useful.
-"Techniques you can actually adapt in a safe AI, come the day, will probably have very simple cores — the sort of core concept that takes up three paragraphs, where any reviewer who didn’t spend five years struggling on the problem themselves will think, “Oh I could have thought of that.” Someday there may be a book full of clever and difficult things to say about the simple core — contrast the simplicity of the core concept of causal models, versus the complexity of proving all the clever things Judea Pearl had to say about causal models. But the planetary benefit is mainly from posing understandable problems crisply enough so that people can see they are open, and then from the simpler abstract properties of a found solution — complicated aspects will not carry over to real AIs later." [https://intelligence.org/files/OpenPhil2016Supplement.pdf#page=13]
+This is an example of what I mean, when I say that the goalpost is set at an easier spot by MIRI: "Techniques you can actually adapt in a safe AI, come the day, will probably have very simple cores — the sort of core concept that takes up three paragraphs, where any reviewer who didn’t spend five years struggling on the problem themselves will think, “Oh I could have thought of that.” Someday there may be a book full of clever and difficult things to say about the simple core — contrast the simplicity of the core concept of causal models, versus the complexity of proving all the clever things Judea Pearl had to say about causal models. But the planetary benefit is mainly from posing understandable problems crisply enough so that people can see they are open, and then from the simpler abstract properties of a found solution — complicated aspects will not carry over to real AIs later." [https://intelligence.org/files/OpenPhil2016Supplement.pdf#page=13]
 * will early advanced AI systems be understandable in terms of HRAD's formalisms? [https://eaforum.issarice.com/posts/SEL9PW8jozrvLnkb4/my-current-thoughts-on-miri-s-highly-reliable-agent-design#3__What_do_I_think_about_HRAD_]

← Older revision		Revision as of 09:28, 27 May 2020
Line 5:		Line 5:

	It seems like there's a question of what is the relevant goalpost, for deciding whether HRAD work is useful.		It seems like there's a question of what is the relevant goalpost, for deciding whether HRAD work is useful.
		+
		+	"Techniques you can actually adapt in a safe AI, come the day, will probably have very simple cores — the sort of core concept that takes up three paragraphs, where any reviewer who didn’t spend five years struggling on the problem themselves will think, “Oh I could have thought of that.” Someday there may be a book full of clever and difficult things to say about the simple core — contrast the simplicity of the core concept of causal models, versus the complexity of proving all the clever things Judea Pearl had to say about causal models. But the planetary benefit is mainly from posing understandable problems crisply enough so that people can see they are open, and then from the simpler abstract properties of a found solution — complicated aspects will not carry over to real AIs later." [https://intelligence.org/files/OpenPhil2016Supplement.pdf#page=13]

	* will early advanced AI systems be understandable in terms of HRAD's formalisms? [https://eaforum.issarice.com/posts/SEL9PW8jozrvLnkb4/my-current-thoughts-on-miri-s-highly-reliable-agent-design#3__What_do_I_think_about_HRAD_]		* will early advanced AI systems be understandable in terms of HRAD's formalisms? [https://eaforum.issarice.com/posts/SEL9PW8jozrvLnkb4/my-current-thoughts-on-miri-s-highly-reliable-agent-design#3__What_do_I_think_about_HRAD_]

@@ Line 1: / Line 1: @@
 When thinking about the question of "How useful is [[HRAD]] work?", what standards/goalposts should we use? There's a pattern I see where:
-* people advocating [[HRAD]] research bring up historical cases like Turing, Shannon, etc. where formalization worked well
+* people advocating [[HRAD]] research bring up historical cases like Turing, Shannon, etc. where formalization worked well. There is also the [[deconfusion research]] framing, where just understanding what's going on better is a form of progress.
 * people arguing against HRAD research talk about how "complete axiomatic descriptions" haven't been useful so far in AI, and how they aren't used to describe machine learning systems

← Older revision		Revision as of 06:14, 27 May 2020
Line 1:		Line 1:
−	There's a pattern I see where:	+	When thinking about the question of "How useful is [[HRAD]] work?", what standards/goalposts should we use? There's a pattern I see where:

	* people advocating [[HRAD]] research bring up historical cases like Turing, Shannon, etc. where formalization worked well		* people advocating [[HRAD]] research bring up historical cases like Turing, Shannon, etc. where formalization worked well

@@ Line 11: / Line 11: @@
 ** what will early advanced AI systems look like?
 * how convincing historical examples are (e.g. Shannon, Turing, Bayes, Pearl, Kolmogorov, null-terminated strings in C [https://eaforum.issarice.com/posts/SEL9PW8jozrvLnkb4/my-current-thoughts-on-miri-s-highly-reliable-agent-design#Z6TbXivpjxWyc8NYM], [https://www.facebook.com/danielfilan/posts/10212534556141446] [https://www.facebook.com/robbensinger/posts/10160644236785447], Eliezer also brings up the Shannon vs Poe chess example) See also [[selection effect for successful formalizations]].
 [[Category:AI safety]]

← Older revision		Revision as of 00:53, 27 May 2020
Line 5:		Line 5:

	It seems like there's a question of what is the relevant goalpost, for deciding whether HRAD work is useful.		It seems like there's a question of what is the relevant goalpost, for deciding whether HRAD work is useful.
		+
		+	* will early advanced AI systems be understandable in terms of HRAD's formalisms? [https://eaforum.issarice.com/posts/SEL9PW8jozrvLnkb4/my-current-thoughts-on-miri-s-highly-reliable-agent-design#3__What_do_I_think_about_HRAD_]
		+	** lack of historical precedent at applying "complete axiomatic descriptions of AI systems" to help design AI systems [https://eaforum.issarice.com/posts/SEL9PW8jozrvLnkb4/my-current-thoughts-on-miri-s-highly-reliable-agent-design#3a__Low_credence_that_HRAD_will_be_applicable__25___]
		+	** lack of success so far at using complete axiomatic descriptions for modern ML systems [https://eaforum.issarice.com/posts/SEL9PW8jozrvLnkb4/my-current-thoughts-on-miri-s-highly-reliable-agent-design#3a__Low_credence_that_HRAD_will_be_applicable__25___]
		+	** what will early advanced AI systems look like?
		+	* how convincing historical examples are (e.g. Shannon, Turing, Bayes, Pearl, Kolmogorov, null-terminated strings in C [https://eaforum.issarice.com/posts/SEL9PW8jozrvLnkb4/my-current-thoughts-on-miri-s-highly-reliable-agent-design#Z6TbXivpjxWyc8NYM], [https://www.facebook.com/danielfilan/posts/10212534556141446] [https://www.facebook.com/robbensinger/posts/10160644236785447], Eliezer also brings up the Shannon vs Poe chess example) See also [[selection effect for successful formalizations]].

	[[Category:AI safety]]		[[Category:AI safety]]