Difference between revisions of "Agent foundations"
(→Terminology) |
|||
| (One intermediate revision by the same user not shown) | |||
| Line 5: | Line 5: | ||
* agent foundations: in the [https://intelligence.org/files/TechnicalAgenda.pdf original agenda], "agent foundations" was used as an umbrella term to include all of MIRI's work. (Later, MIRI also created their "Alignment for advanced machine learning systems" agenda [https://intelligence.org/2016/07/27/alignment-machine-learning/], which then got cancelled when [[Jessica]] and [[Patrick]] left MIRI.) This includes the HRAD topics like decision theory and logical uncertainty, but ''in addition'' also includes error-tolerant agent designs (which includes [[corrigilibity]]) and value specification (which includes [[value learning]] and normative uncertainty). Since MIRI seems to have focused most on the HRAD topics, I think some people ''might'' think agent foundations = HRAD and use "agent foundations" where they actually meant "HRAD". | * agent foundations: in the [https://intelligence.org/files/TechnicalAgenda.pdf original agenda], "agent foundations" was used as an umbrella term to include all of MIRI's work. (Later, MIRI also created their "Alignment for advanced machine learning systems" agenda [https://intelligence.org/2016/07/27/alignment-machine-learning/], which then got cancelled when [[Jessica]] and [[Patrick]] left MIRI.) This includes the HRAD topics like decision theory and logical uncertainty, but ''in addition'' also includes error-tolerant agent designs (which includes [[corrigilibity]]) and value specification (which includes [[value learning]] and normative uncertainty). Since MIRI seems to have focused most on the HRAD topics, I think some people ''might'' think agent foundations = HRAD and use "agent foundations" where they actually meant "HRAD". | ||
* embedded agency | * embedded agency | ||
| − | * naturalized | + | * naturalized agency: the old term for embedded agency<ref>https://www.greaterwrong.com/posts/p7x32SEt43ZMC9r7r/embedded-agents/comment/rHjqqouz4KRG8Dj7y</ref> |
| + | * naturalized induction | ||
* highly reliable agent designs (HRAD) | * highly reliable agent designs (HRAD) | ||
* the relation of all of the above to MIRI's new research agenda (which hasn't been released publicly) | * the relation of all of the above to MIRI's new research agenda (which hasn't been released publicly) | ||
| + | |||
| + | ==References== | ||
| + | |||
| + | <references/> | ||
| + | |||
| + | [[Category:AI safety]] | ||
Latest revision as of 19:08, 27 February 2021
Terminology
There are many different terms used to describe MIRI's research, and it's a little unclear how careful people are when using these terms.
- agent foundations: in the original agenda, "agent foundations" was used as an umbrella term to include all of MIRI's work. (Later, MIRI also created their "Alignment for advanced machine learning systems" agenda [1], which then got cancelled when Jessica and Patrick left MIRI.) This includes the HRAD topics like decision theory and logical uncertainty, but in addition also includes error-tolerant agent designs (which includes corrigilibity) and value specification (which includes value learning and normative uncertainty). Since MIRI seems to have focused most on the HRAD topics, I think some people might think agent foundations = HRAD and use "agent foundations" where they actually meant "HRAD".
- embedded agency
- naturalized agency: the old term for embedded agency[1]
- naturalized induction
- highly reliable agent designs (HRAD)
- the relation of all of the above to MIRI's new research agenda (which hasn't been released publicly)
