Difference between revisions of "Agent foundations"
(→Terminology) |
|||
| (4 intermediate revisions by the same user not shown) | |||
| Line 3: | Line 3: | ||
There are many different terms used to describe MIRI's research, and it's a little unclear how careful people are when using these terms. | There are many different terms used to describe MIRI's research, and it's a little unclear how careful people are when using these terms. | ||
| − | * agent foundations: in the [https://intelligence.org/files/TechnicalAgenda.pdf original agenda], "agent foundations" was used as an umbrella term to include all of MIRI's work. This includes the HRAD topics like decision theory and logical uncertainty, but ''in addition'' also includes error-tolerant agent designs (which includes [[corrigilibity]]) and value specification (which includes [[value learning]]). Since MIRI seems to have focused most on the HRAD topics, I think some people ''might'' think agent foundations = HRAD and use "agent foundations" where they actually meant "HRAD". | + | * agent foundations: in the [https://intelligence.org/files/TechnicalAgenda.pdf original agenda], "agent foundations" was used as an umbrella term to include all of MIRI's work. (Later, MIRI also created their "Alignment for advanced machine learning systems" agenda [https://intelligence.org/2016/07/27/alignment-machine-learning/], which then got cancelled when [[Jessica]] and [[Patrick]] left MIRI.) This includes the HRAD topics like decision theory and logical uncertainty, but ''in addition'' also includes error-tolerant agent designs (which includes [[corrigilibity]]) and value specification (which includes [[value learning]] and normative uncertainty). Since MIRI seems to have focused most on the HRAD topics, I think some people ''might'' think agent foundations = HRAD and use "agent foundations" where they actually meant "HRAD". |
* embedded agency | * embedded agency | ||
| + | * naturalized agency: the old term for embedded agency<ref>https://www.greaterwrong.com/posts/p7x32SEt43ZMC9r7r/embedded-agents/comment/rHjqqouz4KRG8Dj7y</ref> | ||
* naturalized induction | * naturalized induction | ||
* highly reliable agent designs (HRAD) | * highly reliable agent designs (HRAD) | ||
* the relation of all of the above to MIRI's new research agenda (which hasn't been released publicly) | * the relation of all of the above to MIRI's new research agenda (which hasn't been released publicly) | ||
| + | |||
| + | ==References== | ||
| + | |||
| + | <references/> | ||
| + | |||
| + | [[Category:AI safety]] | ||
Latest revision as of 19:08, 27 February 2021
Terminology
There are many different terms used to describe MIRI's research, and it's a little unclear how careful people are when using these terms.
- agent foundations: in the original agenda, "agent foundations" was used as an umbrella term to include all of MIRI's work. (Later, MIRI also created their "Alignment for advanced machine learning systems" agenda [1], which then got cancelled when Jessica and Patrick left MIRI.) This includes the HRAD topics like decision theory and logical uncertainty, but in addition also includes error-tolerant agent designs (which includes corrigilibity) and value specification (which includes value learning and normative uncertainty). Since MIRI seems to have focused most on the HRAD topics, I think some people might think agent foundations = HRAD and use "agent foundations" where they actually meant "HRAD".
- embedded agency
- naturalized agency: the old term for embedded agency[1]
- naturalized induction
- highly reliable agent designs (HRAD)
- the relation of all of the above to MIRI's new research agenda (which hasn't been released publicly)
