Difference between revisions of "Agent foundations"

From Issawiki
Jump to: navigation, search
(Terminology)
 
Line 13: Line 13:
  
 
<references/>
 
<references/>
 +
 +
[[Category:AI safety]]

Latest revision as of 19:08, 27 February 2021

Terminology

There are many different terms used to describe MIRI's research, and it's a little unclear how careful people are when using these terms.

  • agent foundations: in the original agenda, "agent foundations" was used as an umbrella term to include all of MIRI's work. (Later, MIRI also created their "Alignment for advanced machine learning systems" agenda [1], which then got cancelled when Jessica and Patrick left MIRI.) This includes the HRAD topics like decision theory and logical uncertainty, but in addition also includes error-tolerant agent designs (which includes corrigilibity) and value specification (which includes value learning and normative uncertainty). Since MIRI seems to have focused most on the HRAD topics, I think some people might think agent foundations = HRAD and use "agent foundations" where they actually meant "HRAD".
  • embedded agency
  • naturalized agency: the old term for embedded agency[1]
  • naturalized induction
  • highly reliable agent designs (HRAD)
  • the relation of all of the above to MIRI's new research agenda (which hasn't been released publicly)

References