Agent foundations

From Issawiki
Jump to: navigation, search

Terminology

There are many different terms used to describe MIRI's research, and it's a little unclear how careful people are when using these terms.

  • agent foundations: in the original agenda, "agent foundations" was used as an umbrella term to include all of MIRI's work. (Later, MIRI also created their "Alignment for advanced machine learning systems" agenda [1], which then got cancelled when Jessica and Patrick left MIRI.) This includes the HRAD topics like decision theory and logical uncertainty, but in addition also includes error-tolerant agent designs (which includes corrigilibity) and value specification (which includes value learning and normative uncertainty). Since MIRI seems to have focused most on the HRAD topics, I think some people might think agent foundations = HRAD and use "agent foundations" where they actually meant "HRAD".
  • embedded agency
  • naturalized agency: the old term for embedded agency[1]
  • naturalized induction
  • highly reliable agent designs (HRAD)
  • the relation of all of the above to MIRI's new research agenda (which hasn't been released publicly)

References