Difference between revisions of "Iterated amplification"
| Line 30: | Line 30: | ||
* [[universality]] | * [[universality]] | ||
* [[narrow value learning]] vs [[ambitious value learning]] | * [[narrow value learning]] vs [[ambitious value learning]] | ||
| + | * [[learning with catastrophes]], [[optimizing worst-case performance]] | ||
==See also== | ==See also== | ||
Revision as of 22:47, 24 February 2020
Iterated amplification (also called iterated distillation and amplification, and abbreviated IDA) is the technical alignment agenda that Paul Christiano works on.
Terminology (not necessarily about IDA, but these are some terms frequently used by Paul):
- informed oversight
- adequate oversight
- overseer
- bandwidth of the overseer
- reward engineering
- HCH, Strong HCH, Weak HCH, Humans consulting HCH
- amplification
- capability amplification
- distillation
- factored cognition, factored evaluation, factored generation
- corrigibility
- benign
- aligned
- robustness
- red teaming
- ALBA
- optimization daemons
- act-based agent vs goal-directed agent
- approval-directed agent
- steering problem
- prosaic AI
- bootstrapping
- catastrophe
- reliability amplification
- security amplification
- universality
- narrow value learning vs ambitious value learning
- learning with catastrophes, optimizing worst-case performance
