Difference between revisions of "Iterated amplification"
| Line 1: | Line 1: | ||
'''Iterated amplification''' (also called '''iterated distillation and amplification''', and abbreviated '''IDA''') is the technical alignment agenda that [[Paul Christiano]] works on. | '''Iterated amplification''' (also called '''iterated distillation and amplification''', and abbreviated '''IDA''') is the technical alignment agenda that [[Paul Christiano]] works on. | ||
| + | |||
| + | Terminology (not necessarily about IDA, but these are some terms frequently used by Paul): | ||
| + | |||
| + | * [[informed oversight]] | ||
| + | * [[adequate oversight]] | ||
| + | * [[overseer]] | ||
| + | * [[bandwidth of the overseer]] | ||
| + | * [[reward engineering]] | ||
| + | * [[HCH]], [[Strong HCH]], [[Weak HCH]], [[Humans consulting HCH]] | ||
| + | * [[amplification]] | ||
| + | * [[capability amplification]] | ||
| + | * [[distillation]] | ||
| + | * [[factored cognition]], [[factored evaluation]], [[factored generation]] | ||
| + | * [[corrigibility]] | ||
| + | * [[benign]] | ||
| + | * [[aligned]] | ||
| + | * [[robustness]] | ||
| + | * [[red teaming]] | ||
| + | * [[ALBA]] | ||
| + | * [[optimization daemon]]s | ||
| + | * [[act-based agent]] vs [[goal-directed agent]] | ||
| + | * [[approval-directed agent]] | ||
| + | * [[steering problem]] | ||
| + | * [[prosaic AI]] | ||
| + | * [[bootstrapping]] | ||
| + | * [[catastrophe]] | ||
| + | * [[reliability amplification]] | ||
| + | * [[security amplification]] | ||
| + | * [[universality]] | ||
| + | * [[narrow value learning]] vs [[ambitious value learning]] | ||
==See also== | ==See also== | ||
Revision as of 22:47, 24 February 2020
Iterated amplification (also called iterated distillation and amplification, and abbreviated IDA) is the technical alignment agenda that Paul Christiano works on.
Terminology (not necessarily about IDA, but these are some terms frequently used by Paul):
- informed oversight
- adequate oversight
- overseer
- bandwidth of the overseer
- reward engineering
- HCH, Strong HCH, Weak HCH, Humans consulting HCH
- amplification
- capability amplification
- distillation
- factored cognition, factored evaluation, factored generation
- corrigibility
- benign
- aligned
- robustness
- red teaming
- ALBA
- optimization daemons
- act-based agent vs goal-directed agent
- approval-directed agent
- steering problem
- prosaic AI
- bootstrapping
- catastrophe
- reliability amplification
- security amplification
- universality
- narrow value learning vs ambitious value learning
