Difference between revisions of "Comparison of terms related to agency"

From Issawiki
Jump to: navigation, search
(Created page with "{| class="wikitable" ! Term !! Opposite |- | Agent || |- | Optimizer, optimization process || |- | consequentialist || |- | expected utility maximizer || |- | goal-directed, g...")
 
 
(4 intermediate revisions by the same user not shown)
Line 5: Line 5:
 
|-
 
|-
 
| Optimizer, optimization process ||
 
| Optimizer, optimization process ||
 +
|-
 +
| Optimized ||
 
|-
 
|-
 
| consequentialist ||
 
| consequentialist ||
Line 32: Line 34:
 
* online vs offline (maybe also one-shot vs continually outputting things)
 
* online vs offline (maybe also one-shot vs continually outputting things)
 
* how well does it continue working when the environment/input changes?
 
* how well does it continue working when the environment/input changes?
* cross-domain optimization power
+
* [[general intelligence]] / cross-domain optimization power
 
* coherence (if it makes sense to assign preferences)
 
* coherence (if it makes sense to assign preferences)
 
* does it pursue omohundro's "basic AI drives"? (i.e. it is the subject of instrumental convergence)
 
* does it pursue omohundro's "basic AI drives"? (i.e. it is the subject of instrumental convergence)
 
* [https://www.greaterwrong.com/posts/DfcywmqRSkBaCB6Ma/intuitions-about-goal-directed-behavior this post] lists more formalizations for what "goal-directed" means
 
* [https://www.greaterwrong.com/posts/DfcywmqRSkBaCB6Ma/intuitions-about-goal-directed-behavior this post] lists more formalizations for what "goal-directed" means
 +
* "For example, consider Dialogic Reinforcement Learning. Does it describe a goal-directed agent? On the one hand, you could argue it doesn’t, because this agent doesn’t have fixed preferences and doesn’t have consistent beliefs over time. On the other hand, you could argue it does, because this agent is still doing long-term planning in the physical world." [https://www.greaterwrong.com/posts/NxF5G6CJiof6cemTw/coherence-arguments-do-not-imply-goal-directed-behavior#comment-dp5S2bzhiknrC2opg]
  
 
examples to check against:
 
examples to check against:
Line 60: Line 63:
 
* a system trained using imitation learning to behave similarly to another agent [https://www.greaterwrong.com/posts/9zpT9dikrrebdq3Jf/will-humans-build-goal-directed-agents]
 
* a system trained using imitation learning to behave similarly to another agent [https://www.greaterwrong.com/posts/9zpT9dikrrebdq3Jf/will-humans-build-goal-directed-agents]
 
* an agent that always twitches; see jessica's comment [https://www.greaterwrong.com/posts/NxF5G6CJiof6cemTw/coherence-arguments-do-not-imply-goal-directed-behavior/comment/ymiuDS8uY44xjXxoS here]
 
* an agent that always twitches; see jessica's comment [https://www.greaterwrong.com/posts/NxF5G6CJiof6cemTw/coherence-arguments-do-not-imply-goal-directed-behavior/comment/ymiuDS8uY44xjXxoS here]
 +
 +
==See also==
 +
 +
* [[List of terms used to describe the intelligence of an agent]]
 +
* [[Coherence and goal-directed agency discussion]]
  
 
==References==
 
==References==
Line 68: Line 76:
  
 
https://www.lesswrong.com/posts/Q4hLMDrFd8fbteeZ8/measuring-optimization-power
 
https://www.lesswrong.com/posts/Q4hLMDrFd8fbteeZ8/measuring-optimization-power
 +
 +
[[Category:AI safety]]

Latest revision as of 19:09, 27 February 2021

Term Opposite
Agent
Optimizer, optimization process
Optimized
consequentialist
expected utility maximizer
goal-directed, goal-based act-based?
pseudoconsequentialist
mesa-optimizer

parameters to check for:

  • is it searching through a list of potential answers?
  • does it have an explicit model of the world? i.e. it has counterfactuals (see drescher on subactivation)
  • can it be modeled as having a utility function?
  • can it be modeled as having a utility function and selecting an action that does an argmax over actions? (note: this does not say how the argmax is implemented, which might be via a search or via some other more "control"-like process)
  • can we take an intentional stance toward it? i.e., is it useful (so far as predicting what it will do is concerned) to model it as having intentions?
  • is it solving some sort of optimization problem? (but what counts as an optimization problem?)
  • origin: was it itself produced by some sort of optimization process?
    • eliezer's GLUT idea of "trace the improbability"
  • does it hit a small target, out of a large space of possibilities?
  • how many elements in the space of possibilities is it instantiating?
  • cost of evaluation of options/how good of a feedback we get for potential outputs
  • online vs offline (maybe also one-shot vs continually outputting things)
  • how well does it continue working when the environment/input changes?
  • general intelligence / cross-domain optimization power
  • coherence (if it makes sense to assign preferences)
  • does it pursue omohundro's "basic AI drives"? (i.e. it is the subject of instrumental convergence)
  • this post lists more formalizations for what "goal-directed" means
  • "For example, consider Dialogic Reinforcement Learning. Does it describe a goal-directed agent? On the one hand, you could argue it doesn’t, because this agent doesn’t have fixed preferences and doesn’t have consistent beliefs over time. On the other hand, you could argue it does, because this agent is still doing long-term planning in the physical world." [1]

examples to check against:

  • humans
  • "subagents" inside of humans
  • human who is modeled inside the mind of another human (e.g. for the purposes of predicting behavior)
  • evolution/natural selection
  • bottlecap
  • RL system playing Pong without an explicit model
  • tool AGI/CAIS
  • task AGI
  • KANSI
  • mindblind AI
  • targeting system on a rocket
  • single-step filter
  • chess-playing algorithm that just does tree search (e.g. alpha-beta pruning algorithm)
  • a simple feed-forward neural network (e.g. one that recognizes MNIST digits)
  • a thermostat
  • a plant
  • multi-armed bandit problem
  • Solomonoff induction (outer layer/top-level reasoning)
  • plagiarizing robot
  • a system trained using imitation learning to behave similarly to another agent [2]
  • an agent that always twitches; see jessica's comment here

See also

References

https://www.lesswrong.com/posts/ZDZmopKquzHYPRNxq/selection-vs-control

https://www.lesswrong.com/posts/rvxcSc6wdcCfaX6GZ/two-senses-of-optimizer

https://www.lesswrong.com/posts/Q4hLMDrFd8fbteeZ8/measuring-optimization-power