Difference between revisions of "Asymmetric institution"
(→Concerns) |
|||
| (13 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
An '''asymmetric institution''' is an institution or mechanism which gives influence asymmetrically based on some property. | An '''asymmetric institution''' is an institution or mechanism which gives influence asymmetrically based on some property. | ||
| + | |||
| + | What is interesting about asymmetric institutions is that (1) they selectively give influence to those with particular qualities, and (2) they constrain the influence one can gain from/within that institution to only be usable in certain ways. | ||
==Examples of asymmetric institutions== | ==Examples of asymmetric institutions== | ||
| − | * science | + | * science: gives status/influence to researchers who generate hypotheses that third parties can replicate; this means that most people's pet theories do not be promoted by science |
| − | * betting | + | * betting: people with better epistemics win money by betting. The problem is that it doesn't stop people from saying wrong things after they've built a track-record (though if they keep betting then eventually they will be "found out"). |
| − | * debate (of the civil/rational kind, not a shouting match) | + | * debate (of the civil/rational kind, not a shouting match): |
| − | * charity evaluation based on some strict criteria | + | * charity evaluation based on some strict criteria: [[GiveWell]] gains influence by evaluating charities, but it cannot promote any charity it wishes; it can only promote e.g. charities whose interventions perform well under a randomized controlled trial |
| + | * Wikipedia's policies like notability and reliable source | ||
| + | * Criteria for acceptable questions on Stack Exchange sites | ||
| + | * banning discussions of politics on sites like [[LessWrong]] (by keeping politics out, it has a better shot of creating and enforcing good norms around discussion, but it cannot then go ahead and discuss politics after creating this environment?) | ||
==Examples of symmetric institutions== | ==Examples of symmetric institutions== | ||
| + | |||
| + | Symmetric institutions/weapons have the property that the effectiveness does not depend on who is using it (although there are things like offense vs defense issues). | ||
* violence | * violence | ||
| Line 15: | Line 22: | ||
* persuasion tactics | * persuasion tactics | ||
* info warfare | * info warfare | ||
| + | * coordination mechanisms like Kickstarter | ||
| + | * [[Donor lottery]] | ||
| + | |||
| + | ==Concerns== | ||
| + | |||
| + | Even asymmetric institutions aren't ''that'' good. Some thoughts here: | ||
| + | |||
| + | * lots of low hanging fruit remain unpicked / inability to resolve coordination failures (e.g. look at all of the criticisms of science in recent years) | ||
| + | * even asymmetric institutions can be corrupted | ||
| + | * halo effect to areas where these institutions don't have expertise | ||
| + | * the asymmetric weapon can sometimes be used against "good" things, e.g. using reason to argue against avoidance to talk on phones; more generally, the accuracy of beliefs is not monotonic in the amount of time a person spends reasoning about a topic<ref>https://slatestarcodex.com/2019/06/06/asymmetric-weapons-gone-bad/</ref> | ||
==External links== | ==External links== | ||
| Line 20: | Line 38: | ||
* [https://ea.greaterwrong.com/posts/4oGYbvcy2SRHTWgWk/improving-the-future-by-influencing-actors-benevolence/comment/vtJg9v9PFdTiWfPgy Comment] by [[Carl Shulman]] on "Improving the future by influencing actors' benevolence, intelligence, and power" | * [https://ea.greaterwrong.com/posts/4oGYbvcy2SRHTWgWk/improving-the-future-by-influencing-actors-benevolence/comment/vtJg9v9PFdTiWfPgy Comment] by [[Carl Shulman]] on "Improving the future by influencing actors' benevolence, intelligence, and power" | ||
* https://slatestarcodex.com/2017/03/24/guided-by-the-beauty-of-our-weapons/ | * https://slatestarcodex.com/2017/03/24/guided-by-the-beauty-of-our-weapons/ | ||
| + | * sort of related: https://www.facebook.com/duncan.sabien/posts/3980522265315794 | ||
| + | |||
| + | ==References== | ||
| + | |||
| + | <references/> | ||
[[Category:AI safety]] | [[Category:AI safety]] | ||
Latest revision as of 21:51, 12 March 2021
An asymmetric institution is an institution or mechanism which gives influence asymmetrically based on some property.
What is interesting about asymmetric institutions is that (1) they selectively give influence to those with particular qualities, and (2) they constrain the influence one can gain from/within that institution to only be usable in certain ways.
Contents
Examples of asymmetric institutions
- science: gives status/influence to researchers who generate hypotheses that third parties can replicate; this means that most people's pet theories do not be promoted by science
- betting: people with better epistemics win money by betting. The problem is that it doesn't stop people from saying wrong things after they've built a track-record (though if they keep betting then eventually they will be "found out").
- debate (of the civil/rational kind, not a shouting match):
- charity evaluation based on some strict criteria: GiveWell gains influence by evaluating charities, but it cannot promote any charity it wishes; it can only promote e.g. charities whose interventions perform well under a randomized controlled trial
- Wikipedia's policies like notability and reliable source
- Criteria for acceptable questions on Stack Exchange sites
- banning discussions of politics on sites like LessWrong (by keeping politics out, it has a better shot of creating and enforcing good norms around discussion, but it cannot then go ahead and discuss politics after creating this environment?)
Examples of symmetric institutions
Symmetric institutions/weapons have the property that the effectiveness does not depend on who is using it (although there are things like offense vs defense issues).
- violence
- shouting match
- spreading viral content
- persuasion tactics
- info warfare
- coordination mechanisms like Kickstarter
- Donor lottery
Concerns
Even asymmetric institutions aren't that good. Some thoughts here:
- lots of low hanging fruit remain unpicked / inability to resolve coordination failures (e.g. look at all of the criticisms of science in recent years)
- even asymmetric institutions can be corrupted
- halo effect to areas where these institutions don't have expertise
- the asymmetric weapon can sometimes be used against "good" things, e.g. using reason to argue against avoidance to talk on phones; more generally, the accuracy of beliefs is not monotonic in the amount of time a person spends reasoning about a topic[1]
External links
- Comment by Carl Shulman on "Improving the future by influencing actors' benevolence, intelligence, and power"
- https://slatestarcodex.com/2017/03/24/guided-by-the-beauty-of-our-weapons/
- sort of related: https://www.facebook.com/duncan.sabien/posts/3980522265315794
