AlphaGo

AlphaGo and its successor AlphaGo Zero are used to make various points in AI safety.

• Rapid capability gain
• single group pulling ahead
• a single architecture / basic AI technique working for many different games
• (for AlphaGo Zero) comparison to Paul Christiano's iterated amplification

Eliezer's commentary

Date	Initial segment + link	Description
2016-01-27	People occasionally ask me about signs that the remaining timeline might be short. It's *very* easy for nonprofessionals to take too much alarm too easily. Deep Blue beating Kasparov at chess was *not* such a sign. Robotic cars are *not* such a sign.
2016-02-08	I have one bet on at 2:3 against AlphaGo winning against Sedol in March - they get my $667 if AlphaGo wins, I get their $1000 if AlphaGo loses
2016-02-29	This suggests that AlphaGo beating Sedol in March might not be nearly as out-of-character fast progress as I thought
2016-03-08	With regards to tonight's match of Deepmind vs. Sedol, an example of an outcome that would indicate strong general AI progress would be if a sweating, nervous Sedol resigns on his first move, or if a bizarre-seeming pattern of Go stones causes Sedol to have a seizure.
2016-03-09	Second match ongoing. #AlphaGo just made a move that everyone is saying nobody else would have played. #Sedol walked out of the room with his clock running, presumably to think about it.
2016-03-10	It's possible that, contrary to hopeful commentators, #AlphaGo is not actually enriching the Go game for humans
2016-03-11	(Long.) As I post this, AlphaGo seems almost sure to win the third game and the match.
2016-03-13	And then AlphaGo got confused in a way no human would and lost its 4th game
2016-03-13	Okay, look, to everyone going "Aha but of course superhuman cognition will always be bugged for deep reason blah": Please remember that machine chess *is* out of the phase where a human can analyze it psychologically without computer assistance
2017-10-19	AlphaGo Zero uses 4 TPUs, is built entirely out of neural nets with no handcrafted features, doesn't pretrain against expert games or anything else human, reaches a superhuman level after 3 days of self-play, and is the strongest version of AlphaGo yet

• https://www.facebook.com/yudkowsky/posts/10154120081504228
• https://www.facebook.com/yudkowsky/posts/10154009668254228
• Alpha Zero: https://www.facebook.com/yudkowsky/posts/10155992246384228
• https://www.greaterwrong.com/posts/shnSyzv4Jq3bhMNw5/alphago-zero-and-the-foom-debate
  • Hanson reply: https://www.greaterwrong.com/posts/D3NspiH2nhKA6B2PE/what-evidence-is-alphago-zero-re-agi-complexity