AlphaGo Zero Mastered Chess in Just Four Hours of Self-study
AlphaZero, a modified version of AlphaGo Zero, the improved AI that ran circles around AlphaGo that utterly trounced the world's best human Go player last year, has taught itself chess from scratch. AlphaGo Zero reached a level where it did not lose a single game in a match of 100 games against the world's strongest Chess AI, Stockfish. In rule-based domains such as Chess or even Go, this system seems to have reached apparently God-like powers. In Go, 3000 years of study by humans was surpassed in mere months of self-study. This time all Chess knowledge, both human and AI, was mastered and superseded in four hours.
The technological singularity is near. Signs of accelerating change are cropping up. The rise of cryptocurrencies is perhaps one of them. The older generations have a tendency to dismiss it as a mere bubble and them having no intrinsic value. It feels as if we were approaching a black hole beginning to rip apart the fabric of our reality.
A question about AlphaGo Zero was how generalizable the approach to reinforcement learning was beyond its mastery of Go. This latest breakthrough suggests that it will generalize, at least to any game with complete information. Now I'm wondering whether a version of AlphaGo Zero that taught itself Go would do better or worse at chess than a version trained from scratch. I hope they pursue that in future, to see whether prior learning makes the system stronger—whether it can infer any general rules of games from Go and apply them to chess.
If I remember correctly, AlphaZero did not learn Chess completely from scratch. The rules of Chess were programmed into it. However, it was a complete beginner as in knowing the rules and only the rules when it started to play against itself and it received no training other than self-play, which means the learning process was unsupervised. The original version of AlphaGo was initially trained by making it learn to predict moves by amateur players and later professionals. After that, it proceeded to play only against itself, though.
I don't believe there is much to be learned from Go that can be applied to Chess. I do have heard of much more primitive AI's that have inferred the rules of go from human play. The legality of moves (or placements) isn't particularly hard to extract from game records. Also, figuring out the difference between a winning position and a losing position (taking prisoners into account) would not take too many iterations. Of course, the system would have to recognize a winning position as a goal or have that programmed into it.
I don't know how AlphaZero was different from AlphaGo Zero other than the fact that it had preknowledge of the rules of Chess. AlphaGo had originally two neural nets. One was called Policy Network and the other Value Network. Policy Network was trained to predict candidate moves that had the highest possible winning probability. Value Network produced a number represeting black's winning probability in a position to be used as the positional value in a mini-max search. AlphaGo Zero is different in that it only has a single neural net.
Thanks for the detailed reply. My suggestion was that, perhaps, transfer learning comes into play. Would there, in any way, be benefit from applying learning of Go to Chess? If not, then in some ways to me it calls into question the value of a general artificial intelligence for specific use cases, since prior knowledge could actually hamper performance. For example, if an AlphaGo Zero that mastered Go was subsequently worse at chess than an AlphaGo Zero that didn't master Go, then it might suggest that general intelligence is a hindrance. A great related essay, if you haven't read it yet, is this recent one by François Chollet: "The impossibility of intelligence explosion."
Thanks for your thoughtful comment! I know a couple of Go players who were Chess players first. Both were fast learners in the beginning. But their advantage seemed to be limited to the beginning of their learning curve. What is common for at least human players in Go and Chess is the requirement to imagine potential sequences and hold them in one's mind when doing a mental search for the next move. I think that skill helped both of the players I knew at first.
My definition of intelligence is the capacity to learn and adapt to novel situations. The more general intelligence one possesses the faster one can acquire any new skills. Thus, in my view, it is impossible for general intelligence to be a hindrance. Too much prior knowledge can be a hindrance in learning another skill because of overapplying the knowledge at first. But that issue is not a weakness of general intelligence.
One thing to note about AlphaGo and its successors is that despite achieving superhuman levels of skill in an extremely short period of time, their efficiency in terms of energy consumption at both learning and applying their skill is very low compared to a talented human. Their training entails millions games of self-play whereas a talented player such as Ke Jie (the best human player at this time) has been able to learn the game by playing and analyzing only some tens of thousands of games. Humans are capable of learning at a much higher level of abstraction, achieving an energy efficiency advantage of several orders of magnitude. These AI architectures are, in fact, of quite low intelligence in that sense. In order for AI systems to gain superhuman general intelligence, much better architectures are needed.
Great points. I agree that, all else being equal, there’s an advantage to general intellectual capabilities, such as the ability to recognize patterns. But I’m not convinced that a general artificial intelligence competing against a narrow artificial intelligence optimized to a single purpose would always win going head-to-head on the same problem. The article I linked to makes some excellent points about how intelligence is situational and depends on the context. It also points out that people with exceptional IQs (assuming this actually measures intelligence) can actually be at a disadvantage, and don’t outperform, as a whole, people with lower IQs in the game of life. Perhaps a modular design, where a general intelligence can choose to turn on or off specific modules optimized to a purpose, combines the best of both worlds. And in fact, the brain seems to contain many such specialized modules.
I read the article. I agree that using a hypothetical general AI instead of a narrow AI to solve a problem wouldn't always be the best choice. And yes, the brain really has different brain areas specialized in different particular tasks.
I'd say that recent advances in deep convoluted neural networks have allowed for the development of astonishingly capable narrow AI systems. Developing artificial minds capable of independent existence and goal setting seems very far off at this stage, however.
Agreed. Good chat. So rare to have productive, respectful dialogue in social media. I wonder if this is sustainable here in the long-term. I hope so.
On Steem, thoughtful conversations have an advantage of monetary worth on top of sentimental and utilitarian worth. I would say that since the platform rewards us to have such interactions we will eventually get conditioned to have more of them.
great post very interesting like it
Hey follow and upvote comment