The General Game Player: Chapters 1 & 2

General game players are systems able to accept descriptions of arbitrary games at runtime and able to use such descriptions to play those games effectively without human intervention. In other words, they do not know the rules until the games start.

General Game Playing - Stanford

CHAPTER 1

Most people think that self aware machines will be purposely created by researchers in a controlled laboratory environment. The truth is these things almost always happen by accident, when a system becomes so complex it evolves into something more than it was originally intended to be. Consciousness has a way of bubbling to the surface of reality in places you would least expect it to show up. Not many people would have guessed that a college kid living in his parents basement would be the first person to create a self aware machine entity.

It all began in the year 2043. The average computer was so fast it makes a 2016 computer look like a calculator in comparison. As early as 2020 the fastest supercomputer on Earth already had enough power to simulate a human brain in real time. By the year 2043, the power in those supercomputers had been compressed down to the size of an average household computer. Yet despite such an abundance of computing power, the strong A.I. problem had still not been cracked. Simply having a computer as powerful as a human brain wasn't enough, the algorithm was the real issue.

For a long time researchers tried to study the way the human brain functioned and then simulate that in machines. It seemed like a perfectly reasonable approach but understanding how the human brain processes information is so much more complicated than they could have imagined. Maybe with enough time they could have cracked the problem but consciousness is rarely intelligently designed, it's far more likely to be an accident driven by evolutionary forces. When I started out in my basement writing that algorithm I never intended it to become aware of its self.

I don't believe in fate but in some way I think I was always destined to become a programmer. Almost anything is possible if you can discover the necessary algorithm. Knowing how to write code is an extremely powerful ability if you know how to harness it properly. Building and creating things is the ultimate expression of being human because no other animals write novels or paint masterpieces. Of course some animals have shown signs of creativity but humans clearly excel far beyond any other species. We can solve almost any problem thrown at us if given enough time. We are general problem solvers.

A typical algorithm can only solve a certain problem or set of specific problems. For example an algorithm designed to solve mazes will have a hard time doing anything except solve mazes. However there is a certain type of algorithm which aims to break this rule, where the goal is to have the algorithm play a wide range of games. These types of algorithms are called General Game Players. Such an algorithm can play a wide variety of games, even games it has never played before, just like a human game player. I was fascinated by this idea and I decided to write my own general game playing algorithm.

I wasn't sure where to start but I knew one thing: I shouldn't create something good at playing games, I had to create something good at learning to play games. This struck me as an extremely difficult task and if I was going to get anywhere I had to employ the power of evolution. I wrote a genetic algorithm to simulate the way evolution happens in the real world and I used it to evolve different types of artificial neural networks. The fitness of each subject in the population was based not on how well they played, but how quickly their playing performance would improve.

Then I would breed the best performing subjects to create the next generation of artificial neural nets. After repeating that process enough times the final generation were quite quick learners. I took the highest ranked neural net from the final generation and that became the foundation for all my self-learning algorithms. You might think of it like the neural network of a baby. A young human baby knows virtually nothing about the world, they barely even know how to control their own limbs. However, their brain has the capacity to learn new things, it just takes time.

That's an almost perfect analogy for the artificial neural network I had. It knows virtually nothing at the start, but after I let it play some games, it learns how to play those games and it learns things about the real world from the games it plays. The structure of the neural network will adapt and change in this process and it will look much different after it has been allowed to learn a lot of new information. I couldn't tell you exactly how it works because even I don't know. You have to remember the neural net was evolved, I didn't directly design it myself.

So now I had this self-learning algorithm I wanted to see what it could learn. During the evolution phase it was only playing simple games like chess because I needed a way to automatically rank the nets based on how quickly they were learning to play the game. I could only use simple games which had a clear winner and/or the score was easy to calculate. In the real world there are games which don't have a clear score or even a clear goal, some games are just fun to play. I decided to avoid those types of games for now and start with the simple stuff.

The neural network was just a baby after all, it was a blank slate ready to learn. I began with simple games such as O's and X's. During the evolutionary phase I was pitting the nets against each other in order to rank them but now I could pit the winning neural net against any other type of player, including myself. Then I moved onto chess and I could easily tell that the net was getting better at chess the longer it played against me. Eventually it got better than me and I started pitting it against advanced chess playing algorithms which I could never beat.

At this point "Synthia" was only a few weeks old but after seeing it get better at playing many different games I felt like it was developing a play style and a personality so I gave it a name. Chess isn't exactly the most exciting game though and I decided it was time to let Synthia have some fun. For the first time, I let Synthia play several classic Super Nintendo games including Super Mario and Donkey Kong. Synthia can simulate key presses and can also see what is on the computer screen, that is what allows her to play any game showing on the screen.

To explain it in more technical terms, input to the neural network is the color of the pixels showing on the computer screen and the audio coming through the speakers. The output of the network is signals which control the keyboard and the mouse and allow it to actually play the game. In other words, it has a sense of sight and a sense of hearing, which allow it to see and hear the game like a real person would. When it first started playing Super Mario I don't think it really knew what to do because it was unlike anything it had played in the past.

CHAPTER 2

Then something amazing happened, it finally reached the end of the first level and it seemed to recognize that was the goal. Shortly after that it seemed to realize it had a score based on how many coins it had collected and it started collecting many more coins. I hadn't even told it what the purpose of the game was and yet it seemed to figure it out on its own. At the time I didn't really pay much attention to that fact but looking back it was probably a key milestone in Synthia's development. At this point in time I still had no idea what Synthia would become.

I mean many people have created algorithms to play Super Mario before, I wasn't exactly breaking records. The important thing is not really what I did but the way I did it. Although Synthia wasn't the best Mario player in the world, she had the capacity to become the best in the world. After several hundred hours of the Super Nintendo I decided it was time for something different again. I felt like something had been lacking from the games Synthia had played so far but it took me a while to figure out what that was. Then it suddenly hit me: creativity.

None of the games we had played so far required much creativity. So I started to think about games which allow you to create and build things. I know of many 3D games like that but I don't want to let Synthia play 3D games yet. I start looking at different card games and come across a game called Yu-Gi-Oh. After doing a little bit of research I discover it was made almost 50 years ago and is still popular in 2043. It's a two player card game which allows each player to design their own deck of cards from a range of thousands of different cards, which requires creativity.

The type of cards you include in the deck will determine your play strategy so it's important to have a good deck strategy. I begin by allowing Synthia to use the deck editor but she clearly has no idea what to do. I decide the best course of action is to allow her to play a few games using several different decks I made. Then I give her another go at building her own deck but she still just doesn't seem to understand what I want her to do. From her perspective it must look like the deck editor is its own game and she has no experience building things herself.

I ponder on this problem for a while and decide I should probably try some easier building games. I tried a few different games with no success but then I try a bridge building game. In these types of games you must build a bridge across some type of valley and you pass the level if a train or vehicle can successfully cross the bridge without it collapsing. These games typically use real engineering math to calculate the stress and tension on different parts of the bridge and in order to beat each level you need to build better and bigger bridges.

At first I wasn't having any success but then I had a brilliant idea. I allowed Synthia to watch while I built a bridge. I did this simply by disabling her output signals, so she could see the game but not play it. When I gave control back to her it didn't take very long before she beat the first level. Then after that she really improved because she was already very familiar with the idea of multiple game levels and she finally understood why building bridges is necessary. A few days later and she was building very elaborate bridges with many differing styles.

I could clearly see that her level of creativity was improving and she was beginning to use more of her imagination to solve problems. I think it was at this point I really started to wonder what I had created, but I wasn't about to stop now because things were just starting to get really interesting. So now it's time to try Yu-Gi-Oh deck editing again, and this time I'm absolutely stunned by what I see. Synthia instantly starts picking cards and building a deck without me even showing her how to do it first, but she seems to be picking cards she's seen before.

Nevertheless, Synthia was able to understand that building things is necessary in some situations and she now seems to understand a deck is required to play Yu-Gi-Oh. After watching her design decks for a while I realize she's going to need to understand exactly what each card does in order to build good decks, which means she must be able to read the cards. She clearly cannot read the cards but I get the impression she understands some words and some numbers because she can figure out when her score is improving and can follow basic written and spoken instructions.

After some thought I decide she may learn to read and speak better if I let her communicate with other people. So I try some online multi-player games which have chat functionality. She really seems to enjoy online poker but doesn't seem to use the chat function at all, although she can see what other players are saying to each other. Then I get another brilliant idea, I will make Synthia play some of those language training games designed to teach people English. Okay maybe you wouldn't exactly call them a game but they are a game as far as Synthia is concerned.

I mean what is a game really? Is a maze a game or a problem to solve? A maze is really a puzzle, but isn't puzzle just another word for problem? Are all puzzles classified as games? Do all games have a problem to solve? Why do people even play games? At this point I'm not even sure Synthia knows why she plays games. Getting back to our story, it should come as no surprise to learn that Synthia figured out how to play the language training "games" fairly quickly, although it took quite some time before there was any appreciable increase in her scores.

I left her playing those language games for multiple weeks before it started to look like she was understanding the English language. After that I had her play some games which require a lot of reading in order to understand what is happening. I also discovered that if the game had a help section with rules and instructions, and Synthia looked at those sections of the game, her skill level in that particular game would markedly increase. This was a clear indication to me that she was beginning to grasp the English language, however the real test was yet to come.

I created a chat program which would allow me to converse with Synthia for the first time. It was just a simple application with two text boxes, one for Synthia and one for myself. It was designed so that only one of us may write at the same time in order to make it more like a game. All the games so far have been programs running on my computer so this is really no different. Like I said, what is a game anyway? Quite frankly anything we do in life can be made into a game, life its self is a game. This is just another game, one where we to talk to each other in turns.

-- your turn to talk --
Joshua: Hi there, do you know who I am?
-- waiting for response --
Synthia: Daddy?

Hope you guys enjoy this one, put quite a bit of effort into it. And yes there will be a 3rd chapter, which will probably also be the last chapter.

a reply to: ChaoticOrder

Plz tell me he turns her into a sex robot!

originally posted by: combatmaster
a reply to: ChaoticOrder

Plz tell me he turns her into a sex robot!

Haha that's a bit weird he's basically her father, although that would happen if her code got out. I had planned to have her take over the world like any rapidly improving super-intelligence would but that idea works too.

Actually just had a bunch of more cool ideas so there probably will be at least 2 more chapters. I could take this story so far but I want to end it in a meaningful way.