On the Road to Artificial General Intelligence • Danny Lange on game engines for AI/ML training

Great talk on the use of game engines (virtual worlds) for AI/ML agent training. Even if you are already knowledgeable about this topic, the examples he shows will be useful to guide your intuition as to what is possible, what is easy/hard with current technology and methods. Don't miss the puppies :-)

Conceptually, I would say there is not much new since the early successes with simpler (e.g., Atari) games. See papers/talks by Schmidhuber in this 2014 post. IIRC, the concept of curiosity: seeking "surprise" = large chunks of information = large model updates, was formulated already some time ago. 

One thing that is new is the use of physics engines in the virtual worlds - i.e., the AI has to deal with dynamics as in the real world. It seems to me that routine task automation, such as in manufacturing, is not that much harder than what is being done here in game worlds with good physics engines. (Note I'm not referring to the mechanical engineering or physical robotics challenges, which could be significant, just the ML part.) Replacement of humans in many routine tasks seems now a matter of economics tradeoffs and application of known technologies rather than big breakthroughs.

I've always thought we'd get to AGI after consuming a lot of FLOPS training agents in increasingly realistic virtual worlds. Of course, this makes one wonder whether we ourselves exist in a simulation ;-)

Meet the Bot: OpenAI and Dota 2

OpenAI has created a Dota 2 bot that plays at the level of human professionals. Humans can look forward to coexistence with increasingly clever AIs in both virtual and real world settings. See also Robots taking our jobs.

OpenAI: Dota 1v1 is a complex game with hidden information. Agents must learn to plan, attack, trick, and deceive their opponents. The correlation between player skill and actions-per-minute is not strong, and in fact, our AI’s actions-per-minute are comparable to that of an average human player.

Success in Dota requires players to develop intuitions about their opponents and plan accordingly. In the above video you can see that our bot has learned — entirely via self-play — to predict where other players will move, to improvise in response to unfamiliar situations, and how to influence the other player’s allied units to help it succeed.

About the game ("Defense of the Ancient").

Wikipedia: Dota 2 is played in matches between two teams of five players, with each team occupying and defending their own separate base on the map. Each of the ten players independently controls a powerful character, known as a "hero", who all have unique abilities and styles of play. During a match, a player and their team collects experience points and items for their heroes in order to fight through the opposing team's heroes and other defenses. A team wins by being the first to destroy a large structure located in the opposing team's base, called the "Ancient".

Related: this is a nice recent interview with Demis Hassabis of Deep Mind. He talks a bit about Go innovation resulting from AlphaGo.

Strategic War (with cards)

War is a simple card game played by children. The most common version does not require decisions, so it's totally deterministic (outcome is determined) once the card order in each deck is fixed. Nevertheless it can be entertaining to watch/play: there are enough fluctuations to engage observers, mainly due to the treatment of ties. The question of how to determine the winner from the two deck orderings (without actually playing the entire game, which can take a long time) was one of the first aspects of computability / predictive modeling / chaotic behavior I thought about as a kid. This direction leads to things like classification of cellular automata and the halting problem.

My children came home with a version designed to teach multiplication -- each "hand" is two cards, rather than the usual single card, and the winner of the "battle" is the one with the higher product value of the two cards (face cards are removed). I thought this was still too boring: no strategy (my kids understood this right away, along with the meaning of deterministic; this puts them ahead of some philosophers), so I came up with a variant that has been quite fun to play.

Split the deck into red and black halves, removing face cards. Each hand (battle) is played with two cards, but they are chosen by each player. One card is placed face down simultaneously by each player, and the second cards played are chosen after the first cards have been revealed (flipped over). Winner of most hands is the victor.

This game ("strategic war") is simple to learn, but complex enough that it involves bluffing, calculation, and card counting (keeping track of which cards have been played). A speed version, with, say, 10 seconds allowed per card choice, goes very fast.

Has anyone seen heard of this game before? It's a bit like repeated two card poker (heads up), drawing from a fixed deck. Note the overall strength of hands for each player (combined multiplicative value of all cards) is fixed and equal. Playing strong hands early means weaker hands later in the game. The goal is to win each hand by as small a margin as possible.

Are there strategies which dominate random play (= select first card at random, second card from range not exceeding highest card required to guarantee a win)?


Don't forget to respond to the reader survey.

Poker prodigies

Video game training allows younger players to take on a dozen tables at a time in online poker? This seems harder than a simul in chess, where a master can "chunk" the position on the board quickly. If there are many players at each of those dozen tables the poker genius has to keep track of perhaps 30 or 50 hands at a time, and the psychological profile of each of those players. The guy in the article seems to be doing this with the help of software. How long until it's all bots, all the time?

NYTimes: ... Within 18 months, Cates went from routinely losing at local $5 games to winning at the highest stakes of online poker for anywhere between $10,000 and $500,000 per night. In 2010, his reported $5.5 million in online earnings was more than $1 million higher than the nearest competitor. Unlike other young poker millionaires who make the bulk of their money by winning televised tournaments — a proposition that, because of the high number of players and the unpredictability of their actions, involves roughly the same amount of luck as winning a small lottery — Cates earned his stake by grinding, the term used to describe the process of pressing a skill advantage over an extended period of time. Because poker is a game of high variance, where a significant difference in ability can be mitigated by a bad run of cards, a player’s Expected Value (E.V.) must be actualized over thousands of hands. Every year, a few dozen kids go on hot streaks and take a shot at the big time. Almost invariably, these kids are eventually ground down by higher caliber players. What made Cates’s run different wasn’t his total winnings or the speed with which he earned his millions. What caught the attention of the poker world was that the 20-year-old top online earner of 2010 won almost all of his money in head-to-head confrontations with poker’s elite.

The gospel of E.V. that keeps the poker hierarchy in order was shaken. Cates had taken on all comers in 2010, including highly publicized matches against top-flight pros like Phil Ivey, Patrik Antonius, Ilari (Ziigmund) Sahamies and his fellow young gun Tom (durrrr) Dwan. Each of these men has helped turn poker into a multimillion-dollar celebrity enterprise. Each ranks among the 20 or so most recognized players in the world. And in each of his matches with poker royalty, Cates came out hundreds of thousands of dollars ahead.

... The vast sums of money shuttled among the accounts of these young professionals — and the shocking aggressiveness and recklessness with which they played — deepened the divide between the young online players and the older guard who earned their millions when poker was still a game played by men sitting around a table. Since the rise of online poker in the early 2000s, every principle of the game, every lesson learned over hundreds of thousands of hours of play, every simple credo uttered in some old Western gambling movie — all those tersely stated, manly things that made up the legend of poker — has been picked apart and, for the most part, discarded.

Patience is no longer rewarded. If an 18-year-old online whiz can play 12 hands at once, then by his 19th birthday, he is no less experienced than a career gambler who has sat for a dozen years at the big-money table at the Bellagio. It didn’t take long before the young players began crushing established gamblers online, and the question rang out across the poker world: How were these kids, many of whom were too young to set foot inside a casino, outsharking the sharks?

In Command and Conquer, the video game that consumed much of Cates’s childhood, a player leads an army into a real-time battle. The combat units are vaguely futuristic and highly specialized. Success depends on the efficiency with which a player can build his resources and the speed with which he can deploy them. It is a difficult game to play and an even harder game to master. The best players develop a predatory instinct for detecting the exact moment when an opponent has weakened. High-end strategy combines lightning-fast reflexes, unabashed aggression and razor-thin resource management. Reckoning comes by way of particle cannon. By the age of 15, Cates told me repeatedly, he was one of the world’s best Command and Conquer players.

Phil Gordon, a 40-year-old poker professional who has won $3 million in tournaments, written three best-selling books and hosted several TV shows, including Bravo’s Celebrity Poker Showdown, says he believes that the early and immersive training offered by video games, paired with online poker’s increasing space in the mainstream, has laid out a practice ground for a militia of young, fearless, invincible players. “The prototypical successful young gun is fast and unpredictable,” Gordon says. “Those traits make them nearly impossible to beat, especially when playing at warp speeds. The manual dexterity required to play 12 or even 16 or 20 tables at one time is enormous. The mental dexterity required to play well while making that number of decisions in a very short amount of time is even more impressive. Many of the video games the kids grew up with like Command and Conquer or Call of Duty required a similar dexterity and gave these kids a leg up — the more tables they could play accurately, the more decisions they got to make, and the quicker they were able to learn.”

Then there’s the fact that high-stakes poker rewards aggression. A player who cannot fire off a bluff because he is worried about his daughter’s private-school tuition will be quickly run over by the players who don’t have such concerns. While heightened dexterity, comfort with snap decisions and the stamina gained from years spent sitting in front of a computer screen give the young online pro an edge over his older counterpart, the greatest benefit borne from a life spent playing video games lies somewhere in the strange, disconnected relationship between what is simulated and what is real. The armies of Command and Conquer do not suffer real casualties. An unsuccessful session of Minesweeper does not result in the loss of a leg.

In online poker, lost money registers only as debits in the player’s offshore account. When a player loses a million-dollar pot, the action plays out in cartoon animation.

“Most of us young kids who play at nosebleed stakes don’t really have any clear idea about the actual value of the money we win or lose,” Cates says. “Most of us see the money more as a points system. And because we’re all competitive, we want to have the highest score. But really, we don’t know what making $400,000 or losing $800,000 means, because we don’t have families or whatever. This blind spot gives us the freedom to always make the right move, regardless of the amount at stake, because our judgment isn’t clouded by any possible ramifications.”

It is unclear whether Cates actually does understand that the money is real. On the second day of my visit, we took a trip to Best Buy. Cates had grown bored of playing poker and wanted to buy a video game. As we stood in the PS3 aisle, discussing which games looked good, I asked him if he had ever walked into a store like Best Buy — or perhaps a car dealership — and thought to himself, Hey, I can buy out this entire place. Cates smiled sheepishly. He said: “I’m not really into material wealth. Plus, I need to save up some more money. My fiscal goal for 2011 is to reach $10 million in liquid cash.” I asked what the difference might be between $5 million and $10 million, especially for a 21-year-old whose relative spending habits sit somewhere on the line between modest and monastic. He explained: “You can do anything with $10 million. Like, you can buy a house and still have around $5 million left over.”

Starship Troopers

This commercial for the video game Halo 3: ODST (Orbital Drop Shock Troops) comes as close as anything I've seen to capturing the flavor of the Heinlein novel Starship Troopers. Forget about the 1997 movie directed by Paul Verhoeven; it's pure camp. Some might classify the original novel as pulp (see excerpt below), but I think it is much more than that.

The ODST project started as a Halo movie project involving Peter Jackson of Lord of the Rings fame. Perhaps this explains the cinematic qualities. (I'm not a gamer; I just noticed the commercial while watching Iowa beat Penn State last night and got interested in what seemed to be an obvious Starship Trooper influence.)

Wikipedia: Starship Troopers is a novel set in an unspecified time in the future, although probably not in the far-flung future. It chronicles the experiences of Juan "Johnnie" Rico, the story's narrator, during his enlistment and training in the Mobile Infantry, and his participation in an interstellar war between the Terran Federation and the Arachnids (referred to as "the Bugs") of Klendathu. It is narrated as a series of flashbacks — one of only a few Heinlein novels to use that narrative device[11] — and contains large sections of character discussion and introspection, as well as exposition, all meant to detail the political theory and philosophical beliefs underlying the society that Juan Rico lives in.

The novel opens with Rico aboard the corvette Rodger Young, platoon transport for "Rasczak's Roughnecks", about to embark on a raid against the planet of the "Skinnies", allies of the Arachnids. We learn that he is a "cap" trooper (called this because they are dropped in capsules from the ship in orbit toward their drop zones) in the Terran Federation's Mobile Infantry (M.I.). The raid itself, one of the few instances of actual combat in the novel, is relatively brief: the Mobile Infantrymen land in the capital city, destroy their targets while trying to avoid unnecessary Skinny casualties, and withdraw, suffering three injured (one fatal) in the process.

The story then flashes back to Rico's graduation from high school and his decision to sign up for Federal Service, rather than attend Harvard University, over the objections of his wealthy father. This is the only chapter that describes Rico's civilian life, and most of it is spent recording the monologues of two people: retired Lt. Colonel Jean V. Dubois, Rico's school instructor in the subject of History and Moral Philosophy, and Fleet Sergeant Ho, a recruiter for the Armed forces of the Terran Federation.

Many readers have felt that Dubois serves as a stand-in for Heinlein throughout the novel. He delivers what is probably the book's most famous soliloquy, on how violence "has settled more issues in history than has any other factor."[12] Fleet Sergeant Ho offers a separate angle on military service to that of Dubois. (Ho has prostheses for several limbs, but does not wear them on duty at the front door of the federal building. This is calculated to remind applicants of the real risks of service, and to weed out those not willing to take such risks in the service of the Federation).

Interspersed throughout the book are other flashbacks to Rico's high school History and Moral Philosophy course, which describe how, in the Terran Federation, the rights of a full Citizen (to vote, and hold public office) must be earned through voluntary Federal service. However, the franchise cannot be exercised until after honorable discharge from the Service, which means that active members of the Service cannot vote. Those residents who opt not to perform Federal Service retain the other rights generally associated with a modern democracy (e.g. free speech, assembly, etc.), but cannot vote or hold public office. This structure arose ad hoc after the collapse of the 20th century Western democracies, brought on by both social failures at home and military defeat by the Chinese Hegemony overseas (i.e. looking forward into the late 20th century from the time the novel was written in the late 1950s).[13]

After enlisting in the Mobile Infantry, Rico is assigned to boot camp at Camp Arthur Currie. Five chapters are spent exploring Rico's training, under the guidance of his chief instructor, Career Ship's Sergeant Charles Zim. Boot camp is deliberately so rigorous that fewer than ten percent of the recruits complete basic training; the rest either resign (with no penalty, save never being able to vote); are expelled (likewise); are given medical discharges (which may however be refused); or assigned to lesser duties (enabling them to vote after their service is finished); or die in training. ...

Here's the version of the book I've had since I was a kid:



Yes, I enjoyed a misspent youth, during which I read all of Heinlein's books and even played the Avalon Hill boardgame version of Starship Troopers.

Give me a few Sergeant Zim's and I'll conquer the galaxy! :-)

Bootcamp for the Mobile Infantry: ... But exercise will keep you warm and they saw to it that we got plenty of that. The first morning we were there they woke us up before daybreak. I had had trouble adjusting to the change in time zones and it seemed to me that I had just got to sleep; I couldn't believe that anyone seriously intended that I should get up in the middle of the night. But they did mean it. A speaker somewhere was blaring out a military march, fit to wake the dead, and a hairy nuisance who had come charging down the company street yelling, "Everybody out! Show a leg! On the bounce!" came marauding back again just as I had pulled the covers over my head, tipped over my cot and dumped me on the cold hard ground. It was an impersonal attention; he didn't even wait to see if I hit. Ten minutes later, dressed in trousers, undershirt, and shoes, I was lined up with the others in ragged ranks for setting-up exercises just as the Sun looked over the eastern horizon.

Facing us was a big broad-shouldered, mean-looking man, dressed just as we were -- except that while I looked and felt like a poor job of embalming, his chin was shaved blue, his trousers were sharply creased, you could have used his shoes for mirrors, and his manner was alert, wide-awake, relaxed, and rested. You got the impression that he never needed to sleep -- just ten-thousand-mile checkups and dust him off occasionally. He bellowed, "C'pnee! Atten . . . shut! I am Career Ship's Sergeant Zim, your company commander. When you speak to me, you will salute and say, `Sir' -- you will salute and `sir' anyone who carries an instructor's baton -- " He was carrying a swagger cane and now made a quick reverse moulinet with it to show what he meant by an instructor's baton; I had noticed men carrying them when we had arrived the night before and had intended to get one myself -- they looked smart. Now I changed my mind. " -- because we don't have enough officers around here for you to practice on. You'll practice on us. ...

Zim turned back to the rest of us, still shivering at attention. He walked up and down, looked us over, and seemed awfully unhappy. At last he stepped out in front of us, shook his head, and said, apparently to himself but he had a voice that carried: "To think that this had to happen to me!" He looked at us. "You apes -- No, not `apes'; you don't rate that much. You pitiful mob of sickly monkeys . . . you sunken-chested, slack-bellied, drooling refugees from apron strings. In my whole life I never saw such a disgraceful huddle of momma's spoiled little darlings in -- you, there! Suck up the gut! Eyes front! I'm talking to you!" I pulled in my belly, even though I was not sure he had addressed me. He went on and on and I began to forget my goose flesh in hearing him storm. He never once repeated himself and he never used either profanity or obscenity. (I learned later that he saved those for very special occasions, which this wasn't.) But he described our shortcomings, physical, mental, moral, and genetic, in great and insulting detail. But somehow I was not insulted; I became greatly interested in studying his command of language. I wished that we had had him on our debate team.

At last he stopped and seemed about to cry. "I can't stand it," he said bitterly. "I've just got to work some of it off -- I had a better set of wooden soldiers when I was six ALL RIGHT! Is there any one of you jungle lice who thinks he can whip me? Is there a man in the crowd? Speak up !" There was a short silence to which I contributed. I didn't have any doubt at all that he could whip me; I was convinced. I heard a voice far down the line, the tall end. "Ah reckon ah can . . . suh." Zim looked happy. "Good! Step out here where I can see you." The recruit did so and he was impressive, at least three inches taller than Sergeant Zim and broader across the shoulders. "What's your name, soldier?" "Breckinridge, suh -- and ah weigh two hundred and ten pounds an' theah ain't any of it `slack-bellied.' " "Any particular way you'd like to fight?" "Suh, you jus' pick youah own method of dyin'. Ah'm not fussy." "Okay, no rules. Start whenever you like." Zim tossed his baton aside. It started -- and it was over. The big recruit was sitting on the ground, holding his left wrist in his right hand. He didn't say anything. Zim bent over him. "Broken?" "Reckon it might be . . . suh." "I'm sorry. You hurried me a little. Do you know where the dispensary is? Never mind -- Jones! Take Breckinridge over to the dispensary."

As they left Zim slapped him on the right shoulder and said quietly, "Let's try it again in a month or so. I'll show you what happened." I think it was meant to be a private remark but they were standing about six feet in front of where I was slowly freezing solid. Zim stepped back and called out, "Okay, we've got one man in this company, at least. I feel better. Do we have another one? Do we have two more? Any two of you scrofulous toads think you can stand up to me?" He looked back and forth along our ranks. "Chicken-livered, spineless -- oh, oh! Yes? Step out." Two men who had been side by side in ranks stepped out together; I suppose they had arranged it in whispers right there, but they also were far down the tall end, so I didn't hear. Zim smiled at them. "Names, for your next of kin, please." "Heinrich." "Heinrich what?" "Heinrich, sir. Bitte." He spoke rapidly to the other recruit and added politely, "He doesn't speak much Standard English yet, sir." "Meyer, mein Herr," the second man supplied. "That's okay, lots of `em don't speak much of it when they get here -I didn't myself. Tell Meyer not to worry, he'll pick it up. But he understands what we are going to do?" "Jawohl," agreed Meyer. "Certainly, sir. He understands Standard, he just can't speak it fluently." "All right. Where did you two pick up those face scars? Heidelberg?" "Nein -- no, sir. Ko:nigsberg." "Same thing." Zim had picked up his baton after fighting Breckinridge; he twirled it and asked, "Perhaps you would each like to borrow one of these?" "It would not be fair to you, sir," Heinrich answered carefully. "Bare hands, if you please." "Suit yourself. Though I might fool you. Ko:nigsberg, eh? Rules?" "How can there be rules, sir, with three?" "An interesting point. Well, let's agree that if eyes are gouged out they must be handed back when it's over. And tell your Korpsbruder that I'm ready now. Start when you like." Zim tossed his baton away; someone caught it. "You joke, sir. We will not gouge eyes." "No eye gouging, agreed. `Fire when ready, Gridley.' " "Please?" "Come on and fight! Or get back into ranks!" Now I am not sure that I saw it happen this way; I may have learned part of it later, in training. But here is what I think happened: The two moved out on each side of our company commander until they had him completely flanked but well out of contact. From this position there is a choice of four basic moves for the man working alone, moves that take advantage of his own mobility and of the superior co-ordination of one man as compared with two -- Sergeant Zim says (correctly) that any group is weaker than a man alone unless they are perfectly trained to work together. For example, Zim could have feinted at one of them, bounced fast to the other with a disabler, such as a broken kneecap then finished off the first at his leisure. Instead he let them attack. Meyer came at him fast, intending to body check and knock him to the ground, I think, while Heinrich would follow through from above, maybe with his boots. That's the way it appeared to start. And here's what I think I saw. Meyer never reached him with that body check. Sergeant Zim whirled to face him, while kicking out and getting Heinrich in the belly -- and then Meyer was sailing through the air, his lunge helped along with a hearty assist from Zim. But all I am sure of is that the fight started and then there were two German boys sleeping peacefully, almost end to end, one face down and one face up, and Zim was standing over them, not even breathing hard.

"Jones," he said. "No, Jones left, didn't he? Mahmud! Let's have the water bucket, then stick them back into their sockets. Who's got my toothpick?" A few moments later the two were conscious, wet, and back in ranks. Zim looked at us and inquired gently, "Anybody else? Or shall we get on with setting-up exercises?" I didn't expect anybody else and I doubt if he did. But from down on the left flank, where the shorties hung out, a boy stepped out of ranks, came front and center. Zim looked down at him. "Just you? Or do you want to pick a partner?" "Just myself, sir." "As you say. Name?" "Shujumi, sir." Zim's eyes widened. "Any relation to Colonel Shujumi?" "I have the honor to be his son, sir." "Ah so! Well! Black Belt?" "No, sir. Not yet." "I'm glad you qualified that. Well, Shujumi, are we going to use contest rules, or shall I send for the ambulance?" "As you wish, sir. But I think, if I may be permitted an opinion, that contest rules would be more prudent." "I don't know just how you mean that, but I agree." Zim tossed his badge of authority aside, then, so help me, they backed off, faced each other, and bowed. After that they circled around each other in a half crouch, making tentative passes with their hands, and looking like a couple of roosters. Suddenly they touched -- and the little chap was down on the ground and Sergeant Zim was flying through the air over his head. But he didn't land with the dull, breath-paralyzing thud that Meyer had; he lit rolling and was on his feet as fast as Shujumi was and facing him. "Banzai!" Zim yelled and grinned. "Arigato," Shujumi answered and grinned back. They touched again almost without a pause and I thought the Sergeant was going to fly again. He didn't; he slithered straight in, there was a confusion of arms and legs and when the motion slowed down you could see that Zim was tucking Shujumi's left foot in his right ear -- a poor fit. Shujumi slapped the ground with a free hand; Zim let him up at once. They again bowed to each other. "Another fall, sir?" "Sorry. We've got work to do. Some other time, eh? For fun . . . and honor. Perhaps I should have told you; your honorable father trained me." "So I had already surmised, sir. Another time it is." Zim slapped him hard on the shoulder. "Back in ranks, soldier. C'pnee!" Then, for twenty minutes, we went through calisthenics that left me as dripping hot as I had been shivering cold. Zim led it himself, doing it all with us and shouting the count.

Theories of games

While visiting Vanderbilt over spring break, I discovered that astrophysicist Bob Scherrer and I share a couple of boyhood interests: science fiction and strategic games. Bob actually writes science fiction, and still plays board games with his kids, whereas I switched long ago to "serious" literature and don't play or design games anymore.

But from age 11 to 14 or so (basically until I hit puberty and discovered girls), I spent every Saturday at the local university simulation gaming club, playing games like Panzergruppe Guderian, Starship Troopers or Dungeons and Dragons with college students and other adults. Bob tells me that I should have saved my collection of these games -- that they'd be very valuable today! Actually, the design of games and rule systems interested me even more than play.

Although I admire the elegance of classical games like Chess and Go, I prefer simulation games. More specifically, I enjoy thinking about the design and structure of these games. A good analogy is the distinction between natural science and mathematics. The former attempts to distill truths about the workings (dynamics) of the natural world, whereas the latter can be admired solely for its abstract beauty and elegance. To me Chess sometimes feels too finite and crystalline. The challenge of formulating a system of rules that captures the key strategic or tactical issues facing, e.g., Stalin, or an infantry platoon, or the ruler of a city state, or even a science postdoc, is just messy enough to be more interesting to me than the study of a finite abstract system. To some extent, every theoretical scientist, economist and financial modeler is participating in a kind of game design -- building a simplified model without throwing away the essential details.

I think role playing games are overly maligned, even among the community of gamers. Under ideal circumstances, role playing games are highly educational, and combine components like story telling, negotiation, diplomacy and team building. At the club I attended one of the "game masters" (I know, it sounds silly!) was a portly older man named Bill Dawkins, who preferred to be called Standing Bear (his Native American name). Standing Bear, though possessed of limited formal education, was widely read and had lived a vast life. He was the most creative story teller and world creator I have known. His ideas were easily as original and interesting as those I had encountered in science fiction and fantasy writing. Each of the role playing campaigns he created, taking place over years, was a masterpiece of imagination and myth building. He attracted scores of players from around the region. I often found myself playing alongside or against people I barely knew, although some came to be close friends.

Exodus to the virtual world

If you have any interest in virtual reality or computer games, I highly recommend this podcast of an interview with Ed Castronova, an economist who studies large-scale online games. He's the author of Exodus to the Virtual World: How Online Fun Is Changing Reality and Synthetic Worlds: The Business and Culture of Online Games. (There is a transcript of the interview at the first link above, as well as references and discussion; you can also listen to it as a stream if you don't have an ipod.) Castronova blogs here.

I suspect Castronova would agree with me that if AI is eventually successful, then many, perhaps almost all, sentient beings in our future lightcone will be game characters who are themselves unaware that they live in a simulation.

Games gone wild

A great article in the Times Magazine on the current state of online gaming. Millions of people now spend a significant fraction of their waking hours immersed in a virtual gameworld. The numbers will only continue to increase, along with the richness and complexity of the worlds. Just as gaming drives the development of graphics cards and even CPU chips (see Sony's Cell processor), it may eventually come to drive AI development, as designers seek more realistic and complex interactions between game characters and players. (Other drivers of AI development: search, human-machine collaboration, cat and mouse security competition between malware/spam automatons and security systems, ...)

Why anthropic "typicality" arguments (all the rage in the current nutty world of theoretical physics) are meaningless without an assumption about the possibility of virtual worlds.

A slideshow of game players and their avators, like the one below.



NAME Jean-François de la Fage BORN 1979 OCCUPATION Journalist LOCATION Paris AVATAR NAME Dark Freeman AVATAR CREATED 2005 GAME PLAYED City of Heroes HOURS PER WEEK IN-GAME 21 CHARACTER TYPE Natural hero SPECIAL ABILITIES Invincibility

NYTimes Magazine: ...More than eight million people around the world play World of Warcraft — approximately one in every thousand on the planet — and whenever Li is logged on, thousands of other players are, too. They share the game’s vast, virtual world with him, converging in its towns to trade their loot or turning up from time to time in Li’s own wooded corner of it, looking for enemies to kill and coins to gather. Every World of Warcraft player needs those coins, and mostly for one reason: to pay for the virtual gear to fight the monsters to earn the points to reach the next level. And there are only two ways players can get as much of this virtual money as the game requires: they can spend hours collecting it or they can pay someone real money to do it for them.

...In 2001, Edward Castronova, an economist at the University of Indiana and at the time an EverQuest player, published a paper in which he documented the rate at which his fellow players accumulated virtual goods, then used the current R.M.T. prices [R.M.T. = real money trading] of those goods to calculate the total annual wealth generated by all that in-game activity. The figure he arrived at, $135 million, was roughly 25 times the size of EverQuest’s R.M.T. market at the time. Updated and more broadly applied, Castronova’s results suggest an aggregate gross domestic product for today’s virtual economies of anywhere from $7 billion to $12 billion, a range that puts the economic output of the online gamer population in the company of Bolivia’s, Albania’s and Nepal’s.

Not quite the big time, no, but the implications are bigger, perhaps, than the numbers themselves. Castronova’s estimate of EverQuest’s G.D.P. showed that online games — even when there is no exchange of actual money — can produce actual wealth. And in doing so Castronova also showed that something curious has happened to the classic economic distinction between play and production: in certain corners of the world, it has melted away. Play has begun to do real work.