Recommend
114 
 Thumb up
 Hide
113 Posts
1 , 2 , 3 , 4 , 5  Next »   | 

Ra» Forums » General

Subject: Computer version of Ra w/ AI 3,4,5 players rss

Your Tags: Add tags
Popular Tags: computer-versions [+] computer [+] [View All]
Ylaine Gerardin
United States
Boston
Massachusetts
flag msg tools
Avatar
mb
New from the man (ok, the men - gotta include Knizia) who brought you Modern Art...

Computer Ra with AI!

I've been working on this for the past season though I actually started it in January. I decided I would include AIs for 3, 4, and 5 players. Let me know if you think the AI is challenging! I hope it isn't boring even for the best Ra players here.

New players should be challenged by the "Easy Mode" AI. If you think the Easy Mode is too difficult/too easy, let me know and I can fiddle with it. Please report any bugs that you find and I will do my best to fix them.

The response to Modern Art was so gratifying that I knew it would be rewarding to put the time in to make another game for BGG.

Enjoy! Tell me what you think!

download at:

http://www.dillgroup.ucsf.edu/~grocklin/ra.zip

The readme is below:

RA*README*10-27-2006************************************************************

A Challenging Game of Gods, Men, and their Monuments
Designed by Reiner Knizia
Programmed by Gabriel Rocklin

Version 1.0.0 - Release Revision 1

This file contains:

-System Requirements
-Required Files
-Revision History
-How to play Ra
-How to use Ra
-Ra Strategy
-Programming Notes
-Credits
-Registration
-Links
-Disclaimer

SYSTEM REQUIREMENTS:*******************

Ra should be able to run on any computer capable of running Windows XP at 1024x768. It also requires a mouse. It should be able to run on older computers with earlier operating systems, but it has not been tested.

REQUIRED FILES:************************

Ra.exe - Ra main executable.

msvbvm60.dll - Main Visual Basic 6.0 Runtime dll. If you lose
it, re-download the program zip file. If you can't do that, the file can be easily found by FTP search.

REVISION HISTORY:**********************

Version 1.0.0 - Release Version - 10/27/06

HOW TO PLAY RA:************************

Ra is a game of strategy that is based on purchasing sets of tiles in sporadic auctions in order to score points, as well as strategically timing those auctions to create the most value-over-cost for oneself and the least value-over-cost for the other players.

The game is played in three epochs, during which the players have the chance to purchase tiles. Each player begins the game with 10 points. A player's score can go to zero, but not below. Each player also begins the game with a set of "suns" or "sun tiles" - four suns in a three player game, or three suns in a four or five player game. Suns are the currency used to bid on sets of tiles, and they are numbered from 1-16. Higher suns are not used in games with fewer players. Each player begins the game with a different set of suns.

3 Player Game 4PG 5PG
Player 1 2/5/ 8/13 2/6/13 2/ 7/16
Player 2 3/6/ 9/12 3/7/12 3/ 8/15
Player 3 4/7/10/11 4/8/11 4/ 9/14
Player 4 5/9/10 5/10/13
Player 5 6/11/12

Which suns each player receives has no bearing on turn order - "Players 1-5" above represents the different players, not necessary the order in which they are seated around the table, which is the order of play. In the computer version, the order of play is from top to bottom of the player areas. Which player receives which suns is assigned randomly, and the player who receives the highest sun (13 or 16 depending on the number of players) begins the game.

A player typically has two options each turn: he or she may draw a tile at random out of the tile bag and add it to the group of tiles already on the auction block, or he or she may call "RA!" and initiate an auction (or use a God tile, which is discussed later). If there are already eight tiles on the auction block, a player can only initiate an auction. If a player chooses to draw a tile, two things can happen: if a red "Ra" tile is drawn, an auction begins as if the player had called "RA!". If any other tile is drawn, that tile is added to the auction block, and the player's turn is over and the next player takes a turn.

When an auction begins, either because a player called "Ra!" or because a player drew a "Ra" tile, the bidding begins to that player's left (the next player in turn order). In the computer version, the bidding begins with the player below the player who began the auction. A player may bid the value of any of their sun tiles, but ONLY the precise value of the sun. A player who bids a 10 sun tile to outbid another player's 4 because 10 is the first player's lowest sun above a 4 does NOT receive "change" in the form of an additional sun. In succession, each player around the table (or down the screen in the computer version) bids once until the bid reaches the player who began the auction, who has the final bid.

The winner of the auction is the player who offered the highest sun. This player takes all the tiles on the auction block and adds them to his or her play area. If the player needs to discard any tiles because of catastrophe tiles on the auction block (discussed later), this is done immediately. This player ALSO takes a sun tile from the auction block, which was the winning sun tile from the previous auction. In the case of the first auction of the game, this tile is a 1 sun tile. This sun is added to the player's play area FACE DOWN, and CANNOT BE BID until the next epoch begins. The player then takes the sun tile representing their winning bid and places it on the auction block, to be auctioned off along with the next set of tiles. This is how sun tiles are replenished. The other players keep the sun tiles that they had bid and can use these to bid on future auctions in the epoch - nothing is lost if a player bids but does not win the auction. Because each player begins each epoch with only three or four suns (depending on the number of players), each player can only win three or four auctions per epoch.

An epoch ends as soon as EITHER of two conditions is reached: either all players have used all of their suns (all suns are face-down), OR the maximum number of Ra tiles (auction-initiating tiles) has been drawn in that epoch - 8 in the three-player game, 9 with four, and 10 Ra tiles in the five-player game. As soon as the final Ra tile is drawn, the epoch ends immediately and NO AUCTION TAKES PLACE. Any tiles remaining on the auction block are discarded. At the end of each epoch, points are awarded based on the tiles that each player possesses - Pharaohs, Niles, Floods, Civilizations, Gods, and Gold. Points are only awarded for monument tiles at the end of epoch three, which is the end of the game. The point system is described later.

After points are awarded, any tiles remaining on the auction block are discarded, the "Ra Track" which keeps track of the number of Ra tiles drawn in that epoch is reset to 0, all players discard their Flood, Civilization, God, and Gold tiles, and all players flip any face-down suns face-up so they can be used. Any sun tiles which were not used in the previous epoch remain face up and can be used to bid in the new epoch. The play begins with the player possessing the highest numbered sun.

At the end of epoch three, points are awarded as in each of the preceding epochs, and points are also awarded for Monument tiles. Finally, points are awarded based on the total suns each player finishes with: the player with the highest sum of the sun values (including both face-up and face-down suns) earns 5 points, and the player with the lowest sum of the sun values loses 5 points. In the event of any ties, all tied players gain or lose 5 points as appropriate. The player with the highest number of points at the end of the game is the winner. In the event of a tie, the player tied for the most points who also possesses the single highest-numbered sun of all the players tied for the most points is the winner.

At the end of each epoch, points are awarded based on the tiles each player possesses, according to the following rules:

+5 points to the player who possesses the greatest number of Pharaoh tiles
-2 points to the player who possesses the fewest Pharaoh tiles

In the event of a tie, all tied players gain or lose 5 points as appropriate.

+1 point for each Nile tile, assuming the player has AT LEAST ONE FLOOD TILE
+1 point for each Flood tile

+3 points for each Gold tile

+2 points for each God tile

-5 points if the player has NO Civilization tiles
+5 points for each DIFFERENT Civilization tile after the first two (5 points for 3 different ones, 10 points for 4, 15 points for 5)

+1 point for each DIFFERENT kind of Monument tile for up to 6 different kinds. The 7th DIFFERENT Monument tile is worth 4 points (pushing the total for all 7 tiles up to 10 points), and the 8th different Monument tile is worth 5 points, pushing the total for all 8 Monument tiles up to 15 points.

+5 points for each Monument tile that is the third, fourth, or fifth of its kind - i.e. a set of three Pyramid Monuments is worth 5 points, a set of 4 is worth 10, and a set of 5 is worth 15 points.

Four tiles, known as "Catastrophe tiles", do not add to a player's score but require that player to take immediate action upon winning an auction containing one of those tiles.

Funeral - the player must discard 2 Pharaoh tiles (or as many as they can, up to 2).

Drought - the player must discard 2 Flood tiles. If a player has fewer than 2 Flood tiles, Nile tiles must be discarded until either the total number of tiles discarded is 2 or the player possesses no Flood or Nile tiles.

Unrest - the player must discard 2 Civilization tiles (or as many as they can, up to 2).

Earthquake - the player must discard 2 Monument tiles (or as many as they can, up to 2).

All Catastrophe tiles are assumed to occur AFTER all the other tiles from the auction block have been added to the player's inventory, i.e. tiles that are won in the same auction as the Catastrophe are NOT protected from the effects of that Catastrophe.

The possession of God tiles, won in an auction, gives a player a third option in addition to drawing a tile or calling "Ra!". A player possessing a God tile may use his or her turn to discard the God tile and take a single tile from the auction block to add to his or her inventory. If a player possesses and discards multiple God tiles, multiple tiles may be removed from the auction block and added to that player's inventory - one tile from the auction block for one God tile discarded. God tiles may not be used to take other God tiles from the auction block. If a God tile is used to take a catastrophe tile from the auction block, the player discards any tiles required by the catastrophe as soon as the catastrophe tile is acquired.

The distribution of tiles in the bag is as follows:

30 Ra Tiles
25 Pharaoh Tiles
25 Nile Tiles
12 Flood Tiles
8 God Tiles
5 Gold Tiles
5 Civilization Tiles - Astronomy
5 Civilization Tiles - Writing
5 Civilization Tiles - Religion
5 Civilization Tiles - Agriculture
5 Civilization Tiles - Art
5 Monument Tiles - Obelisks
5 Monument Tiles - Palaces
5 Monument Tiles - Sphinxes
5 Monument Tiles - Pyramids
5 Monument Tiles - Fortresses
5 Monument Tiles - Step Pyramids
5 Monument Tiles - Temples
5 Monument Tiles - Statues
4 Unrest Catastrophe Tiles
2 Funeral Catastrophe Tiles
2 Drought Catastrophe Tiles
2 Earthquake Catastrophe Tiles
----------------
180 Tiles

No discarded tiles (Ra tiles, Floods, Civilization tiles, Catastrophe tiles, God tiles, Gold tiles) are returned to the bag during the game.

HOW TO USE RA:*************************

To begin a new game, select the appropriate number of players from the "Game" menu. Your suns tiles appear in the top box on the right part of the screen. Each of the other players has their own box below yours. Your score is displayed as a number just below and to the right of your name. If the option is selected, the scores of the other players will be displayed as well. The turn is indicated by the blue box just below and to the right of the players' names.

At the upper left of the screen is the control box. Below that, the epoch and number of Ra tiles drawn is displayed. Below that is the auction block, with the sun tile up for auction at the top. The 8 tiles on the auction block will be shown below as they are drawn. To be reminded of the scoring value of any tile, hover the mouse over it.

The control box gives you the appropriate options on your turn and is where you control the game. When it is your turn, you may choose here to draw a tile or call Ra, and then confirm your choice. If you wish to use a God tile, double-click the God tile in your playing box, double-click which tile you wish to take from the auction block, and then confirm your selection in the control box. On another player's turn, you may continue the game from the control box once you have noted the action that the other player just took. When an auction occurs, the suns that each player chooses to bid are lit up in red, and the highest bid is shown in the control box. When it is your turn to bid, you may double-click one of your sun tiles to bid it, or click "Pass" in the control box. After confirming your choice, the winner of the auction will be shown. If you need to discard Monument tiles or Civilization tiles following winning an Earthquake catastrophe or an Unrest catastrophe, double-click the tiles in your playing box and then confirm your wish to discard it from the command box. To return a tile to your play box should you choose not to discard it, double-click the tile from the auction block. The status bar at the bottom of the screen updates you on what is happening in the game. To review status bar messages that have passed, double-click the status bar to see an expanded status history.

At the end of an epoch, the scores of all the players are updated, and you have the chance to review the tiles each player won in the epoch before Floods, Gold, Gods, and Civilization tiles are discarded. If scores are hidden, then the computer players' points earned in that epoch will appear in [brackets]. This is NOT a cumulative score and is only an aid to see how many points each player earned in that epoch. This indicator will disappear when the next epoch begins. Select the "Epoch x" button from the control box to continue to the next round.

At the end of the game, all of the players final scores are displayed, and the winner of the game is announced in the status bar. To view a complete game history showing where each player earned all of their points, select "Show History" from the control box. The history box can be closed or copied to the clipboard should you wish to save it.

The "Options" menu allows you to change the name of your player. Overly long names will not display properly in the game. The "Options" menu also allows you to select whether you wish to hide the opponent scores and the values of the face-down Sun tiles they have won in auctions. You can also reduce the difficulty of the game by selecting "Easy Mode".

The "Easy Mode" may itself be a challenge. Ra takes many games of practice to play well and the AI was designed to be a long-term challenge.

Selecting "How to Play Ra" from the "Help" menu displays this document so that the player can review the rules and strategy of the game. Selecting "About Ra" shows the player basic information about the program as well as the version number. The player can also choose to receive a hint in the help menu. This hint represents what the AI would do if it were in the same situation as the player. This hint is NOT A DEFINITIVE STATEMENT ON THE IDEAL RA DECISION. The AI plays the game a certain way, and other experienced Ra players might act entirely differently from the advice given by the AI. The hint may appear to be unambiguously wrong to the player - and this may indeed be the case! - but the situation may also be more ambiguous than it appears. In the case of an auction, the hint will give an approximate value for the set of tiles. This value does NOT take into consideration the value of the sun tile on the auction block, although the value of the sun IS taken into consideration in the recommended bid. This value represents the approximate loss in points if those tiles were taken away from the player after they had been purchased. This means that a Civilization tile may be valued at fewer than five points even if the player has no Civilization tiles at that time, because if the player acquires a second Civilization tile that round, removing the first one would not alter their score for the round (unless the first one was a member of a set of at least three). Pharaoh tiles may be valued at more than 5 (or more than 7) points because they contribute to the player's score across multiple rounds. Most importantly, this value is APPROXIMATE.

RA STRATEGY:***************************

The following is my take on the game, which is likely not universal.

Ra is a game of tactics. There are three major decisions a player has to make: how much a set of tiles is likely to be worth, how many points to be willing to trade a sun for at each stage in the epoch, and whether or not to call Ra at any given time. None of these decisions is particularly dependent on a long-term strategy. The key to effective Ra playing is an accurate assessment of the worth of tiles and suns, which is only attainable through practice, and a clever use of the tactics of calling Ra.

Virtually all the points a player earns will be the result of tiles randomly drawn from the bag, and nothing a player can do can influence what will be offered in a given game. A player must be willing to take any tiles that are offered, be they Pharaohs, Niles and Floods, Monuments, or Civilizations. A strategy heavily weighted toward collecting a particular type of tile is unlikely to be successful. That said, all tiles are not equally valuable. Appropriate questions to ask when assessing the value of a tile include: how much is it worth now? How much could it potentially be worth later? What other tiles do I need to make this tile worth its full potential? Are those other tiles ones that I am likely to come into possession of whether I take this tile or not, or would I need to go out of my way to obtain them? Is there enough time left in the game or the epoch for me to get the tiles I need to make this tile reach its full potential? If I don't take a particular tile, does it hurt me to allow it to fall into the hands of my opponents? By taking a particular tile, do I remove a large number of points from a competitive opponent?

Ultimately, assessing the value of tiles is both very easy and impossible. It is impossible to know in Epoch 1 whether your first Monument will be worth a single point or the 5 points earned if it becomes a member of a set of three. It is impossible to know whether a single Nile tile in Epoch 1 will end up providing zero points or three points. However, each player will have their own estimates of how much tiles are worth, and ultimately these estimates do not have be extremely accurate to be successful playing Ra. A general grasp of the approximate worth of each tile is sufficient, which makes this one of the easier aspects of the game.

Determining the value of Sun tiles is much more difficult. A player who underestimates the value of their Suns and bids very highly on the first few auctions of the epoch will find themselves out of the epoch quickly and with few points. A player who overestimates the value of their suns will not bid at all and find themselves with no tiles and no points at the end of the epoch. The absolute limit on the value of suns is the Ra track - there will be only so many auctions in an epoch, and to oversimplify, the value of the highest sun is equal to the value of the best set of tiles auctioned off in a given epoch. Of course, a player does not know whether Ra tiles will come quickly and sharply decrease the points available in an epoch, or whether Ra tiles will be unusually absent and many high-scoring sets will be available.

Even if determining the value of sun tiles is difficult and contains the heart of the game, several general observations can be made which should assist new players. First, the values of the suns do not directly correlate to their number. A 10 sun is not worth twice as much as a 5 sun, and a 13 sun is not worth 13 times as much as a 1 sun. This is for two reasons: first, each sun is capable of winning only one auction, and second, there is little a player can do to limit the minimum value of an auction block in an epoch. A player can reduce the value of auction sets by calling Ra when few tiles are available, but because a player who calls Ra is forced to bid if the other players choose not to, a player can only do this a maximum of three or four times depending on the number of players. This means that even the lowest sun, if held properly, has a certain minimum value limited only by the frequency that Ra tiles are drawn from the bag.

Obviously, the major importance of a sun's number is that the number determines which suns that sun outranks. In most cases, a 12 is no different from a 7 when the suns 8-11 have all been used. Furthermore, if a player possessing the 12 and the 7 is competing for a Flood tile with another player whose highest sun is a 6, the 12 and the 7 may still be essentially equivalent if the suns in between are held by the player who is not interested in the auction. The consequence of this is that the player can be willing to spend their 12 on a set of tiles which they might otherwise think was only worth spending a 7 on, because he or she can be confident that the lucrative set containing the flood can be won over the competition with only the 7. Which suns outrank which also has a large role to play in the strategy of forcing other players to bid higher than they wish. If one player appears ready to take a set with a 4, bidding a 5 in order to force them to bid a 6 instead is only mildly useful. The 6 only outranks a single sun (the 5) that the 4 does not, and the bid runs the risk of forcing the player who offered the 5 to actually take a set they may not be interested in and waste a sun. However, if the next highest sun above the 4 is a 10 or 11, then bidding the 5 forces the first player to use a sun which outranks many more suns than the one they were intending to bid, which can put that player at a strategic disadvantage. Put another way, even if the set of tiles is extremely valuable and worth spending an 11 on, the "value over cost" is much higher if the cost is a sun that outranks very few other suns than if the cost is a sun which outranks many other suns. The cost of purchasing a set of tiles is essentially the opportunity cost of anything else that might have been purchased with that sun.

The opportunity cost of spending a sun decreases throughout the epoch because there are fewer opportunities left to spend the sun before the epoch ends. This means the amount of points a player should be willing to spend a sun on decreases throughout the round. If you anticipate a certain sun to be worth five points, there is no disadvantage in waiting for a significant portion of the epoch to expire before trading the sun for the points. Spend the early part of the epoch looking for opportunities to trade the sun for more than five points and take one of those opportunities if they appear. If they do not, then lower the expectation back to five and take one of those sets when possible.

There are several peripheral benefits to possessing any suns at all. Possessing at least one sun allows a player to use God tiles, and this may be an important consideration if there are several specific tiles a player desires (a flood, the 3rd monument of a kind, the 1st or 3rd civilization, etc). Possessing at least one sun also gives a player control over the timing of auctions, which can be important for preventing other players from allowing the auction block from growing extravagantly long. These considerations must be weighed against the points earned from spending the sun.

All the analysis about the value of sun tiles in your hand holds true for the value of sun tiles on the auction block. The number of the sun on the auction block represents how many other sun tiles that sun can outrank. A 3 sun can get a player just as many points as a 13 sun, but the player must find an opportunity for taking those points when no other player is going to spend a higher sun. Obviously these opportunities are rarer than opportunities for lots of points that are common to all of the players, but a player is not always in a better position for possessing all high suns instead of a mix of high and low suns. Low suns are never superior to high suns (a misconception), but not every auction a player bids in is going to be particularly competitive with the other players. The 3 sun is just as good as the 8 sun if no player is willing to bid (or possesses) any sun in-between. Thus, when deciding on the value of a sun on the auction block, consider how many competitive auctions you are likely to be in in the following round, consider how many suns you have which can win those auctions, and consider how important it is to add a high sun to your repertoire or how damaging it will be to accept a low sun. If you already have the suns necessary to win your competitive auctions (or intend to acquire those suns later), then it may be strategic to accept a lower numbered sun for the benefit of taking a set of tiles worth a larger number of points, confident that that the actual numerical value of your lower suns is unlikely to matter.

The last major decision a player needs to make is whether or not to call Ra. The most important realization of a new player is that calling Ra is not something you only do when you intend to win the auction yourself. A player can call Ra either because they intend to win the set or because they can put an opponent in a situation where taking the set of tiles is not especially valuable compared to what they expect for a given sun, but the set of tiles is just valuable enough that passing on it in the hopes of getting something better is overly risky. Imagine only two players remain with one sun each, a 7 and a 5. The player with the 7 is willing to spend his sun for 8 points. There is no point for the player with the 5 to call Ra when the auction block is worth 4 points, or 5 points, or 6 points. The player with the 5 should wait until the auction block is worth 7 or 8 points and then call Ra. The player with the 7 is then forced to either take the bare minimum they were willing to accept or allow a valuable block go to the other player. If the player with the 5 waits to call Ra until the block is worth 10 points or 11 points, each additional tile past 8 points is simply more points for the player with the higher sun because that player was going to take the auction block anyway. Obviously, the player with the higher sun should never call Ra, unless they are concerned about the epoch ending or a disaster tile ruining a good set. By calling Ra when the set is worth 7 or 8 points, the player with the lower sun maximizes the value to themselves and minimizes the value to the other player.

Of course, the actual game is more complicated. An auction block is not typically worth the same number of points to all players. If the player with the 5 sun also has a 10 sun, that player does not need to call Ra when the tiles on the auction block are worth 7 or 8 points because that player can wait until the block is even longer and overbid the player with the 7 sun. Deciding whether to call Ra early with the option of underbidding or to allow the set to grow is one of the challenging decisions in Ra.

Knowing when to call Ra is heavily based on having good estimates for how many points to expect from your own suns and on how many points your opponents expect from their suns - not what they should expect, but what they do expect based on observations of how they play. Misreading one's opponents can lead to either being forced to take a set one was not planning to take because an opponent chose not to bid when they were expected to, or allowing an opponent to receive more points than they should by allowing the auction block to grow long when the opponent would have gladly taken it with the same winning sun when it was shorter. The more one plays Ra, the more one can develop good expectations for how much tiles are worth, how much suns are worth, and how one's opponents play the game. Though it may be difficult to surpass the 50,000 or so games of experience that each AI player has, presumably a human extracts much more nuanced information from each game than a multi-layer perceptron.

PROGRAMMING NOTES:*********************

Ra was programmed in Microsoft Visual Basic 6.0. There are three components to the AI - one system that assesses the value of a group of tiles, one system that assesses the minimum value of suns (if the value of the group of tiles exceeds the value of the sun, the computer bids), and one system that determines whether or not to call Ra. The first two systems are artificial neural networks and their behavior evolved over tens of thousands of games. The system that determines whether to call Ra, draw a tile, or use a God was hard-coded into the game based on the experience and strategy of the designer.

An artificial neural network is a function approximator. In the case of the first system that determines the value of a group of tiles, there is some function V(I1, I2, I3, ..., In) that represents the (V)alue of a group of tiles based on certain properties or (I)nputs about those tiles and the state of the game. For the purposes of evolving the AI, the value of a group of tiles was defined this way: if the player had never bought that set of tiles, how many points would they have lost? Under this definition, if a player picks up two cultures in different auctions in the same epoch, each culture added 0 value to the set of tiles that it was won with because if that culture had been lost, the player would not have lost any points. The inputs represent all the properties of the tiles on the auction block and the state of the game that may influence the value of a set of tiles. For example, inputs used in Ra include: how many Pharaohs are on the auction block? How many Pharaohs behind the leader am I? How many nile tiles are on the auction block? How many floods are on the auction block? How many suns do I have left? How many other players still have suns left in this Epoch? How many cultures do I have? How many cultures are there on the auction block? How far along is the Ra track? Inputs which are not directly related to the auction block are important because if a player takes a culture tile when the Ra track is empty, they are much more likely to take a second purely by accident which means that it wasn't worth spending a sun to get the first one; whereas if a player is still missing a culture and the Ra track is almost full, taking a culture now will very likely directly affect that player's score. In this version of Ra, there are 46 inputs.

Of course, in two games where every detail is the same at the time when a set of tiles is purchased, the tiles may not end up providing the same number of points in each game. Nearly every tile except Gold can change in value based on random events which happen in the future, thus the value function V is really an average value over many games of how much those tiles are expected to be worth based on the inputs given.

How do we determine what V(I1, I2, ... I46) is? To begin with, we know it is not linear. A set of tiles which contains two Pharaohs is not worth twice as many points as a set of tiles which contains one Pharaoh. A set of tiles containing two Floods is not worth twice as much as a set of tiles containing one Flood. Fortunately, artificial neural networks are very good at learning to approximate nonlinear functions. To understand why, we first have to describe in more detail what an artificial neural network is.

To approximate the value of a group of tiles using an artificial neural network, we start with our inputs I1-I46. We call these "input neurons", and each one is a number. We could say that I1 is the number of Floods on the auction block, or I2 is the number of Pharaohs on the auction block, or I3 is whether the round is half over (-1 if the Ra track is less than half full, +1 if the Ra track is greater than half full). Now we have assigned numbers for input neurons 1-46. After this, we define a set of "hidden neurons". Each hidden neuron is a function of all of the input neurons. Ra uses 21 hidden neurons, because fewer would mean the AI was less intelligent, and more would not improve the AI enough to be worth the additional complexity (which makes the AI slower to train). To find the value of hidden neuron 1, we multiply the value of each input neuron by a value we call the "weight" connecting that input neuron to that hidden neuron. For example, imagine the following values:

Input Neuron 1: 3 (3 Floods on the auction block)
Input Neuron 2: 0 (No Pharaohs on the auction block)
Input Neuron 3: 1 (The round is more than half over)

Weight from Input Neuron 1 to Hidden Neuron 1: 2
Weight from Input Neuron 2 to Hidden Neuron 1: -1
Weight from Input Neuron 3 to Hidden Neuron 1: 0.5

If there were only 3 input neurons, then the value of hidden neuron 1 would be (3)(2) + (0)(-1) + (1)(0.5) or 6.5. This value is then "scaled" by using it as the parameter in the sigmoid function f(x)=1/(1+(e^-x)), where x is the value we just calculated, 6.5. By scaling in this manner, each hidden neuron is no longer a linear function of the input neurons. Each hidden neuron has a different set of weights connecting it to each of the input neurons, so all the hidden neurons typically have different values themselves. Finally, there is a single "output" neuron which is the output of the function. The "output" neuron's value is determined by multiplying each of the hidden neuron values by a new weight, and then scaling again. The output neuron is scaled to the range that sets of tiles are likely to be worth. In short, the values of the input neurons are used to create a set of hidden neuron values, which are used to determine the output value.

How does the neural network learn to approximate the value of the function we wish to approximate? The key lies in changing the values of the weights connecting all the neurons together. These are all assigned randomly at first, and then the computer plays games against itself. In each case where it bought a group of tiles, it knows its estimate for how much they would be worth and it knows the value they actually turned out to be worth. From here we can calculate an error, where the error is the actual value minus the estimate. If the error is positive, the estimate was too low. This means we can look for hidden neurons which gave large positive values and increase the value of the weight between them and the output neuron in order to increase the value of the output neuron and reduce the error. We can also adjust the value of the weights connecting the input neurons to the hidden neurons in a similar manner. By adjusting all the weights, the next time the neural network is used with similar inputs, the error should be lower. This training process is called backpropagation. In Ra, training the neural networks took between 20,000 and 60,000 games depending on the AI.

The second AI system involves estimating how much a sun tile is worth. This is a completely separate neural net that contains 33 input neurons and 17 hidden neurons. In fact, a separate neural net was used for each epoch. To evolve these neural networks, the computer played a modified version of Ra against itself where there is no calling Ra and an auction is held after every tile draw (no player is ever forced to bid). This gave the computer information at each point in the round about how much it estimated a group of tiles to be worth and what the other players were willing to bid or not bid for those tiles. Say the computer bought a set of tiles at some point with a 7 thinking those tiles were worth 5 points. If there comes a time later in the epoch where a 7 sun could have won an auction which that same computer player estimated would be worth 8 points, then that computer player knows it spent its sun too early. It adjusts its "target" value at the time the sun was spent from 5 to 8 so that the sun will never be spent under those circumstances unless the tiles are estimated to be worth at least 8 points.

In another situation, say a computer player ends an epoch without spending one of its suns. This means that at every point in the game where an auction was held, either the computer expected more points from the sun than the tiles were worth, or it was outbid by another player. In this case, the expectation at all of those times needs to be reduced to the maximum number of points it was feasible for that computer player to get with that sun. In other words, out of all the auctions which could have been won with sun X, which one was worth the most points? Because the sun was not actually bid at that time (because it was never used), the expectation must have been too high, so the weights are adjusted in order to lower the expectation so that the sun is used at the optimal time. If the neural net is trained perfectly, the computer player should know at every auction in the round whether that auction is the best chance it will get to spend a certain sun or whether it should hold on to the sun in anticipation of a better opportunity later. Even if the computer could determine this perfectly (which it cannot because neural networks tend to find local minima and not global minima), the computer would still not win every round because unlikely probabilities are not zero probabilities.

Small modifications to these systems were made to take into account the value of the sun on the auction block and the need to play defensively and prevent opponents from picking up slates of tiles of high value to them.

The final system involves determining whether or not to draw a tile, call Ra, or use a God. The computer decides things in this order: determine if I want to call Ra with the intention of winning the set, determine if I want to use a God tile, determine if I want to call Ra defensively, draw a tile. The determination on whether or not to call Ra is based on the estimate value of the tiles and the estimated value of that player's suns and the opponents' suns. If certain hard-coded criteria are met, the computer player calls Ra. I'm not certain these criteria are optimal but they do seem to allow for a competitive AI opponent.

For more information about the specific details and challenges of programming Ra, including:

-What the input neurons are in either neural net and how they were chosen
-Details of the neural net training including formulas and the use of momentum
-Details of the systems for calling Ra
-Advice on programming similar games

feel free to e-mail me at GRocklin07@cmc.edu

This e-mail address will be functional through May 2007. After that, you can always find me at:

http://www.cwal.net/forums/rr

Just post something asking for "Snapper".

CREDITS:*******************************

Gabriel Rocklin (Snapper):
Design, Programming

Beta Testers:

Grace Peng
Eddie Herman
Chip Dickerson
Gilad Landan
Sean McCarthy
Burke Glover
Adrian Weisberg

Thanks to:

Reiner Knizia, Ra Designer
Mat Buckland, "AI Techniques for Game Programmers"

REGISTRATION:**************************

Ra is provided free of charge. However, an incredible amount of effort went in to programming it that almost certainly should have been spent doing other things. In addition, even though the final AIs have only around 10,000 games of training, the AI went through a huge number of revisions before I was satisfied, and each of those needed to be trained to see how competitive they were. At a rate of around 1.3 seconds per game, my personal computer had the training process running for a minimum of 600 hours in the course of designing and refining the game. This kept the CPU and fan running at full blast the whole time and the CPU in the 70-80 degrees C range (at 80 degrees the CPU clocks down). Fortunately, hyperthreading allowed me to do other things while the AI was training, as long as I wasn't playing an intense game. So far, my computer seems to have survived this onslaught.

If you have enjoyed Ra or its free counterpart Modern Art, please consider rewarding the work that went in to producing the programs. A donation of 5$ is minimal compared to the cost of most board games or commercial computer games based on board games. To donate, go to:

http://snapper.rooms.cwal.net/games.html

and click "Make a donation".

A mailing address can be provided if requested by e-mail.

LINKS:*********************************

http://www.boardgamegeek.com
-The best site for boardgaming on the web, including a link to this
program.

http://snapper.rooms.cwal.net/ra.zip
http://snapper.rooms.cwal.net/ra.exe
-Links to the latest versions of this program. Whatever download site
you used might not have the most recent version, but these links
should always be current.

http://snapper.rooms.cwal.net/games.html
-Links to the latest versions of Ra and Modern art, as well as a link to donate

http://www.fairplaygames.com
-The cheapest place I've found to buy board games on the web

http://en.wikipedia.org/wiki/Reiner_Knizia
-Wikipedia's page on Reiner Knizia, the designer of Modern Art

http://en.wikipedia.org/wiki/Neural_networks
-Wikipedia's page on Artificial Neural Networks

http://www.cwal.net/forums/rr
-The CWAL Rubber Room, where you can usually find me hanging out
as "Snapper". A community of StarCraft players who now mostly
discuss other things.

GRocklin07@mckenna.edu

DISCLAIMER:****************************

Do these actually mean anything or are they just for fun? Anything that happens to you or anyone else as a result of anything involving this program is your fault and not my fault.

Any similarities to real-life entities in this document or this program are purely coincidental.

1,159,168 bytes were harmed in the creation of this program.

Fortunately, none of them were animals.

Except Jerry; Barry, Harry and Larry's Brother.

*sniff*

But don't tell anyone.

"Evolution is cleverer than you are." - Les Orgel

"No computer has ever been designed that is ever aware of what it's doing; but most of the time, we aren't either." - Marvin Minsky

"RA! RA! RA!"

Recursion - n. See Recursion.

And in the immortal words of the great Snapper,
25 
 Thumb up
6.00
 tip
 Hide
  • [+] Dice rolls
Joel Glidden
United States
Highlands Ranch
Colorado
flag msg tools
badge
Avatar
mbmbmbmbmb
Thank you! This is great!
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
mrbass
United States
Las Vegas
Nevada
flag msg tools
Avatar
mbmbmbmbmb
thanks works great under Parallels.....it's winXP even though it says win2000....trust me ..long story.
http://membo.org/i/bgg/ra.jpg

I never was able to beat the Modern Art AI...man I felt stupid. I sure hope I can beat the RA AI every now and then though.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ylaine Gerardin
United States
Boston
Massachusetts
flag msg tools
Avatar
mb
Let me know how your games go...
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Seth Ben-Ezra
United States
Peoria
Illinois
flag msg tools
designer
badge
Avatar
mbmbmbmbmb
Downloading as we speak.

If this is anywhere near as good as the Modern Art game, I will be very, very happy.

Thank you VERY much.

Seth Ben-Ezra
Great Wolf
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tejas Mistry
United States
Seattle
WA
flag msg tools
Thanks very much for your effort with this! I enjoy RA very much and its great to have an AI to play against -- and a pretty good AI at that. I played two 3-player games and lost both times, and so I'm hurting, but impressed!


I want to take the liberty of to make a few small suggestions that I hope help:
(I work as a software tester in real life, so can't refrain from making suggestions for what *I* feel might be improvements )
i) perhaps remove the need to press the 'Continue' button after each person's move. If you feel its really necessary to indicate what each person did, perhaps the text can be printed below each players individual play area.
ii) also the confirmation dialogs for "Draw a tile", "Call RA", "amount bid" don't seem necessary.
iii) The partial drawing of RA to indicate number of RA tiles drawn is a difficult to visually read or count (I know you have the actual numbers written right next to it which is great). I wanted to suggest showing individual small-sized RA tiles instead, for each RA that was drawn, so we can get a quick visual count.

Many thanks again!
Regards,
Tejas
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Kane Klenko
United States
Ridgeway
Wisconsin
flag msg tools
designer
badge
Avatar
mbmbmbmbmb
The best I ever did against the Modern Art AI was 2nd place, and that was one game out of many. Usually I'm in 4th by a long shot. However, I stink at Modern Art.

Ra I'm usually pretty good at and I'd say I win the majority of my games. I've played 1 game against the AI and I came in last place (3 player game). Maybe I just had an off game...I'll try it again. Great work though.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Seth Ben-Ezra
United States
Peoria
Illinois
flag msg tools
designer
badge
Avatar
mbmbmbmbmb
tejasm wrote:

I want to take the liberty of to make a few small suggestions that I hope help:
(I work as a software tester in real life, so can't refrain from making suggestions for what *I* feel might be improvements )
i) perhaps remove the need to press the 'Continue' button after each person's move. If you feel its really necessary to indicate what each person did, perhaps the text can be printed below each players individual play area.
ii) also the confirmation dialogs for "Draw a tile", "Call RA", "amount bid" don't seem necessary.
iii) The partial drawing of RA to indicate number of RA tiles drawn is a difficult to visually read or count (I know you have the actual numbers written right next to it which is great). I wanted to suggest showing individual small-sized RA tiles instead, for each RA that was drawn, so we can get a quick visual count.


I concur with these. Actually, what I'd like is the ability to turn the "Confirm" on and off.

Now, I work in IT, so I know that simple feature requests aren't always as simple as they appear. And, I don't want this to seem like ingratitude. I'm thrilled to have Ra AI sparring partners. (I started playing as soon as I downloaded it, but I had to get back to work.)

Seth Ben-Ezra
Great Wolf
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
David Phipps
United States
Winston-Salem
North Carolina
flag msg tools
badge
Avatar
mbmbmbmbmb
You did a great job and I really like it so far. I've played three 5-player games and won two of them on regular mode so far. It is definitely challenging and a good way for people to get their RA-fix.

One thing I'm finding is that I'm having a harder time convincing myself to push my luck when I'm on the verge of the last RA tile coming out. I guess I'm used to hefting the bag in my hand and guestimating how many tiles are left. (hmm, what are my chances of getting that flood or civ I need?)
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Tim K.
United States
Seattle
Washington
flag msg tools
badge
Avatar
mbmbmbmb
Yes, very cool thumbsup
Now can you make a version for the Palm OS? shake
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ylaine Gerardin
United States
Boston
Massachusetts
flag msg tools
Avatar
mb
Keep the suggestions coming, guys. I may implement some of them, but it will probably be about a month until school lets up and gives me the time.

On the specific subject of the Palm OS, I don't know enough to do that, but if anyone who does wants to talk to me, send me a note.
-
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
J
Canada
Slave Lake
AB
flag msg tools
badge
Avatar
mbmbmbmbmb
EvilTimmy wrote:
Yes, very cool thumbsup
Now can you make a version for the Palm OS? shake


Oh, yes, please do!

We also need Palm:

StreetSoccer
Sports Illustrated Baseball
Frank's Zoo
Alhambra
Igel Ärgern
RoboRally

etc.

(OK, so I got a little greedy. But anything good for Palm would be terrific.)
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Andy K.
United States
Sacramento
California
flag msg tools
badge
Way-whu-nuh-now?!
Avatar
mbmbmbmbmb
Two 3-player games down, and I've been spanked hard both times. Ouch!! Must resist...urge...to call RA.

Beautifully realized. Thank you soooooo much.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Nick Fisk
United Kingdom
Stoke on Trent
Staffordshire
flag msg tools
Come on you Seagulls! Sami Hyppia's Blue & White army!
badge
That's weird. This bit used to mention Shire Games, and tell you all how wonderful we are. But it seems to have got deleted. Let's see what happens this time ....
Avatar
mbmbmbmbmb
Does the computer play the rule that whoever invokes Ra must bid ?

I had a quick try last night, but thought I won an auction that I shouldn't have ....


N.

 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ylaine Gerardin
United States
Boston
Massachusetts
flag msg tools
Avatar
mb
Whoever invokes Ra only needs to bid if (1) the slate of tiles is not full and (2) no other player bid.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jens Hoppe
Denmark
Frederiksberg
flag msg tools
What are you, like 80?
badge
It's not the years, honey, it's the mileage.
Avatar
mbmbmbmb
Absolutely marvellous, Gabriel - thank you!

I agree with the other posters talking about usability: There does seem to be an unnecessarily large number of Continue and Confirmation buttons one has to press during the game. Also, I'd prefer a clearer indication of when an epoch ends, with a small scoring breakdown, perhaps.

Again, thanks, wonderful work.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jonathan Tang
United States
Portland
Oregon
flag msg tools
badge
Avatar
mbmbmbmbmb
Great implementation.

Just a quick observation. When using God tiles, it appears that the God tiles get placed on the board. I think they're supposed to be discarded when used.

Oh, and other vote for the removal of the Continue button.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ylaine Gerardin
United States
Boston
Massachusetts
flag msg tools
Avatar
mb
Not sure what you mean that it appears the God tiles get placed on the board. I don't think this happens - is it possible someone coincidently drew a God tile right after you used yours?
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Jonathan Tang
United States
Portland
Oregon
flag msg tools
badge
Avatar
mbmbmbmbmb
Snapper wrote:
Not sure what you mean that it appears the God tiles get placed on the board. I don't think this happens - is it possible someone coincidently drew a God tile right after you used yours?

Ahh, sorry. You're probably right.
It must've happened twice in a row when I was playing, because both times it seems like the God tiles reappeared on the board.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Michael Marvosh
United States
Boise
ID
flag msg tools
Avatar
mbmbmbmbmb
Found a rules misimplementation. The computer disallows you to take your turn when you have no bidding suns left, even if you have God tiles left.

Other than that, everything I've seen so far is fantastic! Thank you!
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ylaine Gerardin
United States
Boston
Massachusetts
flag msg tools
Avatar
mb
I don't believe you're allowed to use God tiles when you have no suns left.

My rules say-

"On a players turn, *if he has at least one face-up sun*, he performs exactly one of the following actions:
-Draw one tile from the bag
-play God tiles
-invoke Ra

Later, it says-
"As a player with no face up suns in his display area may take no more turns in the epoch..."

I take this to mean once you run out of suns, you can no longer use God tiles.

How is everyone doing against the AI? Has anyone tried the easy mode? Do people notice a difference in difficulty between 3, 4, and 5 player games? Seen the computer make any bone-headed moves?

Gabriel
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Mark Edwards
United States
South Attleboro
Massachusetts
flag msg tools
badge
Greatest secret agent in the world!!!
Avatar
mbmbmbmbmb
Snapper wrote:
I take this to mean once you run out of suns, you can no longer use God tiles.


This is correct. All you can do is be obnoxious and chant "Ra". ;-)

Quote:

How is everyone doing against the AI? Has anyone tried the easy mode? Do people notice a difference in difficulty between 3, 4, and 5 player games? Seen the computer make any bone-headed moves?

Gabriel


I've been playing 4 player games exclusively (not sure why, but hey) and after a few losses I've started to win about 80% of the time. The AI seems to undervalue monuments a bit and is terrified of taking the '1' sun to the point where it gives up some good stuff.

I'll third the "confirm" thing, it's a click too many. And the epoch progress needs help. I'd rather have the little red dots ala BSW.

All that said, it's a fantastic piece of work Gabriel and it's much appreciated!!!


 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Stephen Stewart
United States
Visalia
California
flag msg tools
badge
It's sooo Hot out here...
Avatar
mbmbmbmbmb
Looks good, but doesn't the least amount of Pharoahs only lose 2 instead of the listed 5 points?

Just checking
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ylaine Gerardin
United States
Boston
Massachusetts
flag msg tools
Avatar
mb
Oops. I assume you're talking about the Readme because I think it's correct in the game. Good catch. I'll fix it and re-upload it.

Anyone else have any comments on the Readme?
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Bill Koens
United States
Watsonville
California
flag msg tools
badge
marks the spot.
Avatar
mbmbmbmbmb
Let me chine in and say that this is wonderfully done. I'm 2 for 5 vs. the AI right now, with every game being close (except the first, which for some reason I ran away with.)

Some suggestions/quibbles...

1. I'll second the motion to reduce the amount of clicking. The confirm is nice sometimes... maybe confirm after bidding or a God tile use, but not after drawing a tile.

2. It would be nice (it's a VERY minor quibble) to have the 8 different possible Gods in the draw pile. You could keep the display in the players holdings the same. But this is minor.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
1 , 2 , 3 , 4 , 5  Next »   | 
Front Page | Welcome | Contact | Privacy Policy | Terms of Service | Advertise | Support BGG | Feeds RSS
Geekdo, BoardGameGeek, the Geekdo logo, and the BoardGameGeek logo are trademarks of BoardGameGeek, LLC.