Why is Viticulture’s AI so challenging?

If you’ve read my review of Viticulture you’ll know that I was impressed with not only the challenge provided by the AI in the app but also the advanced strategies used even at the easiest setting. Since releasing it I had a number of people reach out to let me know that the main reason Viticulture’s AI was such a success was due to the fact it was a neural network AI. Luckily for me one of those people was also Tysen Streib, who just happens to be the man who developed the AI itself and thankfully was willing and able to explain to me exactly what that meant and why it worked where other board game apps fail, in creating an AI that can beat a human player.

Pretty much everything you read from here on in are excerpts from Tysen himself, dumbing down for my benefit the complex world of AI development. I hope you enjoy pulling back the curtain on the Viticulture app and can begin to appreciate why it is you are being regularly beaten when playing against the computer and why this is such a marked improvement in board game app AI compared to the norm we play against and regularly beat.

Before we go in depth, lets take a step back and describe how AI’s typically work without a neural network. Some AI’s are just a list of rules the developer puts together. If you have less than 2 coins, then go to the bank, etc. however these generally are very weak AI. If you want a thinking AI, the designer will create what’s called an evaluation function that tries to give a numerical score to the game position that predicts who the final winner will be.  For a game like chess you would start with things like pawns are worth 1 point, knights are worth 3, and so on.  Once you had the basic evaluations you can then add in more complex features like additional points for how far advanced the pawns are down the board, bonuses/penalties for king safety, etc.  You can add anything you like, but the designer has to come up with the feature themselves and build them into their AI. 

Once you have all of this information you will see your evaluation function as an equation for example 1.3x + 4.6y – 2.9z = result, except you’ll probably have a few dozen variables. That evaluation function will add up all the features you give it and then come up with a prediction: I think Black will win this game 59% of the time.  The AI will then look ahead and do a search of potential moves and evaluate each board position in the future. If the AI was playing as the White player in chess it can then pick the move that it has calculated will give the best result from the evaluation function that brings that percentage down from 59% the most. Repeating this each turn is how an AI will set about trying to switch the win percentage in their favour until the game is over and they have hopefully won.

However, in game AI, you can use what we call a neural network instead of one of these hand-crafted evaluation functions.  Instead of a few dozen variables that you’re adding together and are being defined by the developer, you have thousands or even millions. Neural networks take inspiration from animal brains, where neurons and synapses form complex connection patterns.  For games, it’s different kind of evaluation function that teaches itself what features to add into it’s calculations instead of having them predetermined. It’s a way of the AI determining what is actually important when it comes to winning a game and weighting actions accordingly rather than taking that information from the inputs of the developer.

The Viticulture AI is entirely self-taught and was provided with nothing but the rules of the game, no understanding of the value of cards, grapes, wine or workers.  It starts off playing completely randomly and begins building an evaluation of what causes a player to win when everyone is random.  It will soon pick up the obvious things such as having more victory points is better, and it slowly begins to pick up more nuanced thinking such as a player will win more when they use visitor cards more and that planting vines are great in the beginning of the game and then not so much in the later rounds.  It can learn that the Reaper card is very valuable if you have 3 good fields to harvest, but maybe it’s not that great in other cases. 

As the AI becomes smarter and plays against itself over hundreds and thousands of games it can start forming more complex behaviours and see how the different actions come together all by itself. It does this after a game is over, by reviewing each move of the game and trying to tweak the evaluation a little bit so that the predicted result is closer to the actual result.  For example it will look back at a position earlier in the game where it thought Player 1 would win 68% of the time, but Player 3 actually won.  It will tweak the parameters of the network so that in this kind of position the evaluation of Player 1 is a bit lower and the evaluation of Player 3 is a bit higher.  Repeat over a few hundred thousand games and you’ve got a top-level opponent.

These hundreds of thousands of games played by the AI give it a unique insight into how important all aspects of the game are, the relation of each action and card and how playing each one impacts the percentage chance a player will win the game. Thankfully we are also able to take some insights into the game from what we can see the AI learning over time. The Mamas and Papas are a prime example of this and how, thanks to the AI, we are able to begin to see how they impact the chance a player will win the game.

Tysen had the AI go through all of the Mamas and Papas cards and do a few thousand 3-player games with them. He then recorded the win rates of each and ranked them by whichever gave the highest win rate when chosen by the AI. The average win rate is 33% because it’s a 3-player game however the win percentage for the Mama and Papa cards varies from this average, if a Mama or Papa had a win rate higher than 33% we know that it is more likely for a player to win using that Mama or Papa and if it was lower then it is less likely. What we learnt was that the Mamas are more balanced than the Papas, with a smaller range of win percentages and closer to the average win rate of 33%, and that the Papas which give you an extra starting worker are the best, winning around 47% of the time. In fact the win percentage for the Papas varies wildly; an AI dealt Rafael with his extra worker will win the game twice as much as one who is dealt Christian with his pre built irrigation. It turns out family really does matter when it comes to the world of Viticulture.

If we look at a more specific example within Viticulture, we can see how the AI will respond in certain situations based the neural network it has created and the assessments it has made throughout it’s hundreds of thousands of games played. Imagine I have just started a game against an AI and we’ll look at some moves. Below is a screenshot of the game and we have fast forwarded to the start of Year 2. 

Although we can’t see it, the AI has a wine order of a 5 Blush and a 6 Blush, and 3 visitor cards: Artisan, Agriculturalist, and Benefactor. They also have Irrigation. The AI can’t see my cards, but knows I have a Yoke and a Cottage. With that information the AI is now considering the wakeup order for spring, I have already taken the first spot and so it estimates its chances of winning if it picks each of the remaining spots:

It apparently thinks it’s already winning as it’s predicting that the chance it does is well into the 60% range, most likely because it was able to purchase 2 extra workers early and I was only able to get 1. Based on it’s predictions the AI picks the extra point in spot 6.

As I’m the first player I sell my field since I wasn’t able to last round and now the AI needs to decide where to place its worker.  It searches ahead, predicting where I will move and winning chances for each move, followup, and reply.  All in all, it searches about 5,000 different positions in about 1 second. It finally believes that it’s chance of winning for each action is as follows:

As playing a visitor gives the AI the best chance of winning it selects this action, however the only visitor it can play now is the Artisan, which gives a choice between getting 3 coins or building a structure at a discount.  It rates those choices as:

Notice that it’s overall estimation of winning has slowly come down from 66.8% to 65.1%. This is due to the AI being able to search a little bit farther after every move since one more option is made certain and it doesn’t have to spend resources searching moves that didn’t happen.  Looks like it was a little optimistic earlier.

As you can see from this example, based on the learning and optimisations the AI has made from the hundreds of thousands of games it has played it has determined how likely it is to win in each situation and is making it’s choice by whichever is going to give it the highest percentage win rate. As you make moves and the options available for the AI change, so does the percentage chance the AI believes it will win.

While this method of AI proves to be a strong one, criticism which has been levelled at the Viticulture AI (specifically the Hard AI) is that for a game based around making and selling wine, the AI doesn’t do very much actually making and selling of wines, instead opting for the use of cards and buildings to generate winning scores. Weighing balance, play style and ‘human-like’ play are all part of the challenge of creating a good AI however the fact that the neural network AI have developed strategies which goes against conventional wisdom is just a further demonstration of the impressive nature of this sort of AI.

For Easy and Medium AI, Tysen doesn’t mind shaping their strategies in certain ways; he will usually try to bias their play in ways similar to how a human beginner plays as it seems more beginner friendly and the right thing to do and thus you will see the Easy and Medium AI attempting to fill orders and play in a way which is perceived to be the ‘right way’ to play Viticulture. However he will let the Hard AI use the most optimum strategy to win regardless of whether criticism is laid at the AI because playing the no-wine strategy is what it takes to win.

You’d therefore be right to question, why a neural network AI is not always used in board game apps and why it is that most apps are still struggling with weak AI opponents. Well, the downside of this type of AI compared to those based around an evaluation function is that the neural network AI is much slower than a simpler function and therefore you can’t search as far ahead within a game, especially if you hope for it to run on our relatively underpowered mobile devices; you can examine a few thousand potential positions within a game instead of a few million.  However the reason this is possible within Viticulture is because it’s not that valuable to be able to search too far ahead because of the random card draw that is present. Visitor cards drawn each autumn immediately change the strategies for all players and thus the need to look too far ahead is negated by the ever changing landscape. The bots in Viticulture don’t cheat, so they don’t know what cards they will get when they draw and they also doesn’t know what cards the opponents hold.  That makes it much harder to predict what moves will happen in the future.  They can however see things like “that player has a lot of Winter visitor cards, so he will probably go to the visitor spot” and form strategies around that based around that.

With the success of the AI within Viticulture can we expect to see neural network AI appearing in more of our favourite board game app releases? Unfortunately the answer to that isn’t as clear. Tysen himself knows the challenge, time and cost involved in developing a neural network AI having attempted to create one for another of Digidiced’s properties Terra Mystica (you can read his full developer diary for this project here) and having to abandon the project. The main reason a neural network AI was successful in Viticulture (and similarly for Race for the Galaxy which also runs a neural network AI) is the lack of spatial positioning on a map as one of the variables to consider. If you had a version of Terra Mystica where you just grew your engine on your own player board without a map, then a neural network AI would excel quickly learning and developing a top level AI opponent. Getting the interactions of neighbours and planning the layout of your towns in Terra Mystica is just almost impossible to quantify efficiently especially for something that has to run on a mobile device. The fact that Viticulture doesn’t have the spacial implications of tile laying or a map means that Tysen was able to use a much smaller network for Viticulture and have it work even on mobile devices. And while we will see more challenging AI in digital board games thanks to a neural network it may only be in games where the network is capable of being small enough to run on our phones and tablets although newer devices are beginning to change that.

So if you want an answer to the question of why is the AI in Viticulture so challenging, the simple answer is that it’s played more games of Viticulture than you will play in your life time and realised that Viticulture is not a game about making wine and fulfilling wine orders, it’s a game about gaining victory points as quickly as possible. And while you may not like it’s way of playing a game based around wine making and selling, don’t hate the player, hate the game.

A special thanks for this article goes to Tysen Streib who kindly answered all of my stupid questions on digital board games and AI, he’s a pro in the field having designed AI for titles such as Indian Summer, Patchwork, Isle of Skye, Stockpile, and Castles of Burgundy as well as being a published author based on his work with AI and the strategies it produced within the poker field. You can read more of his work in his books on poker ‘Kill Everyone‘ and ‘The Raiser’s Edge‘ as well as find him on Twitter at @TysenStreib