As anyone who has ever put an opinion on the internet will know, there’s always going to be someone who violently disagrees with you regardless of your stance. This recently happened to me based on my good but not great review of the board game app Terraforming Mars. I was contacted by someone who wanted to tell me I was completely wrong (I’m wrong more often than I’m right so I’m always happy to hear the reasons why) and to complain that I only gave Terraforming Mars a ‘bad’ review because the publisher hadn’t paid me for the review unlike the others who I had given good reviews to. Frankly I was flattered to think that someone who reads my reviews thinks that someone would actually pay me to review board game apps (I can but dream) however it made want to pull back the curtain a little on what running a site like 8bitmeeple.com is actually like, what working with developers and publishers entails and whether my impartiality has ever been compromised in my year as a board game app reviewer.
Luckily for you, and in the spirit of impartiality, I asked my friend Chris over at pixelatedcardboard.com, THE board game app reviewer, to pull back his curtain too so you can see what life is like for more than just me and get a true sense of why on we do this, what it takes to run a board game app review website and what we get out of it.
Why start reviewing board game apps?
If you’ve ever spoken to anyone in the tiny niche within a tiny niche that is board game apps you will know that no one goes into it hoping to find their fortune or retire early, whether they’re developers, publishers, and least of all review sites. The one thing they all have in common is they have a passion for board games and a want to be in the industry in any way they can. When I decided to start 8bitmeeple.com I had around 25 board game apps on my phone and was looking for information about which others I should get and so took to the internet to find some more information. I was shocked at just how little coverage they seemed to get and found that there was only a handful (which is probably being generous) of sites talking about them with any kind of authority. It wasn’t enough to know that a board game I liked was also available as an app, I wanted to know if the app implementation did the game justice and would be worth buying and playing. This kind of information was just not easily available save for pixelatedcardboard.com and so I decided I’d give it a go and see if I could make a place where people could come and get opinions on their favourite board games in app form.
Chris – I started because I wanted to find digital board games I could play with my friends. That meant cross-platform games with asynchronous play. There wasn’t any single resource out there to find games that fit this bill, it mostly involved 1) learning that a digital version of a game existed and 2) reading the app store listing for the game to see if they mention the features I cared about. It was quite a pain. That was the motivation behind the site, it has turned into our Review Database which I hope is as useful to others as it would have been to me way back when!
What does the review process look like?
As with everything created online, I think there’s a lack of understanding at how long things take to make and reviews are no different. Believe it or not, if I was to be given an app to review the quickest I could probably turn it around would be about 24 hours and that’s only if the game is a quick 5 minute game. Each review requires me to play the game multiple times both in solo modes, tutorials, against each AI difficulty, pass and play and online which soon adds up. I estimate that before I have even started putting pen to paper I will have played the board game app at least 25 times and usually far more than that. You can see how time spent on a review can really add up with some of the larger releases such as Through the Ages or Root which have a play time of 30+ minutes.
Once I feel like I’ve played the game enough to be able to give an informed opinion I set about the process of writing the review using my predefined framework. This framework gives my reviews a structure and give me a place to start, getting down my thoughts while ensuring that I cover all of the areas that people think are important when reading reviews. I like my reviews to sit between 1,500 and 2,500 words, long enough to be able to give some in depth thoughts but not so long that it’s unreadable. Writing this will take me around 4-5 hours at which point I need to go through another playthrough of the app to collect all of the screenshots I need to include in the review as well as anything I’ll need to create the photos and banners for the site. Adding this colour to the articles, the images, my review scores and updating the database takes another hour of work before I’m ready to put it out into the world and get started on the next review.
It may seem like a lot of time spent to write a review of a sub £10 board game app however my process has evolved to this after I decided that I wanted this site to be informative, interesting and in depth. The amount of time developers and publishers put into their games mean that I want to do their apps justice when I review them, I want the readers and the developers to know that the review score I’ve given them is based on time and effort and know that if I like or dislike the app it is not on a whim.
Chris – My review process really depends on the game. Not just the weight of the game itself, but the number of features the digital version has. Something like a solo, off-line only game can be reviewed fairly quickly in my book. Opinions will start forming after a handful of plays and that gets stretched out into a review sometimes within a few days of getting the app. Even if it is a very light, easy to learn game, the review can take quite a bit longer if the app is feature-packed with having to test out various game modes, online and off, and generally just play a lot of games to find any potential issues. I think for the heaviest, most feature-packed games, a review might take two or three weeks, but it depends how much time I have to dedicate to playing/writing.
How do you come up with your review ratings?
I know my ratings look low compared to other review sites. I know this because a developer friend of mine once told me that my ratings are low and it looks like I don’t really like any games. My highest rating currently is only 8.6 out of 10 so I can see his point a little bit. What I will say is that because generally most board game apps are great it really doesn’t help anyone if I just rate them all a 9 or 10, this really isn’t helpful for anyone. So instead I made the decision to lower my standard of excellence to give myself more room to distinguish between just how excellent all these games were. Ratings above 6 are very good, above 7 are excellent and higher than an 8 is a must own game; by doing this I was able to not limit my ratings and hopefully give a better sense of how good a game is.
To help with this I also decided early on to breakdown my review into 5 categories and give a total score as an aggregate of these. This gave me the flexibility to review a game with critical flaws in one area well without those flaws completely ruining the over all rating, after all, a game can still be a good one if it looks terrible.
The categories I chose were Gameplay, which really is about how the board game itself plays as well as how this has been translated to the digital tabletop. Presentation, how the game looks, how the user interface works and how easy it is to navigate. Challenge, whether the game provides any challenge for you, possess a solid AI, an Automa or how difficult it is in multiplayer. Longevity, how much replayability is in the game, have I got bored of playing it over and over again, does it have expansions available for me to change options, what modes does it have included, any campaigns etc. Enjoyment, simply whether I enjoy the over all package. The aggregate of these gives me the overall rating and determines a games score.
Chris – This is the question I get asked most, so apologies for the verbose response on this one. Most of my ratings, almost exactly 80% of them to be exact, land between 3.5 and 4.5 stars on my five star system. “Why don’t you post any negative reviews” is commonly asked. The answer is pretty simple: most of these apps are well made! There are very few *bad* digital apps out there. I have no issue dishing out low scores for these apps when I come across them, but they are, quite frankly, few and far between. The way the scene is today the vast majority of apps get published by a small handful of publishers who are, generally, reusing their frameworks which have proven to be pretty good.
There are two difficult parts of scoring to me. First, what if one feature is essentially broken? Say, as a random example and not a comment on one of the big publishers in this space, that a game is well made but online asynchronous games have some key issues like broken notifications or other weirdness. If you plan on playing the game solely as an async online experience, the review score might as well be 0.5. If you don’t care at all about online async play, and everything else is well made, that score is a 4.0 or higher. The games which have core, game-breaking issues are the ones that typically get low scores from me.
Second, how do I rate a game that I don’t particularly care for? I like what you have done with your enjoyment factor to spell out exactly how you feel about the game. Our site doesn’t use something like that so landing on a final score sort of has to be a mashup of quality, features, and personal enjoyment. The games I really love will get boosted, sometimes quite significantly, but I try to stay somewhat objective on games I don’t love and not kill their score over it. As an example, Spirit Island just isn’t the idea of a perfect game night for me, but I know a ton of people do love it and the implementation is nearly perfect in providing a deep solo experience. Those two things are difficult to balance out into a final score, but ultimately I try to land on a score that speaks to how somebody who enjoys the game might find the digital version, making exceptions when the fit to the digital space is overwhelmingly bad. I interject plenty of personal preference into the text and hope people read that for a more complete picture.
What does working with developers and publishers entail?
Working with developers and publishers was not something I expected to be doing when I started out; I had been buying apps for years and I expected that I would simply continue doing so and then review whatever I decided to buy. However I have been shocked and overwhelmed by the support I’ve received so far, without it I don’t think I’d have been able to review as much as I have.
Typically when a new board game is being developed I am approached asking if I’d be interested in reviewing it, asking me if I want to know more and helping me with information so that I can share the news about the game. More often than not I am lucky enough to be given access to the game before it’s release to the public which gives me a chance to do the necessary playing to be able to write a review to coincide with the launch of the game. Having this kind of opportunity really helps both the developer and me, getting eyeballs on both their game and my review.
Even though I don’t get paid for reviews I still play a tricky balancing act of keeping my reviews impartial while not souring the relationships I have built that enable me to run the site like I do. It could be easy for any negatives with the games to be taken badly however thankfully each negative so far has been seen as an area for development in the next release and has been taken as part of the review as a whole, as the counterpoint to the positives I raise. It’s this attitude towards reviews that has made it so easy to work with developers and publishers, to produce what I hope to be balanced and impartial reviews.
Chris – Most developers are very eager to help spread the word about their games and therefore are quite fantastic to work with. Most of them approached me on their own with review keys, early access, or requests for news stories. While I appreciate the review keys, I try my best to not let $3-$10 worth of free game slant my review in any way. It’s up to the reader to decide if they believe me or not, of course. There are some who are unresponsive, but those are few and far between. I’m certain much of my early following was due to these companies sharing our content early on, so I’m very thankful for the kindness of the developers.
What drives you to continue putting in the time and what do you hope for your site in the future?
Based on the fact that 8bitmeeple costs me more money than it makes to run I’m clearly not driven the the money in board game app reviews. It’s a side project that I spend far too much time on because I love the games, enjoy talking about them and like to think I make something that either helps or entertains people. There’s a few key milestones that have driven me to keep going over this last year, the first time someone read a review I’d written, the first time I was given a review copy of a game, the first time someone tweeted to say they were waiting to buy the game till after they’d read my review and the first time I received a physical press kit through the post from Nomad Games and my girlfriend said “What? So people actually read your site and care about what you say? Huh who knew!” before she went back to ignoring my ramblings. I hope to be able to keep doing this in the future, I want more people to find my little site and see it as a place to get information and start a discussion on board game apps and I hope to be able to keep playing as many board game apps as possible, no I don’t think I’ll be able to retire early off the proceeds of 8bitmeeple.com but I do think that when I do retire I’m going to have a pretty impressive collection of board game apps to fill my time.
Chris – The excitement in creating content for the site comes and goes. I’m in a bit of a “goes” phase currently, as I haven’t been too quick to get reviews up for recent games. I continue because I enjoy these games and enjoy the thought of helping other people find new games to play. And no, the luxurious life of a digital board game reviewer isn’t all it’s cracked up to be, so I definitely need another retirement plan!
If there’s one thing I’d tell my Terraforming Mars loving friend it’s that I’m really pleased he loves that app so much it drives him to start a dialogue with someone who views it very differently, board game apps are so under discussed even within the board game world and therefore any discussion about them is a good one. Hopefully you’ll read this and see that we don’t just pluck our reviews out of thin air, that the time we spend reviewing games far outweighs what we get from our sites much like the time spent making these games far outweighs the appreciation they seem to get and that while we’re not perfect, we all try to be as impartial as possible. Oh, and that no one pays us for our reviews, if you want to I’m pretty sure me and Chris wouldn’t say no…
If you don’t already you should definitely be following Chris on his website, Twitter and his Discord where a bunch of us chat, play games and discuss board game apps but what am I saying, if you’re here you obviously already follow him!