CW's Guide to RPG Reviewing
Let's begin by examining the most obvious criteria, the physical object. Even this aspect of the product has a mix of subjective and objective criteria, and most aspects of the book will involve a mix. Of most immediate note are qualities such as cover art, cost, binding, interior artwork, formatting, editing, and indexing. Cover and interior artwork are similar. Unless the cover is an elegant affair then it should be judged just as the interior artwork. Interior artwork needs to inspire or inform as it appears throughout the text. Most importantly, though, it needs to be connected to the text. A page about cannibalism in a horror game should not feature a cartoon bear, to offer an extreme example of a poor connection between artwork and text. RPGs will vary greatly in how well they connect artwork with text, though today most do a fine job of it. Nevertheless, it is the most objective of artistic considerations and the easiest to evaluate without having to become an art critic.
Of course, being an art critic is part of the job. Pay no heed to those who claim that artwork is completely unimportant in an RPG. They tend to argue that it's only the game in play that matters, and in spirit I agree with them. A great game, however, inspires the reader at every turn and it does this through artwork. One final thought on artwork before moving on: consider that even low budget games can make good use of artwork. Just because a work is a small press game doesn't mean it gets a pass for using bad clip art throughout, as many authors have found a way to include art on a tiny budget that does a fantastic job of supporting the game.
Binding is only mentioned when it causes a problem. If pages are falling out or the book is unstable, especially if this happens after a little bit of play, then it may be worth mentioning. I recommend checking with other book owners before evaluating this as a problem, however, as it's possible that a given reviewer's copy is simply defective. Further, some publishers will hold on to problem copies for the purpose of handing out to reviewers and it's important to realize this before declaring a book's binding to be poor.
Formatting, editing, and indexing can be more difficult for the new reviewer to judge in some respects. Formatting should result in an easy to read text, first and foremost. White text on a pale background, tiny font, or anything else that makes reading the book a headache should be considered a problem. Great formatting also results in an artistic presentation of the material, with interesting borders and a nice consistency of fonts. Editing, closely related in concept, is more than just misspelled works and grammatical mistakes. Words are rarely misspelled today, thanks to computer editing, but incorrectly placed words that are spelled correctly have become more common (e.g. “next” instead of “text”).
The index should be judged by how useful it is. After all, the index is included as a tool to assist the reader in finding information. A good index will bold particularly relevant pages to cut down on the number of pages the reader has to reference to. A good index anticipates what a person playing the game might want to know about, indexing references to rules alongside game and setting concepts.
Those considerations are only the tip of the iceberg, however, as they have little to do with the actual game being evaluated. Our next batch of considerations include purpose, innovation, mechanical implementation, and inspiration. A subsidiary consideration, quality of fiction, is rolled into inspiration. Each of these is abstract and encompasses a variety of concepts, and you'll find that every reviewer weighs different qualities of a book in different ways. There's nothing wrong with staying at an abstract level while you discover what qualities of a game are really important to you.
Let us begin again by focusing in on the purpose of a book. I'm sure you're familiar with the simplistic academic exercise of determining the purpose behind an author's writing. While it's something we've all had some experience with in school, now is a great time to dust off that old skill. Every author of an RPG book has a goal, but we can't be so general as to say that they're writing to entertain us or to create a game for us to play. By narrowing our evaluation we can cut to the heart of the writer's purpose here. Consider, for example, a horror game. Is the game about telling horror stories? Dark action stories? Does it set out to enable the group to craft the story with meta mechanics? Does it remove mechanics related to player authorship, and if it does is the goal to emphasize horror play by removing player power within the game world?
The goal isn't to be a mind reader, but a good reviewer does not need to be one. A good RPG clearly conveys its purpose, often with explicit statements among the introduction, text on the back cover, or simply through the mechanics themselves. An RPG should be criticized when it lacks purpose or where the text does not support the author's purpose. For the horror RPG example, if the fiction, artwork, and stated goals of the game are all to produce grim horror and the mechanics instead produce high action and adventure then it may be the case that this game has a flaw.
Coherence of purpose is just one consideration, though it's more impartial than that of innovation. Innovation is a criteria that some reviewers may not care for, and it's up to you to decide whether it's worth considering as part of a review. Some reviewers believe that yet another d20 based game, or another d6 dice pool system, that does not do any more than merely duplicate what has come before should be criticized as a mere reproduction. I agree with this view, but only to an extent. There is merit in reproducing a great system where that system fits with what the author sets out to do, though I do look for ways in which that system has been altered to make it work well with the new setting or game goals.
Speaking of dice systems, mechanics is a solid topic for consideration with any RPG. Some games will have dice abnormalities that result in bizarre probability curves or serious mechanical problems. On the less extreme end of the problem spectrum are games that purport to have mechanics that accomplish one goal, but actually have mechanics that do another. A gritty game simply isn't gritty, for example, perhaps because the mechanics that encourage characters to be cautious aren't serious enough to produce a gritty feel at the table.
Yet another aspect of game mechanics is game balance. Not all games are about game balance. The classic example here is a super hero game more interested in what the characters can do then how powerful they are in relation to one another. However, many modern games do strive for a measure of game balance so that a variety of different character concepts are all just about as effective as one another. In a fantasy game where spellcasters are incredibly effective in all situations and warriors are not, and where the game is about playing a mix of characters, then it's perfectly reasonable to criticize the game for a lack of game balance.
On occasion a game comes along that does something revolutionary, or as near to it as our hobby experiences. It has an exceptionally imaginative setting, a system that offers a new type of play experience, or otherwise showcases the very best we might expect from a publisher. This, the last topic I'm presenting, is inspiration. Inspiration is most often found in the text. While reading the book the reader wants to go be that character and do that thing. The reader wants to stop reading the book and start playing. The reader wants to tell all their friends about it and assemble a group as soon as possible. While it's at least in part a subjective evaluation, the degree to which the book inspires the reader to want to tell stories about the adventures in a given game world should be considered as important as any other quality. Even a game with great mechanics, if it's dull as dishwater, is unlikely to actually make it to the gaming table.
In conclusion, a reviewer should consider a broad variety of criteria involving the physical aspect of the product and the game in play. There are even more criteria that could be included in a greater discussion, but this is plenty to think about for anyone wanting to think more critically about how they evaluate games. As long as you've thought through your criteria they aren't wrong, so don't be afraid to weigh one area more heavily than another so long as you're consistent and understand why it is that you have reached this decision. Critically reviewing a game can be a thought provoking process, and I hope the next time you're looking at a game you'll think about what you believe makes up an excellent RPG.
