Spacer The Oracle Spacer
GreenLine
About Spacer Home Spacer Authors Spacer Books Spacer Links
GreenLine

A Set of Rules for Game Reviews

Guidelines for the writers -- and the readers

Note: This article was originally published in issue #80 of Dragon magazine.

by Ken Rolston

A reviewer of games assumes a great responsibility. Though it is said with some truth that even a negative review will sell more games than no review at all, the impact of a bad review on the career of a game designer and the reputation of a publisher can be very harsh, while a good review can increase sales. When money and reputations hang in the balance, the integrity and professionalism of the reviewer must be above reproach.

Unfortunately, little has been said in print about the standards a reviewer should observe in his work. The purpose of this essay is to open a discussion of reviews and criticism of games, and of the responsibilities incurred by reviewers, editors, and publishers. The essay is directed not only at those who write and print reviews, but at those who read them, on the theory that those who understand the issues involved will be better able to judge the quality and reliability of the reviews they read.

How important are reviews?

Ironically, the delay between the release of a product and the publication of a review is too great to offer purchasing guidance for most readers. For many of the game products in this industry, 50% or more of the product's sales come in the first quarter (three months) after release, and most reviews are not published until at least 3-4 months after a product's release. Most gamers like to get new releases, preferably as soon as they are available. Such purchasers are at the mercy of skillful promotion and advertising. They may have been bombarded with colorful and enticing ads for months in the magazines; compared to inspired advance ad campaigns, a review that appears many months later has predictably a proportionately small impact on the purchasing behaviors of game fans.

In this sense, there is a good case for discounting the importance of reviews. I infer that the relative dearth of articles, essays, and opinion papers published on the subject of game reviews is a result of a pragmatic resignation on the part of most would-be critics and theorists in game reviewing and criticism. After all, why make so much of a fuss if reviews have so little significance?

It would be difficult to compile reliable data on the relationship between reviews and sales, though common sense suggests that reviews must have some effect. Even more intangible is the influence reviews have on the reputations of designers and publishers. There is no way to measure the dollar value of a reputation, but a gamer is more likely to recognize and purchase a game by a designer or publisher whose products consistently receive good reviews, even to the point of the development of cult followings for the acknowledged leaders in the field. Most important, readers of reviews may receive benefits from reading reviews; if they are patient enough to wait until they've read reviews before they buy games, it may help them to avoid wasting money on turkeys or items that turn out to be something other than what they appeared to be. Further, reviews can inform the reader about the publication of games he will find enjoyable.

These readers are the heart of the matter, and the primary reason why we should consider how to improve the professionalism of game reviewing. Discussions of whether reviews are important or not too often center on the effects of reviews on the game industry; when viewed from the perspective of the individual consumer, the issues are much clearer. To those who read reviews, they are undeniably important. It is to those readers, and the reviewers who care about them, that this essay is directed.

Three types of reviews

The interests of getting a review published promptly are directly in contradiction to the interests of getting a detailed and reliable review. The more thoroughly a game is studied and playtested, the more reliable the critical judgement of the reviewer; but careful, critical reading and playtesting of game rules and supplements takes a great deal of time. There is no such thing as a full-time, professional games reviewer; most reviewers are fans or game designers who do a little reviewing on the side. Many games require hours to learn the rules, then hours to play, and one may only find or recognize a serious flaw in the design after half a dozen playings. This may be too much effort to expect of even the most conscientious reviewer.

Role-playing games are particularly difficult, considering the volume of text, complexity of rules, and the need for several playtesters. Further, the enjoyment of these games is greatly dependent on the quality and imagination of the gamesmaster and players. How clearly can a reviewer distinguish between a flawed game and uninspired playtesters?

On the other hand, an experienced gamer develops a feel for a good game which is less dependent on extensive study and playtesting. He can often tell whether a game is a winner or a stinker just by spending a half-hour examining it. His approach to evaluating a game is less systematic and detailed, but may be just as reliable a method of judging a game's value. Nonetheless, a game designer or publisher might justifiably resent a pronouncement based on such a cursory examination.

To better serve the needs of readers, I suggest three different types of game reviews, each with its own particular objectives and acknowledged limitations:

1) The capsule review. This review comes out as soon after release as possible. It should be very brief (100-1,000 words), making no pretense at exhaustive analysis or playtesting. The reviewer need not apologize for not having spent hours with the game before arriving at his judgement; his function and the limit of his responsibility will be implied by the brevity of his analysis, and should also be stated explicitly in comments at the beginning of his review. This will permit the reviewer to handle a larger number of reviews with a minimum of invested time. His review should primarily address any distinctively new features of the game and consider whether the reader will feel justified in spending his money on the package. The reviewer must be a very experienced gamer, confident enough to "shoot from the hip," with a strong feel for game quality. He should accept the premise that initial impressions may be deceiving, and that he may blow a call from time to time; he must console himself with the fact that he provides prompt and valuable information to a prospective buyer.

2) The feature review. This is a more extended review (1,000-3,000 words) which assumes a greater reviewer responsibility for analysis and playtesting. Ideally the reviewer should have done most of the analysis and playtesting, but he may rely heavily on the comments of others who have also read and played the game. This review would not be expected to be written any earlier than one or two months after the release of the product, when there has been some time to judge the response of purchasers to the product. Such a review would be appropriate only for "important" or "significant" games. (Assessing what is "important" or "significant" is not an exact science, and need not imply good quality. Unhappily, a successful advertising campaign for a terrible game may qualify it as a very important game . . . to be avoided.) Ideally this review will offer commentary into useful strategies, applications, or game variants -- subjects of interest to the reader that don't fall strictly within the function of a game reviewer, but which are valuable in their own right.

3) The critical review. These detailed and substantial pieces would address the classic masterpieces (and occasionally the classic turkeys) of gaming. They would usually be written by an acknowledged fan or critic -someone thoroughly familiar with the game and its place in the history of game development. For new gamers, these articles would provide necessary background for understanding the current state- of-the-art game, while introducing them to treasures they may have missed. For experienced gamers, such articles might stimulate waves of nostalgia and a desire to pull out that old chestnut and give it a second look. For game professionals, it will consider the question of "what is a good game," and from study of its virtues, and analysis of its limitations, new games and game systems may evolve.

The reviewer and his audiences

As a reviewer prepares his commentary, he should keep in mind the three types of readers he may be addressing. Each has particular interests that should be taken into account in shaping the review. The writer should also keep in mind the background and sophistication of the specific audience of the magazine he is writing for, neither patronizing the readers nor baffling them with unfamiliar and specialized jargon.

1.) For the reader who has already bought the game, the reviewer should help the purchaser to evaluate his own response to the game. Many of us are unclear about our own standards and motivations when we buy a game. We may feel vaguely pleased or ripped off, but it may take another's analysis and verbalization to give us a handle on our real attitude toward a game. For example, I may be initially pleased by the style and flavor of an FRP supplement I have bought, but feel puzzled and disappointed as I notice it gathering dust on the shelf. A reviewer may draw attention to the limited utility of such a package, in spite of its surface qualities that may be attractive, and may enable me to more carefully shop for products that I can use easily in the future.

2.) For the reader who is a prospective purchaser, the reviewer should give guidance on how to get your money's worth. The writer should attempt to counterbalance the impact of hype and advertising, and draw the reader's attention to the difference between substance and appearance. He should particularly inform the reader about superior products from small companies with limited advertising budgets and small distribution networks. I think of this as a sort of "people's lobby," an attempt to encourage publication of quality games by providing publicity and support for companies, based not on the basis of the capital they marshal, but on the quality of the games they publish. The reviewer should also offer suggestions to the reader on how the usefulness or flexibility of the game may be increased. Particularly with FRP supplements, a product's adaptability to various systems is of particular interest to a reader. Derivative, unoriginal games that are misleadingly marketed as new releases are a specific class of turkeys that a reader must be warned against.

3.) A small but significant portion of the audience addressed by a reviewer is the collection of serious fans and amateur and professional game designers and publishers who are interested in the art and craft of game designing itself. For this audience, the reviewer speaks to acknowledge achievement, encourage improvement, and criticize flaws and faults in game design. Word-of-mouth, sales, comments from peers and competitors are all important sources of feedback to designers and publishers, but formal reviews probably have the greatest impact on reputations when viewed from outside the inner circle of fans and professionals.

Features to assess in a review

The following is a list of characteristics which should be discussed in a review. Depending on the length and purpose of the review, not all of these features would receive comment. (This list reveals my predominant familiarity with role-playing games; for historical wargames and boardgames there may be important features I have overlooked.)

1.) Time and effort necessary to learn.

2.) Playability.

3.) Presentation (layout and illustration, writing and editing, game design and development).

4.) Game quality (originality, effectiveness of simulation, dramatic potential, general effect).

5.) Rules quality (completeness, originality, organization for reading and reference, clarity and freedom from ambiguity).

6.) Complexity (intentional level of detail - not to be confused with games that are "complex" because they are badly designed or organized).

7.) Explicit and implicit applications.

8.) Description of contents (in terms of perceived value: Does it seem like a reasonable value in terms of the price?).

Style guidelines for reviewers

1.) The first paragraph (preferably the first sentence) should contain a clear indication of the reviewer's opinion of the game. It should also mention the designer(s), publisher, and cost, and briefly describe the content and purpose of the product. (In- depth and critical reviews, because of their greater length and broader purpose, are less obligated to observe this dictum than the capsule review.)

2.) The final paragraph should summarize the reviewer's opinion and support for that opinion. Any reservation about his judgement (due to his lack of playtesting or admitted preconceptions or prejudices) should be stated here. If appropriate, the reviewer may wish to call the reader's attention to other products by the same designer or manufacturer.

3.) The length and detail of a review ought to be some indication of the significance of the product. A review of 500 words should not be sufficient to discuss a new role-playing system; neither should 3,000 words be needed to discuss a routine game supplement.

4.) That a reviewer has read a game ought to be taken for granted. That he has playtested the game in many cases may be less than obvious. If a reviewer omits mention of his playtesting, it may be inferred that he has not playtested the game. If the game has been played more than once, that fact ought to be mentioned. If a reviewer is unwilling to admit in a review that he has not playtested the game, he shouldn't be reviewing the game.

5.) Too much style and ego-tripping will certainly interfere with the quality of a writer's commentary, but considering the work and responsibility he assumes and the relatively small reward he can expect, it is fair to indulge him somewhat in these matters. If a reviewer can be entertaining and still perform his primary functions, all the better, but entertainment is not a necessary feature.

6.) Unless the target is undeniably deserving, ridicule and derision are inappropriate critical modes. No matter how appalling a product, there are equally effective alternatives to spitting and invective. (I must confess to a fondness for satire and irony, but the emotional responses they evoke may obscure the critical purposes of the review.)

7.) Never underestimate the fan's innocent ignorance or his willingness to be instructed.

Editor and publisher responsibilities

Ultimately, the editor and publisher are responsible for judging the professional qualifications and performance of a reviewer. It is the editor and the publisher who must decide whether a review will be published. These persons must be very careful to screen all reviews to the best of their ability. They should solicit reviews whenever possible from those who have established their credentials as reliable reviewers. Unsolicited reviews must be judged on their own merits, and should receive close scrutiny by the editor until he is satisfied with the reviewer's reliability.

An editor has no responsibility to review a game just because he is sent a free copy. He must make the practical decisions about how much space will be devoted to reviews, and what products ought to be reviewed in that limited space. There are many more games published than can be reviewed in a single magazine. It is the editor's responsibility to decide which materials are important enough to be reviewed. He is also responsible for specifically soliciting and assigning reviews for more important releases; he must therefore keep current his knowledge of new products as they reach the market. An editor should also publish designer's or manufacturer's rebuttals and corrections to reviews when they are reasonable in their statements and length. These designer and publisher responses deserve respect and publication, even though they are inevitably self-serving. They are an important form of feedback to readers, editors, and publishers concerning the reliability and accuracy of their reviewers.

An editor is ultimately responsible for the quality of the reviews in his magazine, but it is very difficult to judge the accuracy and reliability of a reviewer without feedback from readers. If a reviewer writes clean, reasonable, literate copy, it may get past an editor even if it is blatantly inaccurate and unfair. It is the reader's responsibility to help an editor judge the quality and usefulness of a reviewer. Examples to support opinions

Because of the limited space allotted to most reviews, particularly in a capsule review, there are practical reasons for not supplying detailed support for statements of opinion. One or two examples will usually be sufficient to illustrate the virtues or weaknesses of a game. A review is a value judgement; the reviewer must not get caught in the trap of believing his judgement to be a simple matter of fact and logic. I have seen extensive, closely reasoned rebuttals of reviews from game designers and manufacturers to support conclusions directly contrary to those of the reviewer. The point is not the facts or the logic, but the interpretation made by the reviewer, and though a reviewer's judgement must be ultimately rooted in fact and logic, it cannot thereby be irrefutably established.

Final notes to readers

Readers: Remember that reviews are no substitute for the other important methods of evaluating games. And these are:

1.) Examine the contents of the booklets and boxes. Go to your game store or local game convention and look through the package before you buy. Don't grab the game with the flashy cover and the hum- drum contents. Don't succumb to the shrink-wrap intimidation device; if the only way the merchandiser can sell a game is sight-unseen, you may have good reason to wonder about the quality of the game.

2.) Play the game first. Go to a demonstration at a store or a convention, or borrow your friend's copy.

3.) Check the other informal "reviewing services" -word-of-mouth and fanzines. Buttonhole people in the stores or at conventions and inquire politely about their opinion of the game.

4.) Don't let your money burn a hole in your pocket. (Unless you have enough to burn, and you enjoy burning it, which I freely admit is a lot of fun.) Rarely do you really need to have that game or supplement as soon as it's published. Too often we submit too easily to the lust for ownership; instead, cultivate a lust for quality and utility, even if you aren't the first one on the block to own the product.

The skeptical reader

Finally, the reader must take each review with a grain of salt. Carefully separate statements of fact from statements of opinion. Make sure that statements of opinion are supported with at least some examples. For instance, a comment that a product is "badly written" is useless unless the reviewer makes it clear what he means by using examples. Less weight should be given to reviews that predominantly express general opinions without supplying supporting description or analysis. More weight should be given to a review which shows evidence of careful playtesting; many games only reveal their virtues (and surmount their flaws) under regular play conditions. The more familiar you become with the style of a specific reviewer, the more useful he will be in helping you evaluate a product's value. Over a series of reviews it is easier to determine accurately what the reviewer's tastes and standards are, and you will have more opportunity to compare them with your own tastes and standards.

(Author's note: My sincere thanks to Bill Herdle and Bill Watkins for their helpful suggestions and criticisms as I developed this essay.)

Copyright Ken Rolston

Back to Analysis and Criticism

Back to Essays

GreenLine
About Spacer Home Spacer Authors Spacer Books Spacer Links
GreenLine

Brought to you by: RPGnet Logo

Questions or Comments? Contact the Webmaster.