Th.o.R: Thoughts on Reviewing #1 PDF Print E-mail
Written by Lutzifer   
Mittwoch, 05 Dezember 2007
ImageI had planned to do an article on the veracity of the reviewing-process later this month, but since the gamespot-incident made it newsworthy, i have moved it up the schedule. As Mike Krahulik pointed out quite honestly, game reviews are fundamentally flawed. What once has been a pivotal point of social interaction has now become a professional endeavour, turning a friend's recommendation into a business-model. With gaming magazines that has been painfully clear for ages now and i found refuge in the internet for some years to help me make informed guesses as to what is playworthy or not. But the internet-reviewing sites have evolved from enthusiast-driven sites to big businesses also, so we are at square one again. In the following article i will discuss some of the underlying issues with reviewing and try to outline how to make the best off a flawed system.

When gamers go out and hunt down a review of a game, we are usually interested in getting a good description of what to expect and receiving a frame of comparison. While even in the most biased and opinionated reviews the reviewers tend to get the first part right, it's the model used for comparision that is ultimately flawed for several reasons:
  • games are works of art and therefore are subject to individual likes and dislikes. Assigning arbitrary numbers for feats like sound, gameplay, story and art only corroborate this subjectiveness into a meaningless and pseudo-objective rating
  • the numerical rating system itself is the best indicator of its flaws. Most reviewers use a 1-10 out of 10 rating scale, while the actual points given vary only between 7 to 10 for the majority of games. This non-linearity in variation should be enough to discredit the professional judgement of gaming-reviewers / the reviewing process per se
  • while most people play games recreationally and with a friendly disposition towards the game, game-reviewers are forced to have fun within a publishing time-frame. And what could be more fun as rushing through content to get to a deadline (see the link above to the PA article for a well worded opinion on this)? Right, nothing short of replaying the halo library level over and over again... [for those who havent played halo, the library level is the absolute anti-thesis to fun in the history of gaming]
  • professional game-reviews also often fail to assign the right reviewers to the games. For example: If one guy only ever reviews one genre he'll be more inclined to feel less joy in playing games because of fatigue symptoms (been-there, done that judgements) and an added intolerance for minor flaws.

But in an ever growing market with more and more products to chose from reviews become more vital for consumers to rely on. Especially since nobody can afford to buy and try everything, so asking a friend for a recommendation for a good game only gets you so far. So what can you do to make a better informed guess as to what is a good game to buy?
A good benchmark criterion is to only look at meta-review pages (like rottentomatoes or metacritics) and only use really high and really low scores as a guideline, because that usually means that the critics are unified in their opinion about the quality of the product. If you are interested in something that gets mixed reviews, i suggest to read at least two good and bad reviews and see how they resonate with your gaming-experience and your taste. After that you could also always try a demo if available. In my experience i ve always had very different experiences with demos from the finished game, so i wouldnt recommend it (if you can rent the full game or play it at a friend's do that, after you have decided that it might be worth it).
As a rule of thumb i'd also suggest to stay away from review-sites that have an over-abundance of advertisement in their reviews.

Suggestions for reviewers:

The state of computer- and console-hardware has reached a level of detail and speed that renders the use of GFX- and sound-quality as a factor for a review unnecessary. If a game excels in those areas above and beyond other contemporary offerings, you still can verbalise that in your review. Take for example the great Stranglehold soundtrack that too few reviewers looked into and only gave it a number afterwards, reducing the worth of that information to a mere digit.

Rating & Referencing: When comparing things the human brain usually falls back to an internal 5-step scale in most cases, so it is unnecessary to have pseudo-scientific scales to spice up your ratings. Especially if the rating-system gets confounded by good to great scores for sound and gfx, that dont say anything about gameplay and story (leading to overall-scores between only 7-10). If you want linearity go with final judgements like really bad, bad, mediocre, good & great game. You could also give out awards to outstanding games to make sure your readers understand that you like the game alot. Try to compare the features of a game to similar games and how that effects the gameplay and experience to build up a good framework of reference to other games and experiences that gamers can relate to. If you absolutely have to use a numerical rating system for games, try to re-evaluate genre-classics every year to make comparisions easier between old and new games, to show how they stand the test of time and if it was better to buy an old classic than a new offering.

Story and gameplay: This is THE utmost problematic field in reviews, because it's the most subjective. My suggestions is to embrace subjectiveness to a point where it becomes more objective. That involves some amount of professional introspection and self-monitoring. State your opinion and make sure to verbalise that it is indeed an opinion, so people can relate to their own tastes more easily. Use examples of what you strongly dislike and don't generalize like many reviewers do, if it is indeed only your opinion. And if you feel that a game is overall bad only for subjective reasons, either trash your review and let someone else review the game that actually liked it OR do a co-review with someone that likes it to give the reader a counterbalanced review. For example, have a look at the Kane & Lynch review that allegedly got the guy fired. You 'll notice that he over-generalizes his subjective dislike into statements like "it s an ugly ugly game", while from what i saw, the game itself is pretty decent. I d give the game an above average review for some of the fun-levels like the disco alone, while not judging the story (which will be an article in itself about depiction of violence and grim-story-lines in games).

See also: T.h.o.R. #2: the pain of party-games

::: Feel free to leave a comment below ;) :::
Add NewSearch
Write comment
Security Image
Please input the anti-spam code that you can read in the image.
Last Updated ( Dienstag, 08 Januar 2008 )
< Prev


Lo Wang:
That's no ordinary rabbit.


Biofeedback and / or human-machine interaction via brainwaves (e.g. EEG or EOG) has been used in the scientific community and in military R&D for quite some time and has yielded a broad  spectrum of different results. Still it struck me as odd, when i saw techreport reporting on a new gadget for gamers: a neural headband controller. I really like innovation in the field of input-controllers, but this took me by surprise. And surprise  (or rather the startle reaction) mapped onto a controller might be just the thing to come in handy when in close-combat battles in an FPS and is something i often wished for already.
I highly doubt that the controller will be something that sells well or even work well enough to be usefull as a single controller. As an addon for pro-gamers this might give a certain edge in the competition though, as the startle response is much faster than other motor reactions in gaming. The more one practices games though, the less startled one gets, so this might also be a short-lived gain in superiority only. I d love to get my hands on one of those to test it myself. The last thing i was so excited about was 3D-Shutter glasses and they turned out to be a nuisance rather than aiding the experience. I really hope for OCZ to succeed with this so they can perfect the design or come up with other innovative ideas to spruce up the gaming world.