Reviewing Reviews

Chris an I had talked about dong a podcast concerning the concept of review scores. While movie scores are for the most part standardized (at least when it comes to the more respected reviewers), although it is far from perfect. It is still better than the video game review scores. They are completely out of whack that still needs a ton of development. We may in fact do a roundtable podcast eventually about the subject but there are a couple of things bothering me about what I see in print and on the web.

Movie reviews tend to follow a general mold in where you either get a rating from 0 (or 1) to 4 (or 5). My reviews are based on this system because I think it’s the most appropriate. It’s also very easy to defend because I find the larger the scale, the harder it is to justify how one item gets a slightly better mark than another. On the five-point scale, an average item gets three stars. Below average gets a 2 while something awful gets 1. On the better end of that, above average gets 4 and an instant classic gets 5. There are some other film review models employed by more respected reviews like a simplistic thumbs up or down, a smaller four-point scale, or a letter grade resembling something you’d see on a high school report card.

One thing about film reviews that rarely are they influenced by the film industry itself. Some studios embargo the film until the release date to prevent bad films but rarely do studios attempt to bribe reviewers for good reviews (or in the very least are not successful in their attempts). In fact, more often than not, most films that have their DVD boxes adorned with quotes from reviewers tend to be of poorer quality. The biggest example that comes to my mind of a recent DVD release that is guilty of this is the film Duplicity. The quote on the front cover is almost as big than the title of the movie. I remember more than one customer asking what I thought about the “new movie” Sexy Caper with Julia Roberts and Clive Owen. Reviews for the most part are honest but the film industry has done a pretty damn good job at using even the slightest praise to their advantage.

Sites like Rotten Tomatoes and Metacritic are convenient ways to seek a consensus opinion about a given movie. The problem with that the scores given can be a bit misleading. Rotten Tomatoes gives a score out of 100 based on the percentage of positive reviews a movie gets (called the Tomatometer. This percentage though does not take into account the degree of good or bad it is. The site does offer an average rating but it is written in smaller print and the movies that are currently in focus on the main page only have their Tomatometer score shown. Metacritic does a better job since it has a scoring system that uses an algorithm that weighs the both the review score and the reviewer into the equation. The difference is that while a film on Rotten Tomatoes may receive an 80 because it received 80% positive reviews, it may receive a score of 50 on Metacritic because while some less important reviewers may really like the film, more established reviewers. My problem with this system is that this system increases the weight of some reviewers that I don’t care for. Why would I want to take the advice of someone I don’t like and consider it just as important as someone who’s opinion I respect and generally agree with.  What would be great would be a system that allowed you to remove reviewers you don’t like and only keep the reviewers you do. If anybody comes up with that system, I want half the profits.

Video games are a way different story. It’s pretty bad actually. First off, there really is no general consensus when it comes to delivering a review score. While many employ the same styles that most film reviewers use, there are some really piss-poor ones that many well-revered performers use. The worst one on the top of my mind is the 100-point scale which seems absurd to me. How can you differentiate between an 81 or an 82? It seems kind of nuts. Also, the concept that an average game can receive a score higher than the medium score on the scale is nuts. On a scale of 1-10, an average game should be 5, not something like a 6 or a 7. Unless a game is ridculously bad, many places will seldom give a score on the lower end of the scale.

Another is giving scores to various aspects of the game (like graphics and gameplay) and either using those scores with a rubric to compile a final score or using them only as a guide to a final score. The first iteration of this website back in the Modblog days used a rubric based scoring system for games. The problem I had with this the more I thought about it is that even though you would only give a certain percentage of a game to an aspect like sound or graphics, they don’t make the game; the overall experience does. Since technology allows for graphical advances much quicker than ever, it’s harder than ever to evaluate something solely by how good the game looks. If I were to devote a chunk of a review to how good it looks, most Nintendo 64 games would be given a bad review just because the graphics look incredibly dated and blocky. Granted, the scores could be adjusted to reflect the graphical capabilities of the game from it’s original time of release but the idea of reviewing something is to promote (or warn against) a particular item. The idea of telling people that Starfox for the SNES is a great game so they will go out an play it. Giving an older title a good score in the graphics department would not make sense considering that many titles that produce a similar experience on the current generation of consoles look a million times better. In the end, it’s got to come down to the overall reaction to the game rather than give scores to every little part of the game. That’s what paragraphs are for.

Even though the industry has a problem of bad review systems. The situation isn’t remedied by the fact that many video game companies do a really good job at meddling in the review process. The industry is so cutthroat at the moment that companies are stuck at doing what the video game companies want them to do that it influences their editorial process. One of the largest examples of this was Kane & Lynch fiasco in which there were rumours that a reviewer was fired for giving a game a bad review when the publisher, Eidos, was advertising on the same site for the launch of the game. Other internet sites and TV shows make it difficult to tell whether previews (or even reviews) were actually written by the reviewer or by a PR person for the video game company.

In Hollywood, the competition is so cutthroat that any news that is available is published immediately. It’s hard to hide potential flaws or negativity concerning a project and embargoes are rare. The opposite is the same for the video game industry. Games are constantly revealed to journalists but they cannot talk about them until the developer or publisher says it’s ok. The idea of a exclusive premiere of just the announcement of a game is a problem because it removes all the honesty from the announcement. The journalist becomes a tool of the game maker. This shouldn’t be how it works: the journalist should be the critic. It is their job to deliver the news; good or bad.

When it comes down to it, the definition of a review is that it is someone’s opinion of something. Therefore, it’s best to seek out the opinion of a a few individuals that you particular enjoy and generally agree with rather than go to a site where several individuals may handle the reviewing duties on a revolving schedule. A word of warning about my reviews though; I will rarely review the very latest stuff with the exception of movies. I’ll never dirty my hands to get my hands on a game early to review so unless it’s a game like New Super Mario Bros. Wii which I know I’ll be getting the day it comes out, it’s not going to get the early review treatment. I’ve had companies ask me to review software but they all seemed more interested in me doing stuff like posting links to their site and me preaching how amazing their wares are on stuff like Twitter. I’m still not sure if reviewing software is what I want to do on the site but maybe in the future. I’m not against receiving an item for free for review as long as there are no strings attached. Then again, I love cheesy 80s hair metal so I might not exactly be the best judge of quality

–

No extra news for today other than some site news. You may have noticed a couple of missing posts. They’ll be up sometime this weekend. I had a problem uploading some of the stuff and they’ve disappeared (including the original rant for today which wasn’t very good…so sometimes computer problems are beneficial). I’m not having the same problem though with my laptop so I’ll move the stuff from one computer to another and put them up during the weekend but not all in one shot so that stuff won’t get lost in the huge drop of content this weekend. The roundtable podcast is on its way. Audacity has been giving me problems and I’m going to look for a better audio program. I used to have Acid Pro but I’ve lost the disc during one of my moves. You could hear some of the difficulty that I’ve been having just from the two podcasts that I was able to post (like clipping and sound spikes). I’ll be on the road this weekend although with plenty of time on my hands so expect any back log to be cleared up and I might even have an extra podcast up too.