It may have been a bit hard to tell but during this weeks podcast I put forth the following question as this weeks Talky Thingy discussion point – What is your opinion on MetaCritic?
Is it a good thing? Do you use it to guide your purchasing? Would you rather it didn’t exist? I am not a fan, a fact that was probably clear from my rant in the show though my reasons probably were not. First thing to say is that my problem is not specifically with MetaCritic – its more aggregated score sites in general.
My primary from problem is a complete lack of context that result from them taking only the score. I know I have written reviews for games that I have loved playing and made that clear in my word, but due to some major flaws that could be game-breaking for most other people I have had to give it a lower score. I know people who have had it the other way round they have absolutely hated the game personally but can find nothing technically so had to give a good score.
This is a very important aspect of the review in my opinion and taking that away is misrepresentation of the reviewers opinion as far as I’m concerned.
My Second Problem with it is MetaCritic specifically – The fact that not content with removing the context they further manipulate and alter the scores in a way that may not be the intention of the original reviewer. One good example are the poor guys out there that use the A-F scoring system, such as http://www.gamerevolution.com/ for you see MetaCritic have their own scale for what the various letter grades mean. For GameRevolution a B- is about an 80% for MetaCritic they have for some reason decided that it is 67% – that is a massive difference of opinion and intent right there.
In fact for GameRevolution it has actually resulted in them being on the receiving end of several emails complaints about their review scores, if you check out this page Mind Over Meta: Page 3 in their article on the same topic they actually talk about one of the complaints they received from a developer about the fact that ‘their’ low score was stopping the game breaking the 9.0 MetaCritic average. As they say, its their job to review the games using their standards not to fix MetaCritic’s interpretations of it.
What makes that complaint all the more incorrect is that, unknown to me until I started looking into this article, critic scores are weighted when they are taken into the overall average. This means that if we were to add 8BitAlliance to MetaCritic (its not going to happen and were not going to be using scores anyway) our scores will be given a lower priority than IGN etc. Now while I can understand why they have done this and in a way it makes sense, it once again means that the meta-average is not a true representation of the reviews given. It is tainted by MetaCritics own views of Who is Who in the games reviewing business.
The final and worst part of the whole thing for me though is the fact that the aggregated score can now effect someone’s livelihood. Back in March Fallout: New Vegas developers were denied a bonus as there MetaCritic average for the game was not 85 or higher, poor guys only managed an 84.
Now I know I am probably jumping to a conclusion here, but its kinda telling that the day before that missed bonus was announced to the public Obsidian announced staff cuts and the cancellation of a future project. You are left with the very clear opinion that had the bonus been paid then perhaps these cuts could have been avoided – however that can not be confirmed. If you want to read that specific story check out Metacritic Scores are Ruining Videogames, Just Ask Obsidian.
Obsidian is not the only one held to this standard either, I have heard of other games companies on similar contracts & it is apparently quite a common thing amongst games PR as well. Personally I would not sign such a contract if I could avoid it but I can understand that bills have to be paid and it has become impossible to get a contract without that then your hands are tied.
It’s just a bit of a bitch that your bonuses are in the hands of an aggregator that could screw you over by simply giving the wrong site a low rating or misinterpreting their B- as 67 not 80. That’s all without even bringing up the fact that for a Movie, Book or Music to be considered to have ‘Universal Accliam’ they only need 81, but games apparently have to have 90??
However don’t let my opinions sway you, we want your opinions on the subject. Hell if you think I’m wrong with any of the above then let me know as well.