Roundtable: Review Scores
By DeltaGamer Staff | November 30, 2011 | Features | 1 comment | Share
Reviews! They’re controversial, helpful, biased, bought and paid for, informative, or useless, depending on whom you ask! Due to lots of overly angry, knee-jerk reactions from certain games’ fanbases, as well as overly enthusiastic reviewers as of late, we thought it was high time to address this for our November roundtable. Sit back, grab some coffee or tea depending on which side of the pond you make your residence, and prepare for DeltaGamer’s extremely coherent thoughts on game reviews, critics, and their place in the industry.
Kyle: Recently, we’ve seen games like Skyrim garner a huge amount of “perfect” 10/10 scores around the Internet, and our Dustin here at DeltaGamer felt it was worth a 9.5/10. While I think it’s generally agreed-upon that no game will ever achieve absolute perfection, it’s interesting that a game with some obvious technical issues and flaws can get such a huge number of 100′s, 10/10s, and 5-star ratings. What are your thoughts on the place of “the perfect score” in game reviews?
Miodrag: First off, while we’re all in this business because we enjoy good games, reviews should differ from Internet comments. I don’t think they should resort to completely ignoring the shortcomings of a game just because the reviewer got hyped or is talking about his favorite game. I’d personally praise God Hand and defend it like a wolverine with rabies, but if I had to review it, I’d probably point out where it falls short.
Kyle: I generally agree, but for the sake of argument, what does the reader care about? “Am I going to have fun with this game?” is usually the extent of what an average gamer comes to read a review to find out. If God Hand is going to blow them away, influence their gaming tastes for years to come, and make them toast and eggs, then what weight should the reviewer give to any objective flaws in the game design or execution?
I think it’s fine to separate subjective love or hate for a game from the objective evaluation of its design.Kyle
Yannick: I have to say that my preconceptions in this regard have completely changed in the past few months, and have taken a full 360 degree turn with the release of Skyrim. Prior to that, I assumed that indie games would constantly have the scales weighed against them by the community of meta-critic hungry gamers and critics out there because they lack the funds to support AAA production values and extensive QA testing. Even a game like, say, Call of Duty begins to seem more iterative and uninteresting with every new release (this, by the way, is in reference to a general complaint of the series rather than my own opinion here), but the game has a polish and beauty to which most games, even other AAA titles, can only aspire. So anything lower than a 9/10 seems unconscionable unless you’re an overzealous Battlefield 3 fan. But then bug-riddled games like Skyrim come out and most reviews don’t even mention performance issues. Our own review of Dead Island wondered “how a game could be released with so many preventable issues,” but still gave the game an 8/10, meaning it was “excellent.” I gave the game a 2.5/5 in my own review for another site, but that was influenced more by my general qualms with the game’s story, pacing, and repetitive design rather than the buggy-ness of it.
So, long story short, I’m not so convinced what the standard between AAA games and indie titles is, and what makes it acceptable to pick on some games for a lack of polish and give others a pass. The cynic in me guesses that it’s probably a combination of aggressive PR tactics performed by companies with legitimate marketing budgets combined with an overcrowded market for online video game journalism that forces most sites to rely on access for “exclusive” content for their success or survival.
Miodrag: I’m going to use an example from my recent Heroes VI review. There, I pointed out that there are a ton of little issues that add up to the game, while there was one glaring issue of DRM that, if the reader chose to ignore (because they have a constant Internet connection), they could just ramp the score up by 1 point. I considered the lack of something like town screens trivial, but it was a deal breaker for some people I talked to. This just proves that not everyone will give the same weight to each issue. And that’s ok. I expect comments like that when talking to, you know, people, casually. While speaking to my friends and while reading something like Reddit. When I read a review, I actually expect at least some balance from the reviewer. You don’t have to be objective, but try to keep your cool at least somewhat. You aren’t writing just for the average gamer, you are also speaking to the developer and the publisher, to an extent. I know it’s “fun to bash games”, but in all honesty, if we’re throwing around max scores like they’re candy, the medium won’t strive to improve.
Yannick: I agree with Mio for the most part, but I think there is also a larger question of what kind of video game criticism we want to offer–by this I mean each of us as writers, as well as the larger editorial identity of a particular source. I think this larger question often gets lost in the frantic pace of game reviewing, and it’s another way that companies exercise a great deal of power given their ability to offer teasing bits of information game journalists all clamor for the be “ahead of the story.”
Whenever I set out to review a game, I’m reminded of a comment I always received on English papers in college: you’re trying to do too much. It’s impossible to sum up a book in five to ten pages, just as it’s impossible to describe the entirety of an experience I had with a video game in 1,000 words or so. I take issue with scoring or grading games in general because it asks to do something that can often be counter-intuitive and simplistic. The only times that high scores actually help me make a decision to buy a game is when a small or obscure title that hasn’t received enough attention suddenly starts receiving rave reviews. And the only recent example I can think of this happening is with Bastion. I can think of many more recent examples of grade inflation that gives decent games, or downright terrible games, a misguidedly positive reception.
Miodrag: I view a max score as something that cannot get any better with the current technology we have at our disposal and possibly even setting new standards. It doesn’t have to innovate, but it if it refines the genre to an extent that 
anything after that seems… worse, then I’d possibly give it a perfect score. However, this is an ideal standard and I am perfectly fine with reviewers giving perfect scores to something that I would consider “near perfect” (because, well, I doubt we’ll ever see a perfect game). But when a game like Skyrim has issues that detract from the gameplay, like the bugs, the awful NPC voice acting and behavior, the unengaging melee combat, the abysmal UI and the utterly broken crafting system, and you still give it a max score, then it’s detrimental to the whole industry. Why? Well, Oblivion and Fallout 3, both Bethesda games, had a ton of perfect scores (although not as much as Skyrim). The issues from those games carried on into Skyrim. They didn’t fix the wonky animation, they didn’t improve the NPCs, they didn’t make the whole game more stable. And why should they? More effort and they still know all publications will rave about their game to no end if they focus on other things.
Yannick: I mean, let’s be perfectly honest. When has a game ever had truly flawless execution? I feel like people who love to play games have come to accept games for what they are, warts and all. And a lot of the time that involves silly bugs–falling out of a level, people bumping into you, objects disappearing. It’s harder sometimes to determine when exactly these become overwhelming and make someone dislike the entire game. The only game I can remember playing where I don’t recall noticing a single error was Bastion, to bring it up again.
Kyle: Maybe then it’s okay to absolutely love a game, acknowledge its flaws, and spend hundreds of hours in its embraces anyway. Let me give an example: Borderlands. One of the worst UIs in recent memory, horrible technical implementation, boring quest descriptions, awful storyline–and yet, I probably played it for fifty or sixty hours. I think it’s fine to separate subjective love or hate for a game from the objective evaluation of its design.
Yannick: Well I think we have to separate out the quality of a game’s design from the quality of its technical performance. It sounds like what impresses you both about games like Borderlands or Skyrim is that, despite any technical flaws, there is something creatively compelling about these games that still draws you in. Like Mio said before, many of my favorite games–Fallout, Planescape: Torment, System Shock 2–are riddled with technical problems. Many of these, such as backwards compatibility issues, are probably not even the fault of any designer or programmer involved in the game’s production. But these games still seemed to have a real character and heart that other games never offered me.
Catriona: Well, nothing can be perfect to everyone can it? However…what would be the point in having a limit to a rating system if nothing could ever achieve the top rating? It’s a strange thought, but is the limit of a rating system only there to cast a comparative ‘top bar’ which is never meant to be given to anything or is it there to be achieved? I’m making myself dizzy here!
Yannick: Think of Kirk Hamilton’s great essay in Kotaku where he compared trying to play a new game to buying a new book. Some things have to be smoothed over after a release.
Miodrag: That’s a good argument, Cat, but another problem with a game achieving a top rating is that if a different game surpasses it, you’re pretty much stuck with them needing to have the same score. I don’t think either solution, attainable and unattainable limit are ideal.
Film Reviews Vs. Game Reviews
Kyle: It’s intriguing to compare review scores of popular games with high-budget films. Though game reviewers tend to avoid panning AAA titles, film critics will absolutely slam movies they see as poorly constructed or executed, regardless of who’s behind it or what kind of public response and ticket sales the film achieves. What do you think causes this disparity?
Miodrag: I think the general clingy nature of gamers has some weigh in this. Most people I know, myself included, don’t really care for film reviews. I do not think I have ever read a film review beyond user reviews on imdb, and mostly for movies I have already seen. I think just as you have movie-goers who don’t care about reviews, you have gamers who also don’t bother reading them, but a great amount of them still care how well their game fares. I don’t know why this is, I think it’s a combination of things. Game, platform, competitiveness. There are arguments within other mediums as well. Is Harry Potter better than Lord of the Rings? Is Band A better than Band X? I haven’t encountered such arguments with movies, but I assume they exist. Games just go one step beyond because there have always been platform wars and because games demand such a monetary and time investment, that you’ll either have time for Battlefield or Call of Duty, but possibly not both. Imagine realizing you made the wrong choice after you’ve invested a great amount of time? I’d sure as hell lie and say “No! I’m having a good time! Really! *sob*”
Kyle: You bring up a great point: the time and money thing is absolutely huge. It also may affect game critics, since they often rely on publishers or developers to acquire copies of games without wasting a bunch of money per year on games just for review. Whereas a blossoming film critic can plop down $9 or $10 to see a movie just as the rest of us would, a game reviewer can be more dependent on a healthy relationship with a publisher or PR company, which sadly may subtly affect the way he approaches the review. Pan a movie, and nothing of value is lost–pan a game, and your site could get blacklisted if a nasty PR agent decides to act shady.
Games just go one step beyond because there have always been platform wars and because games demand such a monetary and time investment, that you’ll either have time for Battlefield or Call of Duty, but possibly not both.Miodrag
Yannick: I think there’s also different expectations for production studios and distributors that gives film criticism a different position relative to the film industry, for better or for worse. I remember Michael Bay came to speak at my school my Freshman year at Wesleyan (his alma mater), and someone asked him how he dealt with negative reviews and cynical critics that didn’t see much value in films like Transformers or The Rock and, perhaps more importantly, how the studios and producers that bankrolled him dealt with this negative reception of his movies. With his usual Michael Bay-ness, he responded: “It’s called making seven billion dollars for them.” People like the Weinsteins don’t have to care what critics say if everybody and their mother is going to see a movie anyways. With the video game industry there is an entirely different culture, and entirely different economy. I don’t know the full answer of how much influence game reviews have on sales figures, but I know that developers jockey a good deal for high scores on metacritic. Why else would the employees at Telltale Games or Bioware try to sneak in positive user reviews? I think video game journalism as a whole is still learning how to flex its muscles as a fully autonomous unit separate from the rest of the video game industry. Given the general state of contemporary journalism these days, it will be interesting to see how it develops.
Miodrag: Also, watching movies and playing a game is quite different. Even if the game takes maybe three hours, if it’s a bit on the demanding site, you might have to take breaks. From the reviews I’ve done for DeltaGamer and Press X or Die, I know that not all developers expect you to finish their game in one go, nor was it designed with that expectation. Imagine me finishing a frustrating platformer in one session and that fact affecting my opinion? Whereas if I had played it on and off, I wouldn’t have minded.
Kyle: It could also be that games that merit reviews by a major gaming site are simply much more likely to be quality compared to big-name movies. Whereas the dregs of gaming is generally seen to be things like cheap iPhone apps or crummy XBLA zombie games that wouldn’t even merit a mention, in film, weak ideas are often hugely commercially viable and get a big budget and big actors on board for it. If you’re developing a hundred-million dollar game, it’s very unlikely that it’s total crap.
Yannick: But if you try to acknowledge that there are some crappy parts suddenly people are telling you to go die in a hole somewhere. When I look at it from that perspective, I become very, very pessimistic. Game companies have achieved this incredible thing where their products are so integral to the social identity of their fans as “gamers” that you can start entire flame wars on the Internet just by jabbing people the wrong way. And these same “gamers” don’t even seem to realize that! It serves everybody–game companies because of the ferocious loyalty of their fans, critics and journalists because it gets them more attention and more hits for their website, and the rabid fans themselves because they acquire a sense of community and get to play the games they love so very much. But in the process I think it diminishes the entire level of discourse around video games to nerds bickering about who sucks at what game.
Catriona: Yeah, I’d say Miodrag has it spot-on, I think (and I know this is true of myself) that each game you buy is a gamble, over here it’s generally a £40 bet that you’ll like a game. Based on this, it’s not surprising that people will rely heavily on game reviews and feel the need to defend their own opinion of it, possibly for the benefit of other gamers who might not buy a game based on a single review. I think I’ve tried to read film reviews in newspapers and the like, but I’m more satisfied with user reviews on IMDB, even though they are probably biased. The complete slating you see with games just doesn’t seem to exist with films, and either way I’m not too bothered losing £5 to a film I didn’t really enjoy.
Marking Down a Failure to Innovate
Kyle: An issue a ran into with one of my recent reviews, Modern Warfare 3: do games that fail to innovate in certain ways, or perhaps rely on a well-worn formula, deserve to be dinged? How much? I ended up giving MW3’s singleplayer an 8.5/10, because it did innovate in level design, pacing, and flow, all of which were designed better than most of its predecessors. But do games like FIFA or Madden or even Uncharted call for lower scores with their perceived stagnation? If Hollywood released the same movie with some minor tweaks and changes year after year, critics would bomb it to the depths of Metacritic hell, yet the same doesn’t seem to hold true for games.
Miodrag: The fact that gameplay exists in games makes it hard to compare with movies like that. I can’t really think of a proper analogy, but I guess it would be closest to filmmakers not having to re-invent the camera every time they make a movie? You wouldn’t pan a game for having the same gameplay as its predecessor, but you would pan it if it had the same levels, plot and every other element. Although, I assume you would also score it lower if it didn’t fix previous issues as well. If there was this bug that made everyone fall through the floor, but other than that, the gameplay was flawless, and you didn’t fix this bug in the few next installments, I think people would start to get annoyed.
Time and time again we see the same formula for drama and film, especially since the rise of the ‘twist ending’ and terrible ‘romcoms’, yet for some reason people keep paying to see them and still find some satisfaction in them. I think this may be what happens with games like FIFA and Call of Duty.Catriona
Yannick: Well, again, think of the reception to Skyrim. Beautiful game in so many ways. But people also acknowledged that, performance-wise, it wasn’t really a step up from previous Elder Scrolls entries.
Catriona: Time and time again we see the same formula for drama and film, especially since the rise of the ‘twist ending’ and terrible ‘romcoms’, yet for some reason people keep paying to see them and still find some satisfaction in them. I think this may be what happens with games like FIFA and Call of Duty, people have liked this formula and will continue to want to buy the latest version even though changes usually aren’t great in number, and sometimes even really annoy people. I won’t scorn the fans of these games too much though, because even though I don’t particularly like them, it could be argued that RPGs follow a similar pattern (your character is ‘different’ from everyone else and must go on a mad quest to save or destroy something or other) and I still lap them up – maybe repition ain’t all that bad.
Saying that, however, I did have a bit of difficulty when reviewing Assassin’s Creed: Revelations because of the fact that it was part of a series of games which are based on the same sort of formula. I decided to take into account that it was part of a series and therefore should maybe have worked on the story more, though there were gameplay improvements, but also that as a stand-alone game it would have been amazing.
Yannick: I think game developers deserve some praise here, though the full success of many designers’ achievements is often shrouded by how awkwardly the same company attempts to juggle the transformation of a single, potentially self-contained video game into a series and then, most likely, into a franchise. I don’t think mind-blowing innovation is necessary in every sequel. For the sake of continuity, you may not even want mind-blowing innovation. It’s an incredible sign of the entire medium’s maturation that individual games, corresponding series, even companies and design teams, have significant motifs that carry through their work. The latest Call of Duty game ended the Modern Warfare story arc, as did the latest Gears of War. Both of these series had brilliant design innovations that they continued to fine-tune as the series went on.
So, what I’m trying to say is that I don’t think reinventing the wheel is necessary every time around. I do take issue, however, with a series that fumbles between maintaining continuity for the sake a story world and simply trying to repeat itself. That was one of my larger issues with Uncharted 3 that made me prefer the second game in the series. I still enjoyed them both a good deal, despite whatever the Internet trolls may think…
Best and Worst of Games
Kyle: Finally, a bit of fun: what is the worst game you’ve ever played, and what would you have reviewed it if you were reviewing games at the time? Same for the best game you’ve ever played.
Miodrag: I am stumped as to what the worst game I have ever played was. I was going to say Fallout 3, but I actually only raged really hard at the ending and enjoyed the rest of the game, so it doesn’t count. Possibly Borderlands because it was just so mind-numbingly boring, even when I played it with two good mates. I think I would have given it a 3.5 according to DeltaGamer’s scoring policy. As for best game, God Hand without a doubt, and I would give it a 9.5 according to DeltaGamer’s scoring policy.
Kyle: I want to hack into Liberty Prime’s guidance systems and have him crush you with democracy for mentioning Fallout 3 in such a negative light. Seriously good game, though New Vegas is that much better. As for my worst game, it would probably have to be something from the N64 era, when developers were adjusting to the widespread use of polygons and so much trash came out. Superman 64 comes to mind. I also thought No More Heroes was pretty terrible, though that may have been the point. Superman 64: probably 2/10. Best game, Half-Life 1, definite 10/10.
I doubt I would ever give a game a 10/10 at the time of release because part of what really impresses me about my favorite games is how much they stay with me for months and years after I’ve played them.Yannick
Yannick: I think I tend to be harder on games oftentimes when I have more invested in them as a fan and a player, a tension I’m still learning to deal with as a video game journalist to be honest. In retrospect, for example, I think my review of Rage was too hard on certain flaws in the game, I still consider posting an updated addendum to apologize for some miscalculations I made in that one. I think, at this point, I’ve gone full circle from being too kind to games, to being far too hard on certain games, to finally having enough under my belt to feel generally more informed and even handed. The first game I ever reviewed in a public setting was Pirates of the Black Cove which, when I look back at it, was a terrible game in every sense of the word. Just terrible. I gave it a 3 out of 5. Now I think it would get a 1 or possibly a 0. My favorite game by far is Planescape: Torment, though I’m uncertain what I would have given it when I first played it. I doubt I would ever give a game a 10/10 at the time of release because part of what really impresses me about my favorite games is how much they stay with me for months and years after I’ve played them. I’ve only played Planescape once the whole way through and I still find myself thinking about it almost every day.
Miodrag: It would be an interesting system to only allow 10/10 if the game passes the test of time.
Yannick: Well I suppose that’s the same question as in any other form of criticism. Albums or films can get rave reviews, or very poor reviews, at the time of their release. But their legacy is mitigated through an entirely different process of awards and retrospective articles. Also some of these games start to stand out in terms of their legacy when we see how they have influenced future games. I only played System Shock 2 after playing Bioshock, for example.
well Skyrim was far from perfect. Far to many bugs (bethesda game of course but still) The UI is terrible specially on pc voice actors are used more then one time etc.