At the Office

Home

On the Couch

At the Office

In the Library

Through the Mirror

Mailbox

Neighborhood

Random Thoughts on Games Journalism

There are certain things you can count on from games journalists, among them Top X lists and navel-gazing. The former is an easy tool for filling up space and generating conversation, the latter an inevitable side effect of taking what you do seriously. While that may seem like a contradiction at first, I think it speaks to the diversity of practice in games journalism today. A couple of recent reflective articles on myths of game journalism and reviews have inspired me to put down a few of my own random thoughts on these practices.

I should start by saying that as I am not a games journalist, I can't really speak to the difficulties involved in doing it. Since I constantly rag on consumers of games for presuming that they understand game development, I am very much at risk here of hypocrisy in pointing out issues with journalism without a first-hand or in-depth knowledge of it. What I do know is that it's harder than it looks; while the image most people may have is that it's great to get paid to play games all day, often before the general public gets a chance to, there's a lot of hard work involved, writing and re-writing thousands of words on tight deadlines, having to play stinker games and struggling to get some to work at all, traveling hither and yon to visit developers and events, getting pushed off or ignored by developers and publishers when trying to put together a story, facing the same boilerplate on retread projects from the same PR hacks day after day and trying to find some way to make it interesting to your audience, and on and on. Plus, it doesn't pay all that well.

I should also point out that while I'm going to be critical, I don't mean to tar everyone with the same brush. There are better and worse journalists, just as there are better and worse publishers, developers, architects, politicians, and street cleaners. I have great respect for those who are good at their craft and take it seriously, but at the same time there are some very real issues with some of the practices that are common enough to merit comment.

So, caveats aside, let's start with reviews. Games journalism is not all about reviews, but it's one of its primary functions. Reviews are important to developers and publishers, not just in the GameRankings sense. Reviews can influence buying decisions, and ultimately whether or not people buy our games determines whether studios flourish or fold, whether publishers stay in business or not. Now, reviews aren't the be-all and end-all of sales; licenses can be important, so is timing, so is word of mouth. There are games that get great reviews (Psychonauts, Beyond Good and Evil) and don't sell; there are games that get poor reviews (Enter the Matrix, Postal) that do sell. As developers and publishers, though, we're particularly sensitive to reviews because they're judgments on the quality of our work. So, issues with reviews tend to be something that we get touchy about.

Let's start with an easy one: time. Reviewers are under deadlines, and sometimes games are massive time-sinks, so the conditions aren't ideal, but when reviewers don't spend enough time with a game, it's hard to do a comprehensive review. This can produce simple issues, like a game that's good for the first third or half and falls off in the end can get a better review than it deserves. Or, going the other direction, a game that takes time to get going may get a worse review. One of the other things that falls out of this is that reviewers will sometimes play games on the easiest difficulty in order to get through them faster and see all of the content. This can produce a number of problems, like reviews that claim that the AI is faulty because it's handicapped on the easiest difficulty, or claims that a game is too short because you play through a game much faster when everything is easy.

This also means that replayability - which is an aspect of value that is important for some consumers - can get short shrift. If the game is even better on repeated playthroughs, the reviewer may never see that. It can also be a problem in exploration-oriented games, particularly if the reviewer powers through the main storyline without taking the time to explore; they may never see side content. This can be a particularly difficult problem for RPG's because they tend to be very long (30-100+ hours instead of the 12-20 for most action games or FPS campaigns). MMORPG's are even worse because there you're talking about hundreds of hours of content, and no reviewer has time to see all of it before writing a review, much less play multiple characters from start to level cap to effectively judge how compelling the character system is. If your reviewer just happens to pick a class that is less polished (or over-powered) at release, the review can suffer because of it.

There isn't an easy solution here, unfortunately. There are a lot of games being made every year, and the sheer logistics of having every game get a full playthrough on multiple difficulty levels is next to impossible. Even if it were possible, the economics would be a problem; in order to get more manpower for the same amount of money, you'd have to pay even less for reviews than is currently done, which means you get more reviews from unprofessional reviewers.

And that brings us to our second point: lack of professionalism. While failing to spend adequate time to thoroughly review a game could be lumped under this category, what I'm specifically trying to get at is that reviewers sometimes fail do basic things you would expect from a reviewer. For example, there was a famous episode where CGW pulled a review of NWN 2 because the writer was so antagonistic towards the D&D system in CRPG's that it biased the review. On the one hand, reviewers have to write about their own experience, because it's what they know - they have no way of knowing what someone else's experience of the game would be. On the other hand, reviewers have to be somewhat objective, to judge the games on their own merits. You can argue that these events are fairly rare, but games aimed at kids often get reviewed down for it. You can't expect a 12-year old to write reviews for a major gaming web-site, and you wouldn't expect a thirtysomething reviewer to know exactly what kids will and won't dig, but is it too much to ask to expect reviewers to judge kids games as kids games rather than games for adults? Of course they're going to be less challenging, less deep. Of course they're going to be relatively simple and straightforward. Of course they're going to be somewhat repetitive. That's what kids want. That's what they enjoy. In spite of this, kids games are often rated at 10% or more lower than games for adults.

A subtler version of this is what I call the "carryover" effect. This applies mostly to franchise games, sequels and whatnot, but it can also manifest in attitudes towards games from a specific developer or publisher. The Dungeon Siege franchise is a good example of this. The first game came out, people reviewed it highly; it was the first action-RPG of decent quality that had come out since Diablo II, the world was original, the graphics were good. In fact, you could argue that the reviews were a little too positive. Consumer reaction was decidedly more mixed, with a lot of complaints about linearity, lack of variety, issues with the co-op mode, an AI that did too many things for you by default, and they let everyone know about it, in letters, forums, discussions. In retrospect, I'm sure some of the reviewers felt that they had over-rated the game. So, when Dungeon Siege II came out, there was already a bias in place against it. Even though DSII is a superior game in most ways, the reviews for it were more tempered. Rather than judging the game on its own merits, the carryover efect from the first game dragged down the scores on the sequel.

The reverse of this is when you have a game that is successful and the sequels get a boost from the carryover positive impression of the previous versions. My favorite example of this is the Elder Scrolls series. Morrowind was a good game, but by the time the reviews were due, the reviewers hadn't had a chance to see all of it (see notes above on time and RPG's). So, they continued to play it after the reviews were published, found a lot more in the game, and had a positive carryover. When Oblivion came out, who do you think reviewed it? Rather than being relegated to some random staff member, I guarantee you that the fans of Morrowind fought to get the review, in part so they could play the sequel of the game they liked so much. So, on top of the carryover, there was an inherent positive bias going in. Now, if Oblivion had sucked, the reviews would have been merciless - no one spews vitriol like a fanboy scorned - but it was a good game, so the reviews were absolutely glowing.

What was missed in the process is that the game is exceedingly difficult to learn; the reviewers who tackled this game already knew how to play it, because they had played Morrowind (and possibly the older Elder Scrolls games), so the learning curve was natural to them. It also got a pass on having bugs, both because they wouldn't all show up in the first 20 hours of play and because the players (reviewers) were willing to work around them to play the game. Oblivion was an A game, don't get me wrong, but it didn't deserve the absolute adoration heaped on it; aside from the leveling/balance issues, the learning curve, and the bugs, there was a tremendous repetitiveness of content and a lot of punishing game dynamics. Will this carryover affect the reviews of the next sequel? Time will tell. Certain franchises have definitely gotten a pass on lack of innovation, when the press yammers on endlessly about how they want more of it and slam new franchieses for not having it.

But probably the biggest issue in terms of lack of professionalism is when the reviewers flat-out get it wrong. I'm not talking about opinions, I'm talking about facts. For example, one of the reviews of Titan Quest complained that there was no teleport function, that it was a huge hassle to have to run back to town to sell everything. There was, in fact, a teleport function. Not only was there a big button in the middle of the UI with rollover help that described it, but there were portals that lit up when you went past them; it was covered in the tutorials and the manual. The outlet that published the review eventually changed the text of the review, but they didn't pull it and re-review it; they didn't even change the score. Now, you would think that something as integral to an action-RPG as going back to town to sell loot would have an impact on the experience; after all, you do it all the time, but in spite of the fact that the reviewer experienced it as much more tedious than it actually needed to be by his own fault, there were no substantive changes to his published judgment of the game.

For people on the development side, this is absolutely maddening. It's one thing to be criticized for the faults that you have; it's entirely another to be criticized for ones you don't. Another example was on a different project where the outlet had asked us for a pre-gold build to get an early review out; the reviewer ran into a bug, which we were able to get him around. We tracked it down in our database and verified that it had been a known issue and fixed between when we shipped out the early review build and when the game went gold, and we let the reviewer know this. In spite of this, they published the review with a note that explicitly called out that bug and explained that they would have given the game a better score if it hadn't been there. You can argue that they couldn't just trust us on this, after all, it's in our best interest to spin things positively to get a better review, but they had asked for the pre-gold build; they knew it might have some bugs that the final version wouldn't.

Again, unfortunately, there are no easy solutions here. If games already take too much time to review, you can't exactly have multiple people review each one to back-check against biases and mistakes. For the editors, I imagine it's an endless headache because they not only have to get the reviews done, they've got to manage getting the writing up to spec, handle the business side of the publication, and deal with all of the relationships, from their employees and freelancers to parent companies and advertisers to the developers and publishers. I don't envy them their jobs. I do wish it were easier to bring up the level of professionalism, but that's difficult and expensive and may simply take time.

It can be said that those in game development have a love/hate relationship with reviews: we love the positive ones and hate the negative ones. Beyond that simplistic reduction, though, there is a symbiotic relationship between an industry and the press that covers it, and hopefully we can both be objective enough to recognize the benefits and the flaws openly and honestly. Enough on reviews, though.

The other thought rattling around in my head is the issue of coverage. Now, coverage can be just as important - if not moreso - for a project as the reviews. Coverage builds awareness, it lets people know that the game exists, and it builds buzz, it gets people excited about the game. You can get phenomenal reviews, but if no one knows about the game or is excited about it in the first week of release, you can launch like a lead balloon. One of the reasons why it's so difficult to launch new IP instead of just using licenses and producing sequels is that you're starting from zero. For these projects, coverage is essential.

Now, here's a conundrum. Games magazines put games on their cover to get people to buy the magazine. But, the games that need the coverage the most are the ones that no one knows about. Short of buying the cover (which has, strangely, become a common practice, at least in the form of the paste-overs for subscribers), and I think we would all accept as being unethical, how are you supposed to get a new game on the cover of a major magazine? The answer, apparently, is that you need a tie-in. The fact that it's a new IP didn't stop anyone from featuring Spore, for example. And Supreme Commander got multiple covers, in part because Gas Powered Games has an established audience and in part because the Total Annihilation fans could be counted on to be interested in it. So, at the same time that games magazines are at the forefront of the crowd calling for more innovation and fewer sequels, they're actually locked into the same dynamic: people won't buy things they don't know about. The difference, here, is that the magazines actually have the ability to do something about it. The magazine itself is the brand, and if there were a magazine that developed a reputation for putting the best games that no one knows about on its cover, you can bet that it would develop a loyal audience for that very reason. Would it be enough to outweigh using established properties to sell copies? I honestly don't know. I hope it would, but I have nothing to back that up with.

Aside from projects, though, it seems like the games press also relies on known properties in another area: developers. It seems like it's the same developers over and over again who get interviewed, either as independent pieces or to comment in broader articles. Again, I don't think putting "Exclusive Interview with Joe Q. Developer" on the cover is going to move more copies, but I know for a fact that there are a lot of interesting developers out there with insightful takes on the industry. Now, I'm sure that it's harder to cultivate new sources than to go to people you know will give you quotable responses, but I think there would be a real value in exposing people to more "experts". In fact, doing so might open up more stories about the interesting goings-on in the industry. From time to time, there is a call for more investigative journalism - instead of standard previews and reviews - and maybe with more sources, the press could get more original stories. I'd bet that for most journalists, the number of developers they actually engage with outside of the specific projects they're working on is less than a dozen.

So, why throw stones, you may ask, seeing as how I live in a glass house and the press has a lot of catapults? The games press is routinely critical of game developers and publishers, and sometimes I feel the industry holds back because they don't want to get in trouble. After all, we need them to say nice things about us. However, while I don't think the games press is ignorant of any of the issues I've detailed above, I think they're professional enough to take the critique and not take it personally. At least, the good ones are. Any good relationship is a give and take, on all fronts, and that's worth working on.

Back to top

Home

On the Couch

At the Office

In the Library

Through the Mirror

Mailbox

Neighborhood