Try it, you'll like it!

The reviews for Final Fantasy XIV are rolling in, a month after release because Square Enix had asked reviewers to wait "three to four weeks post launch" before reviewing the game in order "to give the online game time to mature." Apparently maturing didn't help much, and Metacritic records an average score of an abysmal 50 out of 100, with the user reviews being even worse. I especially liked the Gamespy review saying:

I can't help but feel that FFXIV is cosmic punishment, meted out by some avenging massively multiplayer online deity for my years of complaining about the state of modern online RPGs. They're too simple, I've whined; too hand-holdy, too easy, too friendly, and too safe. FFXIV is none of these things. It is the definition of obtuse: poorly designed, aggressively underexplained, and shoddy in almost every respect that matters.

But then of course *some* people love Final Fantasy XIV, and are complaining that these bad reviews are unfair. That is a recurring event in the discussion of game reviews, and especially game review scores. It is somewhat inevitable: How do you boil a complex interaction between thousands or even millions of players and a game down to a single number? What exactly does a review score express?

The one thing a review score never is, is "fair". If you look at sites where players can score a game, you will quickly see signs of manipulation. A natural distribution of scores would be a Gaussian bell curve, but in reality you often see extreme scores of 0 or 100 put in to manipulate the average. Most famously the review scores on Amazon get manipulated all the time, for example by groups of people giving the lowest possible score to a game because of it's copy right protection scheme. Review scores of professional reviewers often are designed to avoid extreme scores, but because the reviewed games come from the companies that pay for advertising in these professional publications, the average is skewed towards a higher value. A score of 75 is an average game, a score of 50 is abysmally low, and lower scores are only given out to low budget games.

Even taking all of this into account, a review score can never measure how much the fans of a game will love it. Or how much other people might hate the game. The best a review score can come close to doing is to give an idea of the probability that an average player will like a certain game. If you had to recommend a game to somebody about whose gaming preferences you know absolutely nothing, a game with a high review score gives you a better chance that a recommendation of "Try it, you'll like it!" will be okay.

And by that measure, I think the reviews of Final Fantasy XIV are fair enough. Even the fans agree that FFXIV isn't for everybody, and that a random player stumbling upon the game has a high probability of being disappointed. That doesn't mean that some people can't love the game dearly. Nor does it mean that Final Fantasy XIV doesn't have good features. Even the bad reviews praised the artwork, the crafting system, and the profession switch system. But the reviewers are right to state that if an average player tried the game, he probably wouldn't like it. And that is the best a review score can do.