The forum lies

I finally got around to watching Benjamin Cousins' (EA) excellent but lengthy talk about paying to win. In great detail and with disarming honesty Ben tells the story of how Battlefield Heroes was changed in December 2009 from a Free2Play game with just fluff and convenience items to a "pay to win" game. Weapons for real money were introduced that were better than any weapons you could get by playing, and simultaneously it was made harder to play for free. The users on the forums were extremely upset, and the press echoed the forum posters feelings and declared that EA had ruined Battlefield Heroes.

And if we wouldn't have the data Ben revealed in his talk, this is what we might have believed, that EA ruined the game. But the data show something completely different: There was no measurable negative impact on player numbers from all that uproar, both the amount of new players coming in and of old players leaving remained constant. And financially the move was a huge success instantly doubling revenue of the game, and turning it from a game losing money to one making a healthy profit. Which also means turning it from a game threatened with closure to a game that will stick around.

Further data analysys showed that the forum posters only represented 2% of the player base, that they spend 10 times more money on the game than the average user, and that they lied when they said they'd quit over this move. Ben even shows some hate mail he got, where he tracked the players and found that in spite of the hate, the players spent more on the game after the change. Thus when the CEO of CCP recently commented the uproar of the EVE community on a similar issue with "I can tell you that this is one of the moments where we look at what our players do and less of what they say", he was completely right. The forum lies.

This goes beyond Anil Dash's insight that if your website's full of assholes, it's your fault, suggesting stronger forum moderation. It tells you that people posting on forums are passionate about their game to such a extreme, that what they are saying is actually not relevant any more. There is no connection between the outcry of the 2% of forum posters, and what the other 98% think. There isn't even a connection between what the 2% say and what they themselves do. Trying to get useful information about a game from a forum poster is like trying to get an overview of how some modern religion is from its most crazy religious nutters. The forum lies, and is not be taken seriously as a source.

And of course the same applies to blogs. Doubly, because bloggers aren't representative of the player base, and the commenters on a blog are neither representative of the players, nor even of the readers of that blog. Most simple statements you can read on a blog or its comment section, like this or that feature or this or that game "sucking", are completely irrelevant. The more visibly angry the poster is, the more likely it is that his rant isn't representative of anything except his own anger. Which might not even be related to the game he is ranting about. Where blogs beat forums on the other hand, is that on a blog you are more likely to find actual rational arguments for a point of view, or original thoughts. Even in the comments section sometimes. Maybe the opinion is not representative, but if somebody manages to pull of constructive criticism (a rarity on the internet), this can be a source for ideas on how to improve a game. But to know whether something works or doesn't work, you still have to try it out and watch what your players actually do, and how their behavior actually changes in response to the changes to the game.