The beta for Wrath of the Lich King just ended, and I consciously didn't play it all that much, wanting to keep the content for the release version. I mainly leveled a Death Knight from 55 to 58, the main conclusion of which exercise was that it was a great experience until level 57, and not so great to do old Azeroth and Hellfire Peninsula again. And I looked at various professions, mainly inscription and alchemy. I had copied my alchemist warrior to the beta server, and was delighted to find that alchemy gets a new recipe for "research", with a 7-day cooldown. I find this is a great way to learn recipes, much better than having to repeatedly do some dungeon and hope for a 1% rare drop.
But as I didn't do a single Northrend quest, I was still level 70 at the end of the beta, and getting the herbs together for that research recipe wasn't all that easy. I first had to gather Goldclover in the Northrend starting zones until I made skill 420, and then go to zones with mobs level up to 76 to gather the other herbs. And that was where I realized a big difference between a beta and a live version: In the live version I will probably be able to buy those herbs on the AH. Overpriced, for sure, but they will be there. And as I plan to level my priest first, that might be the better option in the live version, except for the Goldclover. On the beta servers there were no herbs on the AH. Basically there was *nothing* on the AH. The player economy didn't exist on the beta servers.
The thing is that players behave differently on a beta server than on a real server, because their motivations are different. Why care about gold on a character who is going to be deleted at the end of the beta? There are various motivations to play in a beta: Wanting to help testing, seeing new stuff before everyone else, preparing for the release version, trying things out with less consequences, having fun. But by definition the long-term goals players often pursue on real servers aren't part of the equation on beta servers. Some fundamental parts of the game on the real servers just aren't important on the beta servers.
Other people already remarked that Warhammer Online was a different game in beta than live. People wanting to experience all of it, and testing the new things, lead to far more players doing public quests or open world RvR. Once the game went live, habits changed, because for the long-term goal of reaching the level cap fastest, grinding scenarios was a better strategy. One could say that without the carrot of the endgame dangling in front of them, players did what was most fun for them. Once the stupid idea of "the real game is the endgame" grabbed them, they optimized the fun out of it, reducing a huge game to a very small number of scenarios.
All this has consequences for beta testing itself. If people on a beta server behave differently than on the live servers, then what the devs can learn from the beta test is necessarily limited. Even if Mythic had added an auction house to the beta earlier (it was only put in last minute), not all that many people would have used it during beta, so we probably still would have had the current sub-par system. In the beta the balance between PvE and PvP, instances and open world, looked fine, it was only after release where it broke down. And Blizzard didn't notice anything wrong with retribution paladins on the beta servers, where very little PvP was played, only to have to nerf them "to the ground" repeatedly after patch 3.0.2.
So, are beta tests useless? Certainly not. They may not be a good indicator of player behavior on the live servers, but with beta testers trying all sorts of crazy stuff they are more likely to find obscure bugs. And for things like stress tests and server stability tests you simply need a large number of beta testers, using just a small team of in-house testers won't suffice. You just need to be aware that a beta test can't test everything, because in a beta the goals of players are different.
No comments:
Post a Comment