I was tracking my XBox achievements on
True Achievements the other day when I realized I had statistical
proof that DJ Hero wasn't a very popular game. Specifically, my multipliers are
ridiculous.
First, some mild geekery: when you accomplish certain tasks in an XBox game, you're given achievement points, or "G." Finish the first level, you might get 10G. Kill all the enemies on a level using only your fists, you might get 50G. The harder the accomplishment, the more G you should get.
...in theory. In practice, since developers design their own achievements, some games are overly generous (in one Simpsons videogame, you get 5G for just pressing the start button) and some games are ludicrously stingy (if you can find all 200 glowing balls in Prototype, you deserve more than 50G). So some of the hardest accomplishments may well short you on G, whereas a game that's generous (usually kids games) hand them out like candy.
So to even it out and have your score reflect your actual gaming talent, True Achievements evens it out by assigning multipliers based on how many people have this game, as compared to how many people actually got this achievement. For example, that Prototype achievement may be only 50G - but True Achievements sees that only 12% of game owners have actually gotten this level, and multiplies that achievement by 2.88 for an "adjusted" total of 115G.
(Of course, since you have to sign up for True Achievements before it starts tracking, it means that 12% of
the most devoted gamers in existence have gotten this achievement - the actual number's probably closer to 2%.)
Though DJ Hero has some fairly easy accomplishments, there aren't a whole lot of people who've gotten them. Which means that compared to other games, there aren't a lot of people who've picked up DJ Hero and really gotten into it. In other words, it may have been purchased, but it's not getting
played.
As a game publisher, I'd be fascinated by those numbers. The paradox of the sales is that the sales are
roughly equivalent to quality, but not exactly. A heavily-hyped game could sell a lot before people realize it's not good. The sequel to Halo 3 (or ODST) is going to sell a zillion copies even if it's a remake of Pong. What the Achievements can tell you, if you structured them appropriately, is whether people enjoyed the game once they
bought it.
Just as a quick example, one of the fine tricks of any rhythm game is your choice of songs. Did you get the mix right? Did people, by and large, feel that this is a good mix? Well, if you put in an accomplishment that could be achieved
incidentally, like "Played all songs," then you could see what percentage of people thought your songs were good enough to play all of them. (It wouldn't be strictly true, since there are idiots like me who'll unlock everything, but you can filter me out.)
Likewise, you can see what level people got to on average before they quit. Was that level too hard? Did the story flag here? Regardless, you could analyze those breaking points and try to find out what stopped people, and fix that in the sequel. Or to see whether this game deserves a sequel, regardless of the actual sales up-front.
There's whole worlds of data to analyze. And then I think about Netflix streaming, where you can see not only how many people watched the movie, but how many got to the end - and the points where they gave up. What happened at this point in this film where people flung up their hands and said "Fuck this," and how can you fix that in future movies? Obviously, there will be anomalous points of data, where the baby started crying and they just figured what the hell - but with enough viewings, you can see where people paused, where they walked away, giving you some clue as to what they liked about the movie. And you can see what movies get rewatched, and what sections of movies get rewound, and you can
use that.
I'm not a data miner. But if I was, man, I'd be drooling over the access to all of this. It's incidental tracking to see how you reacted to something, anonymized, and usable. I'd like to think that someone was putting all of this glorious data to good use.