This is something that a fair few older game designers often ignore, and it's because for a decent amount of the history of game development, design has been an airy fairy, hand waving, throw things around and see what sticks, kind of thing. It works to some extent, because a lot of the games we used to make were small enough that what we were actually designing was game mechanics. But that's not true any more. Now, games have a few mechanics they want to explore, and those mechanics might be refined over the development of the game, but in truth, the throwing stuff around and seeing what sticks stage of development is over pretty darned quick now. Plenty of companies that don't work with data have had a "great idea", implemented it, and then at publishing time, found that the thing on which they worked really hard is not received well because it's not very good. It's not even like the developers are likely to have thought it was really good while it was in development either, but for some reason, there's a blind spot for failure to make an engaging design. It's like they're frightened to measure how well the game is doing, in case they have to change something. It's almost like they're trying to win logical fallacy bingo. A bit of cognitive dissonance here, a bit of sunk-costs there, and a bit of gamblers to top it off.
Some companies make a big deal out of measuring their games, and they usually do pretty well. The spate of freemium games could not have landed such huge amounts of money without the constant attention to performance indicators. Measurements are key to a game that isn't actually designed to be "fun", but instead designed to extract cash from people with their consent. To this end, it appears that maybe the reason some people don't want to check to see if their design is good, might be that they want their game to be good without being polished by maths, or they want it to be a cult classic, warts and all. They don't want it to be as good as possible because they've sold their soul to the same people that make Cash of Beach, or Game of Candy. They want it to be as good as possible because they want to be proud that it's all their idea.
Using metrics saves you time thinking about stuff like "would the player find this too difficult" and allows a designer to concentrate on the actual inventing stuff. Being freed from having to do the laborious play-throughs and focus tests with hand written notes, gives the designers the mental space to come up with truly innovative game elements, and in house testing with feedback allows them to be able to see if they will work, or do work, without having to commit vast sums of money to them. Design, and art, love constraints, but hate the constraint of time. Give your designers and artists strict technical constraints and all the time in the world, and you end up with beautiful things. Stop them from working by not letting them utilise a universe of tools and metrics for seeing if their work is successful, and you end up with a big ball of ideas that only work in the heads of those that invented them. Nothing can be designed in a vacuum.
Graham McAllister on Molyneux's strange decision to ignore data.
"However, Peter is not alone in making such reckless statements. I was recently at a conference where a speaker was saying that all that was needed to make a great game was to hire smart people and have good ideas."
He's a different era of game designer. There's nothing inherently wrong with how he designs game mechanics, but my opinion is that he does need tempering, as without data, it's very hard for him to see what it is he does not know. Peter did found the cannery on the tagline "A great team with a great idea leads to great success".
This thinking is lovely, because it makes me sure that at the heart of it, Peter still believes in making a great game. The idea that numbers and metrics are something you don't bother with at the beginning, reminds me that at that desk, in that office, sits a game designer that really believes in striving to create something great, something unique, and something that will make a difference, all by sheer force of will, dedication, and persistence.
I don't think he can, which is why I don't work there any more, but I want to be proved wrong, because I think we'd be worse off without anyone like him doing it his way.