Statistics-Based RPG Statistics

The last post looked at some basic mathematics to show why RPG character statistics do not reflect attribute measurement in the real world. We were left with a basic quandary-- if numbers don't work for attributes, then what are we left with?

Fortunately, another class I took comes in handy for explaining a feasible alternative. Introductory statistics teaches the concept of levels of measurement. Depending on what you're measuring and how you are able to do that measurement, there are different operations available on the resulting numbers. For example, if you start classifying objects by color-- red, green, blue-- it is possible to count how many of each you have. You can say what the most common color is (the mode in statistics), but, reasonably enough, you can't say what the average color is. That type of measurement, where things are named with a label, is called nominal.

The next level is of most concern to this discussion. At the ordinal level, measured items can be put into order. There is no detail given as far as the distance between two items in the list, just that A is bigger or better than B. This is the type of measurement done for standardized tests. The test-taker is pitted against other test-takers and given a percentile rank that tells where in the order they fall.

For role-playing games, character statistics almost always come into play when the character is pitted against another character or a challenge in the environment. To use another real-world example, I knew a guy who was in the top ten list of Halo players on XBox Live. The system kept track of their performances and ranked them over time. He was obviously an extremely skilled and experienced player. I, on the other hand, played Halo once in a while and never really got past the stage of being frag bait for whatever experienced player I happened to materialize near.

Imagine the two of us were going to play head to head. In a normal timed death match there would be absolutely no way I would win. Some extremely unlikely circumstances would have to intervene, taking the game out of the realm of skill. For example, if my opponent had been awake for 3 days straight and could not focus on the screen, I might have a chance of killing him. If his controller started randomly changing his movements, I might have a chance. But aside from such outside "miracles" I would stand no chance.

Back to RPG land, a system realizing that character attributes are ordinal would rank all the characters from worst to best. The exact difference in ability between one and the next cannot be known exactly, but we do know who is better in an overall sense. All of the creatures in a current setting (say the adventure) that are capable of a particular trait can be put in order. If ranking intelligence, a small sampling might go: green slime, bugbear child, adult bugbear, Harak the fighter, Grazz the bugbear leader, Shandar the wizard, and the red dragon. Challenges requiring intelligence would be added to the same list, with their difficulty determining the position. So disarming the pole and pit trap may go before Grazz, while answering the ancient's riddle could go after Shandar.

What we end up with is a deterministic list of what any character is capable of. This becomes deterministic in the sense that without any extraordinary circumstances, the lower intelligence character or challenge will be defeated. Grazz or Shandar could disarm the trap, while Harak could not, and only the dragon could answer the ancient's riddle. While this might seem unsettling in games so wrapped up in dice rolling, another view to take is that it opens up the possibilities for strategy, quests, and other ways for the characters to beat the odds.

Imagine the case of Harak the fighter, alone, trying to disarm the pit trap. He is bewildered as he looks at the mechanism involved. But perhaps the inventive player decides on another route. He finds a nearby statue and positions it near the wall to provide cover. Then, he enters the bugbear lair and taunts the adult bugbear into following him, at a run, out into the hall. Harak ducks behind the statue as the bugbear lumbers forward into the trap. Harak watches as it goes off and slowly resets itself. In so doing, he has gained enough experience with that particular trap that he is now able to disarm it.

Of course, there are other implications to a game using this type of statistical statistics system. There are also ways to deal with randomness to introduce more player dice rolling. I will look at some of these issues in the next post.

The Percentile Realism Myth

My high school chemistry teacher was quite a character. Among other things he was (in)famous for starting class the first day of school by announcing, "In this classroom I am God, and [holding up our textbook] this is the Good Book." He was a very demanding teacher, but he was also very smart and treated us like young adults rather than children. One of the lasting things he taught me was the difference between accuracy and precision. In the decades that I have been around gamers, I have often heard the perspective that role-playing games using percentile dice are superior because of the increased realism. To understand why this is a fallacy, one need only look back to my high school chemistry class.

According to Mr. Klaus and the Good Book accuracy and precision are distinct, and this distinction is important to understand. The term accuracy refers only to the correctness of a measurement. I could measure my desk accurately using feet, centimeters, or popsicle sticks. As long as other capable people would get the same number as I did, it was an accurate measurement. Precision, on the other hand, is a statement of how finely the instrument used is able to measure. In the preceding example centimeters is the most precise measurement because it is the smallest unit. I could be more precise by using millimeters or other small scales. Notice that precision says nothing of my ability to measure. If I hold the measuring stick upside-down, the result could still be precise, but it would be wildly inaccurate.

Now consider the mechanics of your favorite RPG. It is likely that characters' abilities are rated with a number on a certain scale. The d20 systems like D&D use 3-18 for normal attributes and 1-20 for most skill rolls. Other games like RoleMaster and WarHammer Fantasy RPG use percentile dice with a value of 1-100 for these numbers. It seems at first glance like the percentile systems would provide superior realism. Thinking this, however, is confusing precision and accuracy.

Let's look at a real world situation for some perspective. Testing of human beings is certainly a complicated topic, but Intelligence Quotient (IQ) is a well-known measure of intelligence that should suffice for illustrating the point. IQ tests are designed to assign a number to a person's general intelligence where 100 is the average. The high and low scores vary by test, but for the purpose of this discussion, assume the range is 30 to 200, from non-verbal to super-genius.

One would assume that grades earned in school could be predicted by looking at IQ. Researchers have found that only 25% of the variance in students' grades can be attributed to IQ. Even more surprisingly, barely half of the variability in college entrance exam scores for mathematics and English correlate to IQ score. (For references and an in-depth discussion of IQ, see the Wikipedia entry.)

The underlying problem is that we have no precise way to measure general intelligence accurately. Even though IQ tests assign a particular number as the score, that measurement is not necessarily reflective of level of performance even in areas that seem to be closely based on intelligence. In addition, repeated testing can give results that vary fairly widely.

Now let's apply this information to gaming. Most systems give characters an Intelligence score. Imagine that your character is a computer hacker in a cyberpunk-style RPG universe. You are attempting to crack a security system to open a door. This reasonably requires a test of your character's Intelligence. However, as we have just seen, the performance of a specific feat cannot be predicted accurately by a measure of general intelligence. Having a score of 10 on a scale of 3-18 is no more meaningful than having a 53 on a scale of 1-100.

If this is the case, then what are we left with? Do we have to throw away numbers entirely and go with a system like The Window? Don't despair. My next post will look at some basic statistics that might help with this dilemma.

Immersive Virtual Worlds

I have posted a new article about artificial intelligence, moral ambiguity, and virtual worlds on my technology blog.