The Problem With A Rating System

I always get a little irritated when I am driving in my car and a certain commercial comes on the radio. It goes a little something like this; “A scientific study was conducted, 100 people were blind-folded, given samples of Coke vs Pepsi, and asked which one they like better” …….

First of all; THAT is not a scientific study. You’re merely asking 100 random people which they like better; Coke or Pepsi. It is ACTUALLY an example of a subjective observation. A subjective observation is one with a bias or prejudice. There is no right or wrong answer; each of those individuals are entitled to their own opinion. We use subjective observations all of the time. A person that you might find really attractive, might not be as attractive to others. The expression that ‘beauty is in the eye of the beholder’ carries a lot of weight. We use subjective observations with the clothes we wear, the foods we eat, and the games that we play.

Why use rating systems or lists at all? Ratings and lists appeal to most people. Life is full of tons of data, and people are always looking for ways to narrow down that data. Lists and rankings help in that regard and help us to make decisions; and psychological studies have shown that the shorter the list the more people are likely to believe the list and/or rankings. They help in making comparisons; which can be especially helpful for consumers.

I woke up early this morning very excited to create a ranking system that I would use for each of the games that I play to share with you here on the blog. I spent an inordinate amount of time reviewing other rating systems that are out there in order to help me form my own. I started mapping out what I wanted to rate; and how my scores would add up; and then the voice in my head started to speak. If you’re asking yourself; ‘What voice?’ then YES! that’s exactly the voice I’m talking about. My internal voice started pointing out potential red flags and problem areas and the more that I thought about it the less excited I became about a ratings system.

There a couple of conflicts in play here:

The first conflict is that reviews are biased because those reviews are made with subjective observations. The same bias comes into play regarding the computer games you play just as it does when you’re making a choice between Coke & Pepsi. I began to realize I’m not exactly a game aficionado. I have the passion and enthusiasm but part of the reason for my journey is that it is simply one that I have never taken before. So, you dear reader, will be experiencing things for the first time through my eyes. You might appreciate my opinion but you don’t necessarily have to agree with it.

Understanding THAT led to my second conflict which is this: I wondered if a rating system is so subjective, could I instead use objective observations and use a point rubric then total it up? I could certainly do that. Where game mechanics are concerned, for example, I could get into character creation and development, how much choice you have, interactivity with the game, immersion and so on. I could get into comparisons of graphics & sound……..and then I completely put on the brakes. I put down my pen and I realized that this was suddenly looking like an awful amount of work. Would I find it to be entertaining still or would I burn out? Would the readers find it useful and entertaining? Which led to more conflicts.

My purpose here is to take a trip through time and go back and play computer role-playing games in chronological order. It is a quest I will likely never finish because the list of these games is exceedingly long. The greater the list becomes; the murkier the rankings become as the list grows and grows. It should become harder to make comparisons and for readers to interpret when the list becomes too long. The larger conflict is that my journey begins in 1975. Technology in 1975 was vastly different than it is now. The hardware and technology constraints defined the games that were played from 1975 to 1985. So we’re really not comparing apples to oranges here. A game that might have been a crowning achievement in 1975 pales in comparison to Elder Scrolls Online as an example. My journey has started with programs that were played on the Plato system. These games used ascii characters for graphics and there were no sounds at all; not even a single beep. So any ranking numbers we used would be skewed; so it didn’t seem coherent to me to compare a game from 1982 with one from 2020.

I believe if I am doing a good job communicating my experience with the game to you; sharing screen shots; talking about the story & setting, presentation, mechanics (game play & character creation), difficulty level, and the overall enjoyment I personally had with that game, that YOU are the best judge as to whether you want to spend time experiencing the game yourself. I want to share my passion and enthusiasm and love for games with you, not muddy the waters with a subjective rating system which may or may not make sense to you.

I will share with you whether or not my experience with the game was a positive one, the difficulty involved, and how much time I spent with the game, and how you can play the game as well. I think that is information which you may find useful. Perhaps someone will come along with a cognitive argument and change my opinion on the matter; but for now I’ll use my free time to play and experience these games and share that experience with you.

I’d love to hear your own thoughts on this matter.