The Moment of Evaluation

Do you have a book that defines you?

For me it was Don Norman’s The Design of Everyday Things. My copy is worn beyond its years, its pages dogeared from beginning to end. To say that it was formative in my game design education would be an understatement. Still, to this day, if you asked me to recommend one book to a young game designer I’d slap that puppy down on the table faster than you could finish the question.

But… these days it feels a little rougher around the edges. Not in its ability to define design terminology, such as Goals, Affordances, Mapping, and Intentions, but in my ability to apply it into a day to day game design process.

Thinking of your game in terms of Goals and Intentions will provide you with a holistic framework, but it led me to approach system design from a Top Down manner. I’d start from the highest, semantic level possible, and weave my way down until I got to the player’s experience. This is all well and good, and my designs were always, at least initially, strong… but they were brittle.

I can tell you with absolute confidence that not a single one of your designs will survive the process of going into the game. I have yet, in my nearly 15 years of making games, designed something and watched it arrive unscathed into the final product. Sometimes it can be as minor as the color of a scarf changing, sometimes the system’s architecture balloons in ways you never intended, and sometimes… it’s just not fun.

So you make corrections, you try new things. This is good! But… this is also where my problems start, as a process that does not EMBRACE the chaos of implementation is doomed like a person trying to hold up the walls of a sand castle. My old process was about tight semantic definitions, and it was easy to forget that the very legs you are trying to stand on have already shattered into a thousand pieces.

Undoubtedly, a top down focus does not prevent you from handling the ripple effects of minor changes, but it doesn’t make it CLEAR that you need to assess the situation. It is not a BUILT IN part of the PROCESS.

Moreover, it’s hard to communicate with the player when you’re staring down at them from the top, as you tend to make a lot of assumptions and plans for the player, instead of putting yourself into their shoes. Does the player REALLY understand that “flanking” is a possible and valid strategy in your shooter? Or do they just think, “shoot guys till dead, I win” — a completely valid strategy… maybe the superior one.

So I needed a new process. Not just one that sees the game from the top, but one that works from the bottom up. One that, at least in some way, embraces the chaos. I also required a greater parity between my analysis of the game as a set of systems and features, and the player’s experience of the game. Understanding your game by defining goals for the player is still of utmost importance, as it tells you WHAT you want to COMMUNICATE to the player; however, as we’ve discussed above, when you spend all your time analyzing from this singular perspective of WHAT, it can blind you to the other side of the equation. Enter Scenario based design.

Designers tend to aggressively adopt the philosophy of Scenario or Semantics, as I sure did, but these days I accept that you need them both. The Scenario-based portion of my blend is what I call the Moment of Evaluation. Once you begin to think of a users experience in terms of evaluation, you will have a much easier time surviving the prismatic chaos of initial implementation.

So What Is Evaluation?

A defining feature of games as entertainment media, especially action games, is the frequency with which we ask the player to analyze the system; not only in how it responds to your inputs, but also the validity of your past choices. Depending on your style of game, your systems are always in flux, so that which was advantageous before can suddenly become a liability.

Let’s say, for example, we are making an action adventure game, and the player has 3 mechanics: A Light Attack, a Heavy Attack, and a Magic Power. For 80% of the game, light attack has been very effective (note: not necessarily optimal). But now we introduce an enemy that automatically counters any Light Attack.

We are asking the player to EVALUATE the validity of what she is doing; but also, and more importantly, we are asking her to break from a pattern she has built up. Her strategy up until this point was very simple: “Mash the Light Attack button!” And, what’s more, we’ve never told her that there is anything wrong with that (and should there be!?).

How long do you think it will take this fictional player to change her pattern? You might say, “well yeah, but we made sure to tutorialize that some enemies can counter your attacks!” But… that was like 15 hours ago in the beginning of the game with the rest of the tutorial. Maybe 15 DAYS ago, because she’s taking her time to play the game. Why do you assume she remembers jack shit about that? But even more than that, after going 80% of the game with the same general pattern, would she even be thinking about strategy?

In traditional parlance, she needs to be pulled out of her Flow. She must be shocked into a state of Evaluation, where she can actually consider that her current strategies are not working, and there might be a better way.

The above is an extreme example, of course, and Evaluation is not always so sudden and demanding. In a fighting game, as another example, you are expected to be constantly evaluating your opponent.
In either case, whether it is a frequent occurrence or a rare one, the process of Evaluation is a three step sequence:

  1. A Shock to the System
  2. An Analysis of Clues
  3. A Choice of Strategy

The Shock

Your player is not a hyper-aware computer that is purpose-built for analysis. Often when playing games we find ourselves falling into a rhythm and pattern. This is the antithesis of evaluation.

Note: The longer a player goes without evaluating the state of the game, or their strategies, the more friction they will have to the entire process, and the greater the shock they require.

This is not to say that flow is bad, of course, but a shock is REQUIRED if you want to awaken the player out of flow, or they won’t realize that now is the time for thinking. There are a LOT of ways you can do this, but I like to think of it as a 3-tiered system. You have Mild Shocks, Routine Shocks, and Extreme Shocks.

  • Mild – Sounds and color changes on enemies, icons appearing at the edge of your screen, mild controller vibration
  • Routine – An enemy spawning, a loud sound, a flash of the screen, character death, strong controller vibration
  • Extreme – A cutscene, a tutorial popup message, multiple character death

It is possible to be more granular, but I like thinking of it in terms of a 3-tiered system, as it meshes nicely with the next step.

An Analysis of Clues

The Hierarchy of Reasoning is a powerful method of defining communication with the player. A full breakdown of the three clues and what they mean is a bit beyond the scope of this document, so please read more about it here: The Hierarchy of Reasoning. In short, the three clues are defined as follows:

  • Primary – The first thing you notice about something. Usually a strong silhouette. It would stand out in the periphery of your vision.
  • Secondary – The second thing you notice about something. It would stand out as you snap your focus onto it.
  • Tertiary – It would stand out after looking at it very closely. Two things that differ only in tertiary clues will be difficult to judge quickly.

With regards to Evaluation, once the player has been shocked, they must analyze the scene. But again, as I will continue to repeat, the player is not a perfect automaton. Their analysis is limited to that which they can see, and understand. Primary clues are going to dominate over Secondary clues, while Secondary clues are going to dominate over Tertiary clues, at least initially.

This is an important consideration for your designs, as the type of read will determine not only what stands out, but also the time it takes to spot and analyze. There are times where the only differentiation between two enemies is bits of armor and colored cloth. If that is meant to be the deciding factor between two different strategies, then tertiary clues like that will take a while for the player to evaluate, assuming they even spotted it.

Not only THAT, but seeing the Clues is only one half of the process. The player also requires the Holistic Knowledge to interpret those clues. The two types of knowledge (mechanical and holistic) is something I have talked about before. They are very intertwined with this new process, as it relates to the next and final step.

A Choice of Strategy

A strategy is a filtered set of mechanics that either creates an advantage for the player, or reduces the advantage of an opponent.

Back in the day I would have called this the player’s Intentions, and some like to call this the “Verbs” of your combat. A point I will continue to reiterate is that your job is not done once those strategies have been designed. You have to teach those strategies to the player, or they will never understand your game. Failing to teach the player can lead to both frustration and boredom

  • Overwhelmed in Frustration: They lack the holistic knowledge to filter actions and mechanics into strategies.
  • Underwhelmed in Boredom: They lack the mechanical knowledge of an action, or they do not understand the viability of a strategy.

Avoiding Frustration and Boredom are great, but there are even greater benefits of this process. But, to talk about that, we have to start talking about how one actually designs for this. I’ve spent a lot of words defining what Evaluation IS, now we need to talk about the process.

Designing For Evaluation

How do you “design” a Shock condition. How do you “design” a Strategy? There is no RULE that I can give you, but one of the best tools I use is a simple checklist, which you can read more about in full detail here: The System Designer’s Checklist.

As I am mentioned above, we cannot only focus on the player from a purely semantic position. We need a blending of the two, and this checklist is my way of blending the two pillars. The relevant part from that process is where I embrace the scenario driven axis of design, through what I like to call my Evaluation Tables.

Let’s say I’m working on an Action RPG, and my job is to design a bunch of enemies for the player, and at the same time, design abilities for the player that might create interesting gameplay.

The first thing you do is you write out a few sample scenarios that you can picture in your head. Don’t just write down, “enemy with a shield” or something. Describe the full scenario. The POINT of this step is to remind you that nothing in the game exists in a vacuum. Context is almost more important than the thing itself, when it comes to the player’s experience!

Here are some examples:

  1. The player is at the beginning of the game. They are playing a Mage. They do not have any talents. They are wielding a wand. They are fighting 2 Zombies. The player sees the zombies, and shoots them with her wand until they are dead.
  2. The player is at the beginning of the game. They are playing a Mage. They do not have any talents. They are wielding a wand. They are fighting 10 Zombies. The player sees the group of zombies. She shoots them with her wand until 2 are dead, and then moves away to create space. She continues this process until they are dead.
  3. The player is halfway through the game. They are playing a Fighter. They have taken the talent Shield Bash. They are wielding a hammer and shield. They are fighting a Cockroach Queen, and 10 Cockroaches. The player ignores the cockroaches, and focuses on the Queen. When she sees the Queen summoning more Cockroaches, she uses her Shield Bash to stun the Queen. Once the Queen is dead, she focuses on cleaning up the Cockroaches that are left.
  4. The player is near the end of the game. They are playing a Fighter. They have taken the Last Stand talent, and the Shield Bash talent. They are fighting 2 Vengeful Spirits. One has spawned with the modifier Damage Reflection, the other has Spawned with Berserker Frenzy. The player Stuns and Focuses on the Vengeful Spirit that has the Berserker Modifier. Once it is dead, she activates her Last Stand ability and focuses on the Vengeful Spirit with Damage Reflection

It’s ok to have some scenarios that are very similar (as I’ve shown you above) as long as it means the player was forced to Evaluate. Remember, that’s why we are doing this!

Ultimately, there is a LOT more context that I could give, especially as it pertains to the kinds of talents the players have, and the kind of gear they are wearing, but AGAIN… your goal here is just to provide the RELEVANT context, so what I have written above is more than fine.

Once you have your scenarios, now is the time to make a table. I usually do this in my notebook, and it looks sketch as fuck. It doesn’t need to look pretty.

Scenario Act (1-5) Data Action/Mechanics
One 1 Grunt Enemy, Mage Basic Attack
Two 1 Grunt Enemy, Horde, Mage Basic Attack, Movement
Three 2 Lieutenant Enemy, Pest Enemy, Summoner, Fighter Special Attack, Stun
Four 3 Major Enemy, Enemy Modifiers, Fighter Special Attack, Stun, Temporary Immunity

There are four columns in this table: Scenario, Act, Data, and Actions/Mechanics.

Scenario

Use whatever shorthand you want to remind yourself to which scenario this row is referring.

Act

I usually grade this on a scale of (1-3), or sometimes (1-5). One being at the beginning of the game, and three being at the end of the game. I’ll talk a bit more about why this is important down below.

Data

Here I try to list out all of the relevant information that the player must analyze in order to correctly deal with the scenario. When it comes to enemies, sometimes I will call out the specific enemy, and other times I’ll call out the generic archetype of the enemy. (More on Enemy Archetypes, here). As a general rule, if my scenario involves multiple enemies of the SAME archetype, then I will be specific; otherwise, if it just involves different archetypes I’ll stick to that.

Actions/Mechanics

Finally I list out the relevant actions or mechanics that the player would use in this scenario. I try not to get bogged down by “what is an action” and “what is an ability” I just try to list the high level relevant information.

So, why are we doing this? This is the first step in a two step process, and the first thing I’m doing is I’m looking for DATA + ACTIONS combinations. What was the critical data that the player needed to make the correct choices?

Take Scenario 2. In this case, the player needed to understand that a “Horde” of enemies is not meant to be taken on without moving away from them constantly. Is this only relevant for ranged classes? It might not hold true for the Melee hero, but I’m not ready to make that call just yet. At this point, I’d probably add a new scenario for the melee hero, but I’d also jot down “Kiting = Ranged Player + Horde?” I don’t need to answer this question just yet.

The other question we are trying to answer is about communication. How did the player KNOW. Did she have to DIE in order to learn that getting surrounded by a horde is bad? Am I ok with that? I might be. (Don’t be afraid to kill your player as a means of Shocking them). But more than that, WHEN did they need to know?

In Scenario 3 we have a player that KNOWS that Summoner monsters are PRIORITY TARGETS. When did we teach that? Act 2 is halfway through the game, at least. Even more important, even if we HAVE taught this, how did they know in THIS scenario?

The answer to SOME of these questions becomes easier the more of these that you do. If you find that a LOT of your scenarios involve the player needing to know the same bits of information, then you can be very sure that the player MUST learn about it. If it’s more infrequent, then it might be something you are ok with being a spike in your difficulty.

You get the idea. For each scenario, you are looking at your Data and your Actions and you are asking “How did they know, when did they know, and is there a common pattern emerging.” When I sat down and wrote those scenarios, I did not say to myself, “I need to write a scenario that involves Kiting.” I just wrote out some cool scenarios that I could picture, and kiting clearly emerged.

But we’re not done. Once I feel I’ve got a good grasp of this table, I make one more.

Scenario Danger (1-5) Shock Clues
One 1 (1) Seeing an Enemy (3) It looks like a monster
Two 2 (1) Seeing an Enemy (3) It looks like a monster
Three 3 (2) There is a particle effect that players when it’s summoning (2) A special animation for the summoning
Four 4 (1) Enemies with Modifiers get unique nameplates (1) UI Element Color Change

Ok, what is all this telling you. There are 4 columns: Scenario, Threat, Shock, and Clues.

Scenario

Again, the number of the scenario to which this refers.

Threat

This is another shorthand I use. I grade the “Danger” of a scenario on a scale from 1 to 5. One is the lowest threat possible, and Five is reserved for extreme danger. A typical “danger five” scenario might involve the player dying multiple times. If I ever feel that a scenario I’ve described is that dangerous, it’s time to ask myself if that’s what I really want.

Shock

The Shock is the moment that forced the player to realize they must evaluate their current situation. I grade this on a scale of (1-3), where one is a really weak Shock and three is an extreme Shock.

Clues

This refers to The Hierarchy of Reasoning (read more here) of conceptual design. This column is where I specify the HIGHEST clue needed for the RELEVANT information that the player analyzed in order to reach the CORRECT combination of mechanics (strategy). 3 being a Primary Clue, and 1 being a Tertiary Clue…

Yes, I realize that’s BACKWARDS. Just stick with me, there’s a reason.

This can be a tricky one. Take the example above where the player is facing off against a single Zombie. Do you put 1 or 3? Well, in this case, you only have one threat. You define the highest level RELEVANT clue. In this case, you are spotting an enemy out of the environment, which involves it’s Silhouette and Motion. That is a Primary Clue for this monster. The same as with Scenario Two. Scenarios 3 and 4, the relevant clues are not just the enemies themselves, but between different enemies, sometimes of the same unit. The RELEVANT clue might only be tertiary at that point.

I define clues backwards because it allows me to quickly spot potential issues. If “Shock + Clues” is LESS than the Threat, then I have a problem.

1 (Unique nameplates) + 1 (UI Nameplate Color) < 4 (Threat)

We can see in scenario four, the threat that the player faces is pretty great, but the Shock is mild, and the clues on it Tertiary. For quick shorthand, we have already determined that this is going to be frustrating for the player. We should consider increasing the extremity of the Shock, and we should also consider bumping the read on modifiers to be Secondary, potentially Primary.

When combined with the previous table, we can begin to really dig into the questions we are trying to answer. Here are some examples:

  • Are there common groupings of mechanics? These seem like strategies we’d want the player to learn.
  • How often do certain mechanics seem to crop up? Are we making sure to tutorialize the features the player might use the most?
  • Is there a large time gap between certain mechanics? Remember, the longer they go without using something that we’ve tried to teach them, the less likely they are to utilize it.
  • Are there any mechanics we haven’t designed a scenario for? Should we? Do we still need this mechanic?
  • Are there any mechanics we don’t have designs for, but we’ve discovered from writing a scenario? Are we leaving something useful and cool out?
  • Do we have two very different strategies, which depend on certain enemies, and do we ever imagine a scenario where those two enemies can appear together?

That last question is a pattern that will begin to emerge the more you do this, but you might need to create a more robust table solution to really begin to see that kind of pattern emerge, which brings us to some final notes about this

Closing

Semantic driven design is not the answer, at least for my personal process.

But neither was scenario based design. It took many years, but I have finally landed on a blending of the two, and part of that blend is this process of understanding the player in this Moment of Evaluation.

Your player is not a perfect automaton. To properly design your system, you must understand that they only perceive that which you clue them in to, and they won’t listen unless you give them a little shock — but more than all of that you must accept that writing your game design document does not mean you get to walk away.

Your job isn’t done when your write the design doc, and your job isn’t done when it’s first prototyped… you will know your job is done when you see it in the hands of someone else, and you no longer think, “why don’t they get this…”