Intrinsic rewards and grinding in video games

In one of my earliest blog posts, I talked about my model for intrinsic rewards in video games. The basic idea is that many games create a loop, such when you follow the loop the reward for an action is being able to do that action more, and in a more challenging way. Do something to do that thing more is an intrinsic reward: do something to get something else is an extrinsic reward. Seems simple, but gets really messy when applied to different game genres, and my recent experience with Fallout Shelter made me realize this needs a deeper look. This blog post will talk through how intrinsic rewards play out in different games, in particular how they are related to the issue of grinding and meaningful gameplay.

 

To me, the classic intrinsic reward game is Final Fantasy I, so we'll start there. There is a very basic loop created in this game. Namely, you 1) fight monsters, to 2) get gold/exp, to 3) go up levels and buy better stuff, to 4) get stronger, to fight better monsters and repeat the cycle. The key components of the cycle are steps 1) and 3), as these are the steps where a player performs actions and has agency. In this game, the fighting of monsters is somewhat strategic and takes some player skill that is independent from how powerful their characters are (you need to know what attacks to use against what enemies in what order). Buying stuff also has some strategic value, mainly because you never have enough money to buy everything that you want, even if it's always clear what you should buy for what character if you have the money available. But in general, having more money and levels makes you better at fighting monsters.

I'll also mention that Final Fantasy to me has done one of the best jobs weaving grinding into the mechanics and narrative to the game. Not being able to save the game often, especially in dungeons, makes those dungeon expeditions especially epic. You need to build up for them, to make sure you survive through them to when you can save again. Grinding helps you build up the strength to survive the dungeon, and added significantly to the epic feel of the game.

At this point, let's jump to another game, The World Ends with You. In this game the formula really isn't that different from FFI. 1) fight noise, 2) get XP and Yen, 3) raise the levels of your character, pins, and buy better items, 4) get stronger, and fight more difficult battles.  What's really different here is a few minor but important details. I'd argue that 1) is now an even more meaningful interaction than FFI, as every fight is basically a puzzle that requires different kinds of actions to be successful. Additionally, finding the best pin/item equipment is also a puzzle in itself, making 3) a more meaningful interaction. But the biggest difference is that a significant amount of player agency has been added to determining difficulty of battles. Players can constantly decide how difficult they want the game to be, by lowering their level or increasing the game difficulty. Doing either increases the proportion of valuable drops, which allows the player to get stronger faster. In other words, players now also have control over the pace of grinding.

Jumping one step further in this direction is puzzle games, like Osmos (or reflex games like DDR). In these games, you are grinding your skill through repeated play attempts, without any real notion of a reward structure. This is somewhat similar to The World Ends with You in that you did have to grind your battling ability, but in TWEWY you also grew stronger over time through intrinsic rewards built into the games (namely, leveling up). In puzzle games, the game rewards are taken out, so the loop looks like this- 1) beat puzzles to 2) increase your puzzle solving skill to beat more difficult puzzles. Note, that this is similar to how hobbies and other real life activities work.

To take one more example of this point, let's look at the gun mettle campaign in Team Fortress 2. This is basically a questing structure laid on top of a multiplayer game. The quests are accomplished through grinding-like actions. The actions are performed on multiplayer games, and so in some sense aren't like an internal-game grinding structure. The quest rewards are also extrinsic rewards and social in nature (which makes sense, given that the game play experience is social). So the structure is 1) Play multiplayer games to 2) get gun mettle XP points to 3) get better at performing the action that gets you those XP points. Action 4) is to repeat 1-3 until the quest is achieved and you unlock a unique-looking gun that makes you look extra cool in future multiplayer matches. It's unclear whether 4) links back into 1) and creates an intrinsic social reward loop, or simply sits on it's own as an extrinsic social reward. But in either case, there is something that resembles an intrinsic reward loop for a certain period of time, and certainly involves grinding (in a game that otherwise does not involve grinding). What makes TF2's system especially meaningful is how carefully they've crafted their quest point-gaining system. You gain points most quickly by performing specific play strategies with certain classes, meaning it implicitly encourages you to perfect new play strategies and therefore improve your skill at the game.

At this point we can jump back to puzzle games with quest structures, most notably Candy Crush. The classic puzzle-game-with-quests, this calls for a grinding-like reward loop. But, it generally falls short of being meaningful. Let's look at the structure. 1) Play puzzles to 2) unlock more levels, and repeat 1) and 2) until eventually you 3) play more novel puzzles with new mechanics. Some differences- notice that 2) and 3) doesn't always lead to more difficult puzzles, and I think this is important in making the whole loop itself less meaningful. The puzzles don't structure themselves in order of higher difficulty, they structure themselves in ways that induce frustration and promote pleasure at the discretion of the game designer, rather than the player. A few easy levels, with a tough level, followed by a few easier levels, inducing one to feel accomplishment for a while until the inevitable frustration at getting stuck at another hard level. Playing more is also not encouraged, as failure causes one to wait for health to regenerate before being able to keep playing. The only way to get through difficult levels is through paying money, meaning there is a extrinsic input to the system to avoid the frustration of the poorly crafted level structure (poor from a game design standpoint, excellent from a money-making standpoint). In other words, this game does induce grinding, but the grinding does not link into an intrinsic reward loop, but rather can best be relieved by an extrinsic monetary input. Nothing about playing the game allows you to play the game more or better. Although here the mechanics of the game are meaningful, the game structure is not meaningful.

Compare Candy Crush to Tetris, in particular the unlimited play mode. Tetris has incorporated grinding in a nice intrinsic reward loop. 1) Play levels to 2) advance to harder levels. There's also a minor additional loop of 1a) play initial levels better to 1b) start harder levels in a better place to 2) be better at advancing to even harder levels. This minor loop encourages more skilled play, making 1) an even more meaningful interaction. Although I'd argue that based purely on game mechanics Candy Crush is as meaningful an experience as Tetris (or Bejeweled for that matter), I'd also argue that the structure of Candy Crush creates a situation where grinding is used to degrade the play experience whereas in Tetris the game structure causes grinding to enhance the play experience.

Ok, let's jump back to FF1 and go down a different path. The Final Fantasy series has been evolving with time and continuing to embrace its narrative theme with each new title, while simultaneously phasing out grinding. In other words, although you do go up levels, the process of going up levels never really seems to advance or impede your play. They have either balanced the story so well, or have enemy strength respond to your current ability, that you are never stalled from proceeding further in the story because the enemies are too strong. The omnipresent save has also eliminated the idea that there just might be too many consecutive battles for you to survive a dungeon. This then entirely focuses on moving the game through a compelling narrative, while using the battle as a relationship-building exercise between the game characters and the player. The interesting thing is that the reward loop still looks exactly the same as FF1's reward loop, it's just that the action of fighting monsters becomes less meaningful. To compensate, the game designers have created more interesting ways of building up character's abilities, like FFX's classic skill tree. In FFX in particular, the meaningful interaction of fighting monsters has almost entirely disappeared in favor of more meaningful leveling up. So 1) becomes less meaningful as 3) becomes more meaningful.

This raised an interesting point for me, in that now I saw a connection between RPGs and god-view simulation games. If we take a game like SimCity, we can also break it down in this way. So here's the loop in SimCity: 1) city grows, to 2) get income, to 3) improve your city infrastructure (zoning, government buildings) to further grow your city. Notice here that there is essentially no fighting-monsters equivalent- 1) is an entirely passive experience that involve no agency on the part of the player. "Fighting" has essentially been replaced with "waiting." To grind is to wait. But, the use of the currency to level your character has now been greatly expanded and made more meaningful, and by itself creates a fantastic experience. So the gameplay experience is made meaningful with a meaningless grinding action, but a meaningful use of those grinding rewards. The system also recognizes this, and gives you a fast forward button to move the grinding faster when necessary and avoid meaningless gameplay. This is in essence the opposite of the original FF1 recipe, in which the grinding action is the focus, and the choice of rewards was something simply done in service of improving the grinding action.

Of course the issue with such a situation is that we can take it even further, into the realm of the utterly meaningless. Enter Farmville and its genre. Here's the reward loop: 1) crops grow to 2) reap crops to get money to 3) plant more crops, to cycle back through 1). This basically took the god-sim formula for grinding actions and the FFI formula for using grinding rewards and smashed them together. Or, they took the least meaningful element of both recipes and combined them into a product that was even more meaningless that the sum of the parts. By giving you no interesting actions to do with your rewards (there is an always obvious reward you want to go for), they put the emphasis back on the grinding action, but that action is... waiting. Wait, to get stuff, to wait more. Meaningless grinding at it's extreme. It's still offers you a reward that is intrinsic, and there is a very well-defined intrinsic loop here, but it's a meaningless intrinsic reward because it helps you do a meaningless action better. Of course such a meaningless action can get tiresome, so they offer an extrinsic input (i.e. pay money) to bypass the meaningless activity, but that only gets you rewards that help you do the meaningless activity better. And so this extrinsic input doesn't make the game experience better, it simply nets revenue.

Farmville reminds me of another reward engine: slot machines. See reward loop: 3) pull lever, to 1) watch wheels spin to 2) sometimes get money to 3) pull the slot machine lever more. The reward for doing an action (should you get the reward) is you get to do more of an action, but that action is not meaningful, and is never harder or more difficult to do. It's almost directly analogous to Farmville.

Which finally brings me to Fallout Shelter. After playing this game, I'm not quite sure where it fell, and I'm not sure thinking through any of the previous details made things any clearer. I really like grinding heavy RPGs and puzzle games, and also will happily get immersed in a good god-view sim. But I despite the Candy Crush, Farmville, and slot machine meaninglessness. Part of me is enthralled by Fallout Shelter, but part of me in disgusted at the game and myself for being enthralled.

So let's try to break down Fallout Shelter. This seems to be this basic intrinsic reward loop: 1) let your shelter grow to 2) gain caps to 3) buy more room and upgrade rooms to help your shelter grow faster. There's a relatively complex system by which these steps occur, as growing shelters can mean getting more dwellers or leveling up existing dwellers, and there's a "gain resources to keep people alive to get more resources" sub-loop in the shelter growth model. Plus a questing system that gives even more caps for practicing certain strategies. But at its basic level, it seems to follow the Farmville/Simcity model, with the core grinding action being "waiting" and most of the meaningful stuff in the upgrading phase. There is a system that needs to be kept in balance and is rather hard to maintain, and your actions help you maintain the system in balance better- in this way it is very similar to SimCity.

So the real question- is the grind in Fallout Shelter meaningful? There are some differences from SimCity that point to it being less meaningful. Most important, you can't fast-forward in time, but are forced to actually act out your waiting grind. Of course you can rush rooms to fast forward in a small way, but only so often and with increasingly negative consequences the more you do so. So fast forwarding is a mechanic, rather than something free, indicating that it is valuing waiting as a grinding mechanic rather than allowing you to easily bypass it. You can also offer extrinsic inputs to bypass the normal reward system, which certainly points toward a meaningless mechanic. The system is also grossly oversimplified compared to SimCity. Understanding the complexity is part of what makes interacting with SimCity rewards so interesting, and so the lack of complexity would seem to cause Fallout Shelter gameplay to lose meaning.

Fallout Shelter also doesn't have the meaningful elements that are present in other games. Like FFX, Fallout Shelter naturally increases difficulty as your shelter becomes bigger, meaning you don't really see your increase in power realized in a significant way. And it gives you very little control over how you encounter challenges, whereas that control was part of what gave The World Ends with You gameplay additional meaning.

But Fallout Shelter is also more meaningful than Farmville by comparison. First, the system is more complex, and your choices of how to use rewards most certainly has consequences and needs to be chosen well. Most prominently, there is a "loss" condition, and it takes strategic action to avoid that state, which is notably different from Farmville and slot machines.

What's clear is that Fallout Shelter is both more meaningful than Farmville, but less meaningful than SimCity and other games with meaningful intrinsic reward structures. But the real question is: where is the line between Farmville and SimCity, and what side does Fallout Shelter fall on? When does a game have a meaningful intrinsic reward system and grinding structure? When does a game devolve into the addictive-but-meaningless zone so clearly dominated by Farmville and Candy Crush? I'm guessing Fallout Shelter is only the first in a series of games intent on blurring this line further, and making it less clear what is a meaningful game and what is a slot-machine-esqe money-maker.