Operant conditioning and Elite Dangerous

[Samosa]

Rather than get into a long TL;DR, I thought it might be interesting to kick off a discussion and see if anyone has interesting input.

Let's start with a Wikipedia page as a reference:

https://en.wikipedia.org/wiki/Operant_conditioning

So, looking at - for example - reputation decay, we have an example of negative reinforcement. Reputation decay is a penalty for not doing something. Negative reinforcement tries to get us to do something (log in and work on reputation) by stopping decay, which is an aversive stimulus.

Positive reinforcement is a reward, negative reinforcement is a penalty. Both can be effective - in fact, it's possible that negative reinforcement may be more effective than positive - but the difference between the two is fundamentally pain vs. pleasure.

Elite: Dangerous is a game. Should you be motivated by pleasure or pain? Something to consider.

Just some thoughts, not even getting into positive and negative punishment. Reputation decay is just one example. I'm sure one could think of many others. I merely mention this since it strikes me that the above is fundamental to the enjoyment of the game, and ignoring this is possibly behind a large number of criticisms we read every day on the forums.

OK, my coffee is wearing off. That'll do.

 

[Janichsan]

I think many of the complaints actually stem from ED applying too little operant conditioning.

Not few seem to expect the same instant gratification they can get from many other games, especially MMOs, with a constant stream of new shiny virtual rewards served by the minute for jumping through the game's hoops.

 

[Samosa]

I think many of the complaints actually stem from ED applying too little operant conditioning.

Not few seem to expect the same instant gratification they can get from many other games, especially MMOs, with a constant stream of new shiny virtual rewards served by the minute for jumping through the game's hoops.

Absolutely.

I appreciate that there are those who are happy with the current state of things and feel that their imagination is more than enough, but some would say examples of positive reinforcement are too few and far between while negative reinforcement is easier to find - it feels as though behaviour is generally ignored.

 

[Xakto]

Operant conditioning is very relevant in elite because the current reward system is entirely predictable and therefore not compelling over the long term. The most addictive, or shall we say compelling mechanic is random progressive rewards, a surprise each time a new box is opened, and this is something frontier has not taken advantage of at all. In fact Elite may be dangerous but it is rarely surprising. Go and undermine a system for a while (i did it for 4,000 merits) and it is a very unrewarding (apart from aforementioned merits), unsurprising, & undelightful grind. Good operant conditioning would vary the reward and provide some variation. This video from Extra Credits is a very good breakdown of the research by skinner who discovered operant conditioning. https://www.youtube.com/watch?v=tWtvrPTbQ_c

One of the most enjoyable moments I have had in elite so far was early in the game at the strong signal source gold/pirate escapade and it was a wonderfully crafted moment as immortalised in this comic https://fangs.ink/09.html - but that only worked once (well twice in my case but I am stubborn). I would love to see more surprise and delight brought to this game and the current procedural design is probably working against that activity. If there is one thing that will turn people off long term it is not just grind, but entirely predictable grind.

Good topic.

 

[Samosa]

Operant conditioning is very relevant in elite because the current reward system is entirely predictable and therefore not compelling over the long term. The most addictive, or shall we say compelling mechanic is random progressive rewards, a surprise each time a new box is opened, and this is something frontier has not taken advantage of at all.

This.

For those who are curious, google "intermittent reinforcement".

 

[Senex VII]

Full marks to those in this thread who are making me THINK. And great links by the way.

And although you've got my interest piqued, my designer juices running and my old teaching instincts shifting, my pragmatism kicks in and says, "What could I do about it in Elite? What difference could this small voice make?" I'm not asking you that you spoon-feed me an answer - I'm not sure there is an answer - but I wonder what you have decided to do, in addition to commentating and inspiring.

 

[Samosa]

Full marks to those in this thread who are making me THINK. And great links by the way.

And although you've got my interest piqued, my designer juices running and my old teaching instincts shifting, my pragmatism kicks in and says, "What could I do about it in Elite? What difference could this small voice make?" I'm not asking you that you spoon-feed me an answer - I'm not sure there is an answer - but I wonder what you have decided to do, in addition to commentating and inspiring.

My answer at the moment would be to talk about the situation and see if I receive reinforcement via response from FD. If not, then that will be my learning experience.

Just another thinking aloud moment:

A common thread seems to be "What's the endgame?".

Maybe the answer is:

https://en.wikipedia.org/wiki/Extinction_(psychology)

Thus, it's essential for continued gameplay that reinforcement continues. The often mentioned "Use your imagination!" can only last so long and may well be part of an extinction burst. Behavioural science teaches us that without reinforcement, we will decrease our involvement with the game.

The forum, however, is thoroughly reinforcing!

 

[Senex VII]

You are right about the forum!

But take a step backward for a moment. What's in it for Frontier? Their revenue stream is not directly based on keeping players in the game; it's based on attracting new players. That is their chosen business model, as far as we see it at the moment. To do that they don't need psychological depth, fascinating and instructive though it is; they only need to have a wow factor of increasingly broad and superficially attractive content, and they need to have just enough reputation from players and reviewers not to detract from that buyer's euphoria.

That can't last forever though. Eventually single-buy saturation occurs, and DLC marketing is reliant on an enthusiastic existing user base, or at least convincing a jaded user base that DLC can give the product new life.

Extinction... is it indicative that my proportion of in-game time to in-forum time has pretty much swapped over since v1.3? Or is that just ennui?

 

[Samosa]

You are right about the forum!

But take a step backward for a moment. What's in it for Frontier? Their revenue stream is not directly based on keeping players in the game; it's based on attracting new players. That is their chosen business model, as far as we see it at the moment. To do that they don't need psychological depth, fascinating and instructive though it is; they only need to have a wow factor of increasingly broad and superficially attractive content, and they need to have just enough reputation from players and reviewers not to detract from that buyer's euphoria.

That can't last forever though. Eventually single-buy saturation occurs, and DLC marketing is reliant on an enthusiastic existing user base, or at least convincing a jaded user base that DLC can give the product new life.

Extinction... is it indicative that my proportion of in-game time to in-forum time has pretty much swapped over since v1.3? Or is that just ennui?

That's just it - the "wow" factor sells the first few hundred thousand copies, but word of mouth will make or break ongoing sales. Thus, psychology is awfully important here. We want Elite Dangerous to be the one we marry, not just the one with the great figure who we bought a gin and tonic and took home for the night.

What's in it for Frontier is that they're in the unique position of having a community that cares. That we're talking about the game in this manner is a rather wonderful thing, and one would hope the opportunity wouldn't be wasted.

The opportunity to make the most of instrumental learning is massive, but like you, I'm finding that I'm spending much more time reading the forum than in-game - so something is amiss.

 

[Hellhawk666]

... The most addictive, or shall we say compelling mechanic is random progressive rewards, a surprise each time a new box is opened, and this is something frontier has not taken advantage of at all.

[With tongue in cheek] Perhaps they are developing superstitious behaviour in players by pretending there is a background simulation and structured reward/punishment system when it is actually all genuinely random ?

 

[Xakto]

Full marks to those in this thread who are making me THINK. And great links by the way.

And although you've got my interest piqued, my designer juices running and my old teaching instincts shifting, my pragmatism kicks in and says, "What could I do about it in Elite? What difference could this small voice make?" I'm not asking you that you spoon-feed me an answer - I'm not sure there is an answer - but I wonder what you have decided to do, in addition to commentating and inspiring.

Like Samosa said I hope that frontier devs and designers do spend some time reading the forums and having spent some time in the game business I would say they most definitely do but are generally not able to respond due to various corporate comms policies. Fdev being a public company they are probably even more about what gets said in public which means we don't get a natural dialogue.

On the topic of what to work on though, beyond the operant conditioning I would say immersion and opportunity for mastery are the two target rich environments. Immersion is talked about a lot but the cognitive dissonance abounds or in another thread "incongruence".

The original game focused on trading, bounty hunting, smuggling, piracy and these are all activities that do not appear to be valued or rewarded in the powerplay universe. Trucking documents is not trading, the documents are not commodities, there is no reward for shipping the documents 150LY or 10LY, shooting enemy ships gives a standard merit reward regardless of the ships skill or size and therefor bares little resemblance to bounty hunting. I think the opportunity for narrative or mission based routes into the powerplay universe would be a more natural way for players to become aware of and become introduced to the powerplay universe and would provide some opportunity for fun gameplay. It is also a bit of a disconnect that grinding merits also provides no influence or connection to rank within a major faction.

The other point, opportunity for mastery is really providing mechanics that test the skill of players and lead them to a point of flow which is widely considered one of the most enjoyable and deeply satisfying human experiences. The current super cruse mechanic for instance is a prime example of something that could have become a skilful activity but due to the auto pilot aspect of it misses that opportunity. People spend a lot of time in supercruse and clearly managing slowing down from faster than light travel is tricky, which is why I assume the auto slowdown feature became part of it. Imagine though if there wasn't an auto slowdown and your constant challenge was to figure out the fastest way to slow down to get to the station you wanted to dock at. Imagine that the gravity of planets provided some way to progressively slow down from faster than light by getting captured by their gravity. Slowing down from super cruse could itself become a game of skill that as became mastered became an immensely satisfying activity. I have already seen some youtube videos of people showing that it's quicker to circle a station when slowing down than coming straight in. Interdiction is probably another area that could become skilful, making it more or less effective depending on how you approach. Instant interdiction by matching speeds properly. I have experienced some level of flow when interdicting when you slide up behind a ship that's just leaving a planet and just tethering them before you fly by, but currently the opportunity for skill is limited.

Anyway, thanks for the random progressive rep reward, you are right the forums are compelling

[With tongue in cheek] Perhaps they are developing superstitious behaviour in players by pretending there is a background simulation and structured reward/punishment system when it is actually all genuinely random ?

Yes that would produce maybe some paranoid pilots Of course random progressive is not just random although I must say when I first got sucked into a neutron star in my new Asp it seemed extremely random

 

[Surly_Badger]

Many games nowadays are specifically designed with operant conditioning in mind. Farmville is a good example: it's designed to suck you in regularly and make you do the same kind of tasks (with a reward) at more or less the same time of day. It's very habit-forming and it's right out of B.F. Skinner's research on schedules of reinforcement from the 60s.*

The "You are not so smart" podcast has a pretty cool episode that touches - a bit - on how obsessive-compulsive tendencies in gamers can be weaponized to condition them to long-term gaming bouts: here

I used to work with a guy who was completely destroyed by Tetris. He was literally unable to stop thinking about it. He wouldnt' sleep and missed meals. He wouldn't sleep because he had nightmares of the blocks falling without him doing anything about it... One of my Warcraft raid team had similar obssession over her Garrison followers; she'd set the alarm to wake up at 3 am, and 5 am, briefly, so she could collect her followers' quests and reassign them. It turns out that the gratification/reward cycle is pretty easy to weaponize, really. As the folks who design casinos in Las Vegas!

(* Wow! I just used a bit of my psych degree!)

 

[Aewarin Jameson]

Good topic OP

Operant conditioning is very relevant in elite because the current reward system is entirely predictable and therefore not compelling over the long term. The most addictive, or shall we say compelling mechanic is random progressive rewards, a surprise each time a new box is opened, and this is something frontier has not taken advantage of at all.

Very accurate and something I have mentioned on the forums in the past, even as far back as in 2012 during the KS

Rather than rehash that post allow me to paraphrase:

The appeal of games like Diablo 3 is in part down to the randomness of the gear you find / craft. As you progress through the game your character slowly becomes more powerful (gains levels & skills) but at the same time you equip gear that has random stats - part of the fun then is to hunt for items with ever better stats. Whilst playing the game you also have a small chance to find a legendary item that can become a game changer for your character. Even now, 3 years later since release, when they drop there is still a little kid inside me that thinks "this could be the one item" that upon identifying has killer stats and makes me even more godlike. It's exciting to not know the reward.

How does this relate to ED ?

• Every ship is identical : same stats; performance; ability - the only thing that varies is the amount you pay for it. (CG's, PP or Founders)
• Every component is identical : An A6 shield performs like any other A6 shield; a C3 beam laser cuts through shields like any other C3 beam laser. (Admittedly PP has introduced some unique items so that's a start!)

When it comes to outfitting your ship and piloting it, credit where it's due they look very pretty and handle exceptionally nice, but they are in general quite bland. One A rated Cobra is just like any other A rated Cobra

A possible solution:

During the DDF Sandro (Dev) introduced a great idea to do with ships. The purchasing of a brand new ship was supposed to be a rare thing reserved only for the 1% whilst everyone else would buy 2nd hand ships - these battered shells would also have character : random things about them that could go wrong at a moments notice. (Think of that squeaky noise in your car that drives you insane; or the banging on the console in the Millenium Falcon to start the hyperdrive) The point being that these 2nd hand ships would have quirks - there would be a range / selection of them at the ship yards, varying prices, to the point that you could in theory pick up an Anaconda hull for under 10 mill but expect it to malfunction every few minutes or rip itself apart during hyperspace

I would also extend this for the components - introduce some variance into the stats and abilities so that the A6 shield you want to buy now comes in several flavours based upon the system you purchase it from; the manufacturer; even perhaps your reputation with the vendor comes into play. (New to town ? Expect to be offered the poorer items as "a sucker is born every minute" whereas if you've been with a system for a while now and are known to the vendor they might offer you a really "special" item for a fee ) Even significant things - what about for example a component that allows you to boost for 30-60 seconds but burns your fuel at 100x normal ?

Bring PG into play .. introduce some variance / randomness ... make us slightly unique (within a margin to ensure you don't trivialize the game)

Breathe some character into ED.

 

[CATPAIN KIRK]

You are right about the forum!

But take a step backward for a moment. What's in it for Frontier? Their revenue stream is not directly based on keeping players in the game; it's based on attracting new players. That is their chosen business model, as far as we see it at the moment. To do that they don't need psychological depth, fascinating and instructive though it is; they only need to have a wow factor of increasingly broad and superficially attractive content, and they need to have just enough reputation from players and reviewers not to detract from that buyer's euphoria.

That can't last forever though. Eventually single-buy saturation occurs, and DLC marketing is reliant on an enthusiastic existing user base, or at least convincing a jaded user base that DLC can give the product new life.

Extinction... is it indicative that my proportion of in-game time to in-forum time has pretty much swapped over since v1.3? Or is that just ennui?

Good points - and excatly summed up my worst fears when backing the game.

With a free expansion pass and no subscription for backers until PB, it would be in FDs best financial interests to disperse the original backers and purely develop for thus far unexploited markets which is what we've seen since release for the most part.

I.e. almost no DDA features added, adding in features noone ever asked for (CGs and CQC being the most prevalent examples) and resulting in getting rid of a recalcitrant part of the community not offering a lot of potential monetary gain whilst saving a little server load.

So yes - what we currently see is operant conditioning having taken full effect on FD.

 

[Xakto]

Good topic OP
The solution ?

During the DDF Sandro (Dev) introduced a great idea to do with ships. The purchasing of a brand new ship was supposed to be a rare thing reserved only for the 1% whilst everyone else would buy 2nd hand ships - these battered shells would also have character : random things about them that could go wrong at a moments notice. (Think of that squeaky noise in your car that drives you insane; or the banging on the console in the Millenium Falcon to start the hyperdrive) The point being that these 2nd hand ships would have quirks - there would be a range / selection of them at the ship yards, varying prices, to the point that you could in theory pick up an Anaconda hull for under 10 mill but expect it to malfunction every few minutes or rip itself apart during hyperspace

I would also extend this for the components - introduce some variance into the stats and abilities so that the A6 shield you want to buy now comes in several flavours based upon the system you purchase it from; the manufacturer; even perhaps your reputation with the vendor comes into play. (New to town ? Expect to be offered the poorer items as "a sucker is born every minute" whereas if you've been with a system for a while now and are known to the vendor they might offer you a really "special" item for a fee ) Even significant things - what about for example a component that allows you to boost for 30-60 seconds but burns your fuel at 100x normal ?

Bring PG into play .. introduce some variance / randomness ... make us slightly unique (within a margin to ensure you don't trivialize the game)

Breathe some character into ED.

That sounds wonderfully quirky, I like. Many rpg's have the concept of wear and tear, you use a sword a lot and eventually it breaks unless you have buffed your repair skill. I do quite like the idea of cmdr's developing tinkering skills, mechanic, weapons, drives etc. giving them the ability to buy beat up used components but through some crafting/reparing mechanic and even, lets say the addition of some rare minerals (another reason to mine maybe) are able to bring some of these things back to life. Would give us some more things to do in the dock and would really connect people to their ships. I have heard a lot about the DDF and it sounds like some amazing stuff was discussed.

 

[Surly_Badger]

Well, from a standpoint of conditioning people to care about in-game stuff, being able to name, customize, and personalize is really important. So far, the folks at FD have done a remarkably bad job of that. Discoverers' names are displayed on systems and you can color your ship with about a dozen options. That's not designed to bind people to their in-game identity, at all.

Most game designers get this, which is why they have characters you can assemble from parts (like my avatar, which I D/l'd EVE just to create) and the ability to name or visually mod stuff. Blizzard's got their share of flaws, for sure, but they appear to really "get it" on that score in WoW: you can create your character, transmog their gear and actually care about and be proud of how "you" look, name your pets, etc. And people take that stuff seriously: look how many people will grind for a certain transmog item! (I ought to know, I ran Black Temple every day for 2 months to get my Bulwark of Azzinoth) Sure there are always issues to worry about, like people having ships with dumb names but... we already have that (it's just a question of how dumb is tolerable and whether there's a "report" button that actually works) again, WoW has this all figured out fairly well. Not perfectly, but pretty well.

Edit: To the other posters' comment about crafting - the crafting mini-game and marketplace are brilliantly executed in WoW except that the boneheads at Blizzard don't understand Economics 101 and created a marketplace where some things rapidly approach zero value and other things get massively inflated. CCP has done a better job of that with EVE. But from a standpoint of getting players to feel tied to the game and want to log in periodically to do stuff, EVE and WoW are lightyears ahead of E/D. I hope it's just a matter of time before FD figures this out and I see a lot of what's been put into PowerPlay as an attempt to add game features that encourage long-term emotional commitment.

 

[Roybe]

Interesting subject and one of the differences that is obvious between Theme Park MMO's and Open World MMO's. Some call it hand holding...but what it boils down to is how much operant conditioning a game offers its player base. A lot of discussions when dealing with this come down to Risk/Reward...the possibility of loss is outweighed by the ability to 'beat' the game. Some games use RNG boxes to supplement real operant conditioning...this is one of the reasons I have taken a break from MMO's currently.

RNG in general is not a reward based on skill, it's a reward for tenacity. If you log in often enough and kill enough, you will feel you have a better expected chance for a positive outcome....which is a falsehood...you just have more chances for a positive outcome..the odds do not change from box to box.

Anyway, I digress.

My ultimate point in this is I do not want to see this type of game play creep into the design of this game. The RNG in this game was punishing enough just with the USS's....and still is if you try to do the high dollar pirating missions...you know the ones with the day long timers worth 100'sK credits. Any other amount of RNG is unnecessary and should not be incorporated.

I play this game not for the content that is provided...but the content I create myself...and the content I create with those I play with. This type of reward is intrinsic, rather than extrinsic, and not for everyone's taste.

The designers of this game knew they would have a niche game from the start. So did the backers. The idea for the game was to create a niche game that would not be made by a publisher. We have that. The player base wants more extrinsic game play options...but when given that, they turn their noses up....I'm looking at you Power Play. <shrug> I've said this before...this community is schizophrenic. It's at odds with the design of the game constantly. Fdev keeps holding to their design ideas....and the community keeps bucking against it.

In the end...FDev will have a choice. To give the community 'what it wants' and watch their game die...or watch as the 'community leaves' and do the same. However, I am expecting neither of these outcomes...I think the game will remain true to the ideas set forth by the devs and will be successful.

 

[Berty Dastard]

Xakto has positively reinforced my avoidance of reading posts which are difficult to read (for me most of his post the text is light grey - maybe this is only because I'm using the forum default skin (is there another?))

 

[Yaffle]

There are several. Try the dropdown at the bottom left.

 

[Aewarin Jameson]

That sounds wonderfully quirky, I like. Many rpg's have the concept of wear and tear, you use a sword a lot and eventually it breaks unless you have buffed your repair skill. I do quite like the idea of cmdr's developing tinkering skills, mechanic, weapons, drives etc. giving them the ability to buy beat up used components but through some crafting/reparing mechanic and even, lets say the addition of some rare minerals (another reason to mine maybe) are able to bring some of these things back to life. Would give us some more things to do in the dock and would really connect people to their ships. I have heard a lot about the DDF and it sounds like some amazing stuff was discussed.

Another fine example of a game that used W&T effectively was Dark Age of Camelot. All items had 2 stats : condition and durability - as you used the item the condition reduced and the effectiveness of the item lowered. When you repaired the item the condition went back to new (100%) but the durability dropped - once that reached 0% the item effectively broke and had to be replaced. Combine the W&T with the fact that you could buy items from the vendors (worst quality overall); find them in dungeons / monsters (medium quality overall) or have a player make them for you (best quality overall) it ensured a thriving player economy that kept players coming back.

In a similar way I would love for FD to introduce wear and tear on items - remember right now in ED that at a certain condition items have a chance to fail, so combine that with unique / randomised items (for the character aspect) and a W&T concept (so that components fail quicker the older / more work out it is) and you have a living, breathing ship rather than the clinical sterile one we have today (still looks damn nice though )