This recent interview with Gabe Newell of Valve caught our interest because it’s so rare that a game developer talks publicly about the potential of physiological computing to enhance the experience of gamers.  The idea of using live physiological data feeds in order to adapt computer games and enhance game play was first floated by Kiel  in these papers way back in 2003 and 2005.  Like Kiel, in my writings on this topic (Fairclough,   2007; 2008 – see publications here), I focused exclusively on two problems: (1) how to represent the state of the player, and (2) what could the software do with this representation of the player state.  In other words, how can live physiological monitoring of the player state inform real-time software adaptation?  For example, to make the game harder or to increase the music or to offer help (a set of strategies that Kiel summarised in three categories, challenge me/assist me/emote me)- but to make these adjustments in real time in order to enhance game play.

One downside of real-time software adaptation is that it’s a lot of extra work for the game developers.  Rather than designing a linear system where the game goes from easy to hard in a predictable sequence, game software must be created that is essentially modular.  So, the game software comes with an expanded repertoire of possible responses (challenge/assist/emote) that can be deployed at any point in the game narrative.  Speaking as a researcher, the idea of adaptive software that provides timely and intuitive interventions to enhance game play is a fascinating problem – but I can understand the reticence of those in industry to adopt this model.

According to the interview, Newell explains that Valve have been experimenting with gaze tracking, skin conductance measures and heart rate in order to represent the player state (how the player is feeling).  No surprises there, this type of research is very much in the tradition of affective computing literature.  However, he points out another aspect that came as a surprise to Valve and perhaps less of a surprise to us , although we haven’t explicitly addressed it in our own work.  Here’s the quote in full:

“And then there’s some surprising side-effects that we didn’t expect, like what happens when you expose that information in a social gaming context. It surprises us that how much value there is to the people who are playing. So if you’re in a competitive situation, and you see somebody’s heart rate go up, it’s way more rewarding than we would have thought. And if you see somebody in a co-op game who’s sweating, people tend to respond to that way more than we would have thought.”

The surprise for me personally is that I always thought of physiological computing in a gaming context as one way to enhance human-computer interaction.  But of course the same data can be used to augment human-human communication in the context of social gaming.  This kind of shared biofeedback has the potential to provide an interesting addition to certain types of collaborative games where the players need to be aware of the performance capability of their team-mates.  For example, if Fred’s heart rate is extremely high then we might think that he is struggling to cope with game demands, and as his team mate, we might want to help him out.  For competitive games, there is a different dynamic – in this case, high levels of stress probably indicate that we will be the victors shortly if we continue to do whatever we’re doing.

The challenge for the game developers is not only how to measure player state in a way that is psychologically meaningful (i.e. has a connection with subjective feelings and is predictive of performance) but also how to represent that in a simple, intuitive but sensitive way to the other gamers.  Computer games are fun because they demand a high level of attention and players’ ability to assimilate supplementary information is very limited.  Of course if the interface is designed in the correct way, the player will receive dynamic yet digestible updates of the states of their colleagues or competitors.  For example, our own use of colour coding for this web site that is linked to a real-time heart rate feed is one instance of simple representation of activation state.

I also wonder about if receiving feedback of one’s own state during game play would help or hinder performance?