Recent posts on the blog have concerned the topic of psychophysiology (or biometrics) and the evaluation of player experience.  Based on those posts and the comments that followed, I decided to do a thought experiment.

Imagine that I work for a big software house who want to sell as many games as possible and ensure that their product (which costs on average $3-5 million to develop per platform) is as good as it possibly can be – and one of the suits from upstairs calls and asks me “how should we be using biometrics as part of our user experience evaluation?  The equipment is expensive, its labour-intensive to analyse and nobody seems to understand what the data means.”  (This sentiment is not exaggerated, I once presented a set of fairly ambiguous psychophysiological data to a fellow researcher who nodded purposefully and said “So the physiology stuff is voodoo.”)

Here’s a list of 10 things I would push for by way of a response.

1. Make the point that psychophysiological data provides continuous monitoring of behaviour that delivers quantitative data without the need to interrupt the player.  This form of measurement is superior to subjective methods either in real-time (no need to break from experience) or on a retrospective basis (no memory bias).  It is more sensitive than monitoring gaming performance (because good performance can be achieved at low or high levels of psychophysiological activity) and besides the risk of intrusiveness due to actually wearing sensors, the approach makes no other demands on the player.
2. Educate the company about what these measures do.  It is not a literal quantification of an emotion or a thought or a feeling.  It is not an “inside-the-head-oscope”.  It is the electrical activity of human nervous system recorded from the heart, lungs, skin, eyes, muscles and the brain.  To begin with, how do we expect our best games to make the player feel?  Excited? Engrossed? Exhilarated? Challenged?  Let’s think about those psychological states in experiential terms – now translate them into the electrical language of the human nervous system with some physiological know-how.  For example, excitement = increased sympathetic activation = higher heart rate, blood pressure, faster/shallow breathing.  Too often people work in the opposite direction and wind up with one-to-many inferences that don’t stand up to scrutiny, e.g. increased heart rate = excitement/frustration/anxiety/happiness.
3. Make an argument that we focus on some particular aspect of the player experience in order to focus our selection of measures accordingly.  This means using the psychophysiological measures at our disposal in a strategic way – if we are interested in emotional experience, we focus on level of activation/positive affect experienced during game play.  If we want to measure persistence, i.e. what stops a player from tossing the handset aside due to repeated experience of failure, we may look at physiological measures of motivation.  If we want to capture the level of mental workload during the training/familiarisation phase, we would look at measures of cognition.  I would basically argue for hypothesis-led approach to capturing player experience.  I feel that psychophysiology is particularly vulnerable to misinterpretation when tasked with capturing ‘experience’ or ‘flow states’ or similar phenomena that are poorly defined.
4. Suggest that we could have naive participants play our best games, the ones that sold millions, in order to understand the linkage/convergence between player experience and the corresponding psychophysiological signature or pattern of reactivity that would be our ideal.
5. Remind them that the human nervous system responds to stimuli in the external world, it also reacts to internal cognitions, changes in motivation and emotional states.  The latter can occur in response to external event or result from pure introspection.  It also responds to movement, eating/drinking, changes in posture, fever, pain, noise, changes in room temperature, a wide range of chemicals etc. etc.  The human nervous system is a very sensitive instrument.
6. Add the further point that psychophysiological activity does not discriminate between physical activity and psychological events.  If we record a pattern of activity during a game or gaming event, we must recognise that there are at least three components involved: (i) effects of physical activity (movement) and environment (noise, room temperature), (ii) an emotional component, and (iii) cognitive activity and motivational context of behaviour (see Introduction from this paper for detailed analysis).
7. Based on points (5) and (6), argue that we have a signal-to-noise issue if we want to use psychophysiological measures to gain insight into player experience.  Therefore we have to either (i) enforce rigid experimental control over testing (control environment, player movement and discourage verbalisation) or (ii) record movement, room temperature, talking and other potential confounds throughout testing so we can remove their influence at a later point.
8. Carefully monitor and control player experience and skill level prior to testing.  Psychophysiological reactivity is very sensitive to novelty and habituation. The physiology of gamers will probably vary according to the gaming situation and their level of experience with that situation. It makes sense to test gamers according to experience/exposure for purposes of recruitment and data analysis.
9. Carefully control the gaming environment if we are comparing prototypes.  Use the gaming engine to systematically manipulate the environment in order to obtain an unambiguous psychophysiological response.  Do not change too many variables at the same time – this tends to make data interpretation difficult if we have lots of interacting variables in the game world.
10. Use psychophysiology in conjunction with measures of performance and behaviour – physiological measures are very sensitive and markers of behaviour can provide context.  Investigate the relationship between overt markers of behaviour and covert measures of psychophysiology.  Use subjective self-report measures pre-test or post-test only, never simultaneously with psychophysiology – the act of verbalising during a test will wash over any psychophysiological response.