You can only optimize what you measure
One of the successful products we had at EmSense was an analysis tool for video game studios allowing them to measure the response to each element of their in-production games relative to their competition. It allowed teams to both identify/fix weaknesses and managers to predict in-market performance. EA, THQ, Activision and many other studios used it across more than 50 AAA titles including multiple Call Of Duty, Tony Hawk and many other franchises.How it worked
The game would be broken down into its top 20-30 repeated elements (Melee, cutscene, machine gun nest, driving, flying, etc and tagged to build an average measure of engagement across each. It would also be recorded level-by-level to identify the average most and least engaging elements. |
| An example of the engagement over time for 4 players in Gears of War with the average value aggregated at the bottom of the graph as they play through the level. |
Engagement was measured using a combination of heart rate, EEG, blinking, breathing and accelerometer readings which directly correlated with an intense experience. The system would end up with more than 1 million data points for each hour of gameplay across an audience and then aggregate them into an aligned data set which allowed extraction of the engagement and success of each element.
The key takeaway that studios found was that they didn't need to have the most intense response across the board. They needed to not have a key element which failed.
Balanced gameplay wins
An example is below. For the 5 FPS games, 3 of them have a very strong engagement during machine gun turrets (A part everyone loves). COD3 had a flat response due to the game developers only giving turrets to the enemy so when you shot them, you had no one to shoot. Resistance is the game that had problems. It actually had people disengaging when shooting turrets rather than engaging. The reason is that the designers only put machine guns inside safe concrete bunkers so it removed the adrenaline experience of being out in the open like in Halo. This was one of many weaknesses in Resistance which hurt its success. |
| Shooting a machine gun in Halo is one of the most engaging experiences because you are out in the open, scared and exhilarated. As a contrast, when players have to use a machine gun in Resistance: Fall of Man, it is one of the least engaging experiences due to being fully sheltered and save. |
We all love Halo energy swords
Within each game, getting the mix of engagement right massively influenced the outcome. Below is Halo 2. It shows the average change in engagement when driving, sniping, firing, having your shield down, throwing grenades, etc. What is amazing is that across all of these elements, engagement rises. This is not the case with most games. Usually driving or another element will result in a drop in engagement.The two most engaging elements of the game are melee combat and the energy sword. For anyone who has played the games, they probably still feel the fear and exhilaration of not knowing if you are going to win or die.
 |
| Halo's energy sword is the most engaging element of the game across the board. |
