Frenzies: Auto body correction

This was an interesting gameplay issue I encountered during my time at Near Light which tested my skills with 3D spacial maths, so I thought I'd discuss it here.

•  The issue

So Frenzies, as a multiplayer shooter, needs clear targets for people to shoot. But as a VR game, the game also needs to respect (to some extent) the movement of your head. If you try to move your head forwards, and it doesn't move in VR, it feels really bad (trust me on this), plus Meta doesn't let you put your game on their store. So player's HMD (head mounted display, where in game their head is) must have unconstrained movement, but also we need to make sure their body must follow this. If we didn't have the body follow the head, players would move to a corner, then just take a step round it in the real world, leaving their body safely behind cover but letting them see round - not what we wanted.

To give a bit more basis from a technical perspective, the issue here is that there are two points that can move. One is what we called the 'ground' - basically the base of the character, and the other is the HMD. The ground moves as the player moves in game with the control stick (also the dash), whereas the HMD moves with the player's physical head. The character's skeleton uses the ground as a base point, so the IK can lean the player as the HMD moves around it - until the body can't physically stretch far enough, and then we have a problem.

• The solution at launch

When Frenzies released, we had a pretty simple solution in place for this. If the player moved too far away from their ground (or put their head into a wall), they would see their display fade to black as they got too far away, before after a certain distance they wouldn't be able to see anything until they brought their head back to where it was before (or recalibrated). Simple, but we thought it worked well enough.

But... the players disagreed. It became clear that unlike in development where we mostly played seated, players would often play standing, and not stay on the spot either, but step from side to side. We never supported room-scale movement (full moving around a room while playing), but it seemed this was occurring even without players moving around too much. We initially tweaked some of the values to account for this, but that alone was not enough, as either it didn't black you out when it should or blacked you out too harshly. Another fix was needed.

• The second problem hiding behind the first

Potentially you might be thinking that the solution is to just move the body under the head, and you'd be right. I made a system to do this, taking into account things like giving leeway before moving so it feels natural, what happens if the mesh is blocked and so on. This worked pretty well... until you turned your head. It turned out this work had highlighted an issue we already had. Soon after I'd joined, I'd worked on a feature to rotate the mesh with the player's head rotation, so the body will follow the direction the player is facing, complete with all the things you'd need to make that comfortable (thresholds, dead zones, locks on rotating if you're looking almost straight up and down and so on. I'd spotted an issue which I hadn't been given permission to fix at the time, which was a slight movement when you turned your head, rather than just the expected rotation. After moving in real life, this movement was no longer slight, but both very noticeable and uncomfortable.

After looking into this, I discovered the cause. I'll try to explain it as simply as I can:

1. The head is attached to the body
2. When auto rotating the body, we therefore have to rotate the head by the same amount, but in the opposite direction. This keeps you looking in the same direction
3. However... The body and the head rotate around different points. The body rotates the pawn (and therefore the head), whereas rotating the HMD rotates around the ground node.
4. If the ground node is at the centre of the body, then we're all good. 

However, it wasn't. In the normal case it wasn't, because the head was offset forwards from the body. This was perfectly expected, as human heads sit forwards slightly, which is matched in-game by the IK, but it was this offset that was causing the slight movement I'd seen initially.  In the case of players moving around, their mesh was being auto-corrected, but the ground wasn't, and shouldn't be. The ground effectively marks the real world location of the headset at the last calibration, which obviously shouldn't be moved around by things happening in game. What was happening is the two rotations, around the mesh and the ground, were rotating around different points, causing a net movement.

So, the fix: Calculate the net movement, and offset the mesh by that much. That way, the HMD does not move, the model correctly rotates, and the player doesn't feel uncomfortable by unexpected movement. Easier said than done. Why? Well, I eventually realised, applying the yaw input for the body rotation also changes the position of the head and the ground node, so the updated positions needed to be calculated and used to find the net movement. After squaring away the vector maths, I got it working; smooth auto rotations, meaning we could have smooth body following, meaning players could move however they wanted without separating from their mesh.

To solve this problem, I'd used a number of methods:

• Classic debug prints. While simple, having exact values printed to the terminal allowed me to perform calculations on them to work out if they were what I expected. I could log where I predicted things to end up, then print their actual location next frame to check.
• The Unreal visual logger. This allowed me to do something in VR after tracking certain points, then play back the recording. This gave me a visual reference for how points such as the ground were moving.
• Reductive methods. Rather than trying to fix the whole calculation at once, splitting it into just the body rotation or just the head rotation allowed me confirmation of when I was correct and when I wasn't by checking the logs, which proved incredibly useful. One downside though, just for this specific case... having incorrect rotations/translations applied to me in VR made me feel very sick... 😅