Measuring user movement in Mixed Reality apps

Intro

For a business HoloLens app I am currently developing - as well as for my app Walk the World - I needed a way to see if a user is moving or rotating in excess of a certain speed, to make certain control elements that are floating in view disappear when he/she is on the move - and come back when the movement stops. How this is used in detail I will describe later, but first I want to describe an easy helper behaviour to sample and measure speed, movement and rotation.

The actual tracker

using HoloToolkit.Unity;
using UnityEngine;

namespace HoloToolkitExtensions.Utilities
{
    public class CameraMovementTracker : Singleton<CameraMovementTracker>
    {
        [SerializeField]
        private readonly float _sampleTime = 1.0f;
        
        private Vector3 _lastSampleLocation;
        private Quaternion _lastSampleRotation;
        private float _lastSampleTime;

        public float Speed { get; private set; }
        public float RotationDelta { get; private set; }
        public float Distance { get; private set; }

        void Start()
        {
            _lastSampleTime = Time.time;
            _lastSampleLocation = CameraCache.Main.transform.position;
            _lastSampleRotation = CameraCache.Main.transform.rotation;
        }
   }
}

The behaviour is implemented as a Singleton. Although that is not strictly necessary, it makes sense to do so as there can also be only one Mixed Reality Camera and there is only one user. There is only one public property - that is the sample time. The idea of a sample time is simple - if you want to measure speed, or rotation, or movement - you have to do so over time. Default it samples location and rotation every second, and then it's up to you to decide to do something with it. At the start, it simply sets the sample time at now, the first sample location to the camera's current location and the rotation to it's current rotation.

In the update method (called every 60th of a second) we simply check whether the sample time period has expired, and then we get a new sample of location and rotation

void Update()
{
    if (Time.time - _lastSampleTime > _sampleTime)
    {
        Speed = CalculateSpeed();
        RotationDelta = CalculateRotation();
        Distance = CalculateDistanceCovered();
        _lastSampleTime = Time.time;
        _lastSampleLocation = CameraCache.Main.transform.position;
        _lastSampleRotation = CameraCache.Main.transform.rotation;
    }
}

The calculations itself are rather simple:

private float CalculateDistanceCovered()
{
    return Vector3.Distance(_lastSampleLocation, CameraCache.Main.transform.position);
}

private float CalculateSpeed()
{
    // return speed in km/h
    return CalculateDistanceCovered() / (Time.time - _lastSampleTime) * 3.6f;
}

private float CalculateRotation()
{
    return Mathf.Abs(Quaternion.Angle(_lastSampleRotation, 
                                      CameraCache.Main.transform.rotation));
}

The distance is simply the difference between the previous and the current Camera position. Time.time is always in seconds since the app started, so dividing the speed through the elapsed time results in the speed in meters per second. Multiplying it by 3.6 makes that km/h - I presumed that to be a unit most people have a feeling for. Feel free to adapt this to your needs and have it return miles, yards, feet, furlongs, stadia or your outdated/obscure distance unit of choice ;).

So what is this good for?

Well, simply put - to take action when some threshold for rotation or movement is crossed.Like I mentioned, it's particularly useful for determining if control elements that should be more or less in the user's field of view should be moved - but not too often or too brusque, or else it is not possible to properly view or interact with them. In the demo project I have created a little demo behaviour that shows speed, distance covered and rotation in a floating text, and it also uses that data to decide whether or not it's time to move the text back into view.

This is a picture of the text just after it moving back in view. It will rapidly go back to showing all zeroes as it only measures these values over the last second.

The demo behaviour in a bit more detail:

using HoloToolkitExtensions.Utilities;
using UnityEngine;

public class ShowCameraActions : MonoBehaviour
{
    private TextMesh _mesh;

    [SerializeField]
    private float _rotationThreshold = 10f;

    [SerializeField]
    private float _moveTreshold = 0.4f;

    [SerializeField]
    private float _moveTime = 0.2f;

    private bool _isBusy;

    void Start()
    {
        _mesh = GetComponentInChildren<TextMesh>();
        MoveText();
    }

    void Update()
    {
        SetText();
        if ((CameraMovementTracker.Instance.RotationDelta > _rotationThreshold ||
            CameraMovementTracker.Instance.Distance > _moveTreshold ) && !_isBusy)
        {
            MoveText();
        }
    }

    private void MoveText()
    {
        _isBusy = true;
        LeanTween.move(gameObject, 
                        LookingDirectionHelpers.CalculatePositionDeadAhead(), _moveTime).
                  setEaseInOutSine().setOnComplete(() => _isBusy = false);
    }

    private void SetText()
    {
        var text = 
            string.Format(
             "Speed: {0:00.00} km/h - Rotation: {1:000.0}° - Moved {2:00.0}m",
            CameraMovementTracker.Instance.Speed,
            CameraMovementTracker.Instance.RotationDelta,
            CameraMovementTracker.Instance.Distance);
        if (_mesh.text != text)
        {
            _mesh.text = text;
            Debug.Log(text);
        }
    }
}

Long story short:

• The text will be updated in every call to Update (which is 60 times per second) but since the CameraMovementTracker updates itself only once a second by default you should see the text change only once a second. I have also included a Debug.Log so you can see the numbers change when the text is still outside of your view. This of course only works in the Unity editor.
• If the rotation threshold (10 degrees) or movement threshold (0.4 meters) is exceeded, the behaviour will attempt to move the text back into view (if it is not already doing so), using good old LeanTween. The "setEaseInOutSine" will make the movement start and stop fluently.

Conclusion

It's not hard to measure these things and the code is not complicated, but as is my custom - if I need to make something the 3rd time, it's time to make it into a generalized reusable class. And there you have it. Have fun with the demo project.