Skip to content

Getting Started with AR/VR

1. Courses

A starting point compiled from Coursera, YouTube, Unity, and other sources:

  • Unity XR: How to Build AR and VR Apps Specialization — a 3-part tutorial series for understanding AR/VR.
  • Extended Reality for Everybody Specialization — another 3-part tutorial series.
  • Introduction to Augmented Reality and ARCore — a good starting tutorial for learning AR, covering some of the main problems in AR to tackle during project implementation.

2. Unity

A recommended starting tutorial for learning Unity (straightforward to follow).

Using Unity to Implement VR Projects

A YouTube channel that implements a VR game from scratch, worth watching even without a VR headset and controllers to implement it yourself.

Animation Rigging in Unity

Instructions covering: how to import a character into Unity, how to download the Animation Rigging package, how to rig the imported character, and a short description of animation in Unity. An accompanying Animation Rigging.pdf was attached on the source page; re-attach it once migrated.

3. Setting Up the Meta Quest Pro for VR Development

A link to set up the Meta Quest Pro headset with Unity: https://developer.oculus.com/documentation/unity/unity-tutorial-hello-vr/

Unity VR Setup

The following outlines how to configure a Unity project to run VR apps for the lab headsets (Meta Quest Pro and 3, at time of writing). Meta provides helpful documentation (https://developers.meta.com/horizon/documentation/unity/unity-tutorial-hello-vr/) on configuring a project in Unity; be sure to complete the prerequisite steps listed there. Summary of steps:

  1. Complete hardware and software prerequisites: https://developers.meta.com/horizon/documentation/unity/unity-before-you-begin
  2. Configure Unity for XR Development: https://developers.meta.com/horizon/documentation/unity/unity-project-setup
  3. Set up the device (currently connected over USB, so follow the appropriate sections): https://developers.meta.com/horizon/documentation/unity/unity-env-device-setup
  4. Explore Meta SDK samples under Assets/Samples/Meta XR Core SDK in the Project tab, which should be available after the previous steps; these examples have a configured XR camera rig that serves as the base for VR development in Unity.

A good resource for developing interactions with the Meta SDK: https://www.youtube.com/watch?v=4kGD8q5kEx8

Simultaneous Hand and Controller Tracking (Multimodal Input)

Within the first half of 2024, Meta released an XR SDK update enabling simultaneous tracking of the user's hands and controllers, called Multimodal Input (documentation: https://developers.meta.com/horizon/documentation/unity/unity-multimodal/). The following adjustments to that documentation, based on troubleshooting, were found to work:

  1. Install Meta XR Core SDK Package V59 or higher (already downloaded if you completed the Unity VR Setup section above).
  2. In the Project tab of your 3D Unity project, search for OVRCameraRig. Drag this prefab into the project's hierarchy, and delete the main camera instance initially created with the project.
  3. Click on OVRCameraRig and find the OVRManager section. Check "Simultaneous Hands/Controller Support". Under "Quest Features" below, set Hand Tracking Support to "Controllers and Hands", set Hand Tracking Frequency to HIGH, and set Body Tracking Support to "None". Below Quest Features, check the box under "Launch simultaneous hands and controllers".
  4. Under OVRCameraRig/TrackingSpace, search for and add an instance of OVRControllerPrefab (Project tab) to both LeftHandAnchorDetached and RightHandAnchorDetached, nested under the respective anchor game objects. For both, set the Controller to "Touch" and Show State to "Controller Not In Hand" in the Inspector.
  5. Drag one OVRControllerPrefab to both LeftControllerInHandAnchor and RightControllerInHandAnchor (nested under the respective hand anchor game objects). Set the Controller to "Touch" and Show State to "Controller In Hand".

These steps deviate from the Meta documentation's steps 13-16, since otherwise hand visualization is restricted to when the user is holding the controller; following these steps instead visualizes both the controllers and hands regardless of whether the user is holding them.

  1. Drag one OVRHandPrefab to both LeftHandAnchor and RightHandAnchor. Set the Hand Type to the corresponding anchor the prefab is nested under, and Show State to "Controller Not In Hand".

Accessing controller tracking data and hand tracking data is demonstrated in the Handover_V1 Unity project.

Migration note: the source page references several attached files (Animation Rigging.pdf, the Handover_V1 Unity project) that should be re-attached or linked to their current location (e.g., a GitLab repository) once this page is migrated.