VR-labs on smart devices for education 

A Mobile AR Extension to the Virtual Physics Laboratory to enhance the VR Learning Experience

Check out our IP5 project, below you will find the official documentation as well as a demo video of the build application and some screenshots from both the VR headset and the AR viewer.

Summary

In this IP5 project, we designed a mobile augmented reality (AR) extension for the established "Virtual Physics Laboratory" (VRLab). For experiments, only one user requires a virtual reality (VR) headset, while additional users can join through mobile AR on Android devices. This extension enables users to connect, observe, and interact with the VR environment via their mobile phones, reducing the reliance on costly VR headsets significantly. The development process is structured into three key phases. Firstly, we created mock VR and AR applications, allowing users to join virtual rooms and observe VR players. Secondly, dynamic loading of game objects, such as building models, was implemented. Thirdly, we researched, implemented, and tested various interaction concepts between AR and VR players. To assess the AR application's effectiveness, we conducted a user study comprising a task-based workshop followed by a questionnaire.

Keywords

Unity Engine/Unity 3D, OpenXR, ARCore, VR, AR, XR, C#, Photon Engine/Photon Unity Networking (PUN), Human-computer interaction paradigms (HCI), User Experience and Usability

Goal

The goal of this project was to extend the current virtual laboratory that runs on VR headsets by a mobile application, where mobile users can join a VR user to observe and interact with each other. To achieve this collaboration, the VR world will be visualized for the mobile user in AR and interaction concepts must be found. By the development of such an AR-extension, the use of expensive VR headsets can be minimized.

Initial Position

The existing application "VRLab" is a VR application that allows to observe and modify a setup of solar power cells placed on a virtual building. The setup is simulated on a server hosted by the customer. As current hardware serves the Oculus Rift S, due to it being an affordable and already available solution. The Project was originally created on Godot but is currently being ported to Unity Engine. VRLabs uses OpenXR as the XR Interface for VR. The building and solar panel placement is currently done on the Solextron Web page. The setup is then saved as a JSON file, which can be transmitted to the VR environment to be viewed. Interactions in VR include only changing the panel type so far. Tilting, moving, adding, or removing panels as well as modifying the building are not yet supported interactions but will be implemented in the future. The porting process and the implementation of additional VR interactions are not within the scope of this project. The use-case of the application is within education. The teacher presents the experiments by using the VR headset and displaying them on a bigger display to view or hands out headsets to have the students test the experiments by themselves. The problem is that VR headsets are expensive. Therefore, a solution has to be developed to get an inexpensive way to allow students to view and participate in the experiments without needing a headset.

Results

The AR-extension "AR-Viewer" allows mobile users to join, observe and collaborate with a VR user in the same game world. The VR user has the teacher role and has therefore more rights than the AR viewer, for example to open and close a game room. Within a game room, the VR user can load and configure the project building and its solar panels, while the AR user can observe by moving around in the game world - this can either be done physically in the real world by walking around the marker, or by rotating and scaling the VR space. The AR user can interact with the VR host by using a feature called "Tap-Ping", which allows to place a pin anywhere on the building. Further and for better sense of having other users around you, both the VR and the AR users are visible.

Demo