Virtual and Augmented Reality

Since the origin of the center, CAVS has been working with virtual reality. Our earliest research efforts investigated driving simulators using monitors, projectors, and head-mounted displays. Integrations with our motion capture systems provided low-latency head tracking on monochrome and low-resolution displays. Today, CAVS uses state-of-the-art low-cost commercial virtual reality head-mounted displays to study human behavior in the laboratory in situations that are difficult, dangerous, or expensive to study in the real world.

Education and Workforce Training

Virtual reality is proving to be a powerful tool for education and workforce training. Immersive, episodic virtual reality (VR) training, coupled with traditional training methods increases cognitive recall, improves training outcomes and allows users to obtain experiences which may not be without production risks through any other formal experience.

Virtual reality allows students to safely gain experience interacting with and using tools to perform tasks that may be too dangerous or too expensive to allow them to do in the classroom. For example, CAVS, in partnership with the USDA and industry, is building a prototype virtual greenhouse that includes a simulated environmental control system. In the classroom, there are not enough real greenhouses to provide every student with the opportunity to use an environmental control system to manage the greenhouse environment. Also, real greenhouses are home to living plants that provide resources for teaching, samples for research, and revenue for student organizations. If a student were to inadvertently damage the plants through an accidental mismanagement of the environment, the implications could be significant for the student and many others. In virtual reality, students can experiment with systems, experience both success and failure, and learn from their virtual mistakes with no chance of embarrassment or loss of resources.

VR Highlight
guy in VR headset

Related to workforce training, CAVS has developed virtual reality training tools that places workers in risky working conditions and teaches them to identify and resolve safety concerns. The scenarios presented in these tools can be relatively benign, such as a stack of boxes precariously positioned on a shelf, or very dangerous, a fire at a workstation. By using virtual reality, workers can gain experience in any unsafe condition and learn the proper response. In virtual reality, the employer can score the workers response and the system can provide real-time instruction and feedback.

Pedestrian Research

girl in pedestrian VR

Since 2016, CAVS researchers have used virtual reality technology to study how pedestrians will interact with autonomous vehicles in the future. As the number of self-driving vehicles increases, pedestrians will need to be able to safely interact with the vehicles when they shared the roadway. Building and testing the many possible methods for communicating between vehicles and pedestrians would be expensive and dangerous. Instead, CAVS researchers can rapidly build and test many different communication methods on many different vehicles in virtual reality. Then, participants come to CAVS where they wear a head-mounted display, usually an HTC Vive Pro or an Oculus Quest, and walk across streets filled with vehicles with CAVS-designed interfaces. CAVS researchers record the participant’s position, walking speed, the direction they’re looking, walking speed and any hesitations and use that information to interpret how participant’s respond to different communication systems. In virtual reality, participants can cross virtual streets with simulated traffic hundreds of times giving CAVS researchers an enormous amount of data about how pedestrians will interact with autonomous vehicles.

Driving Research

The CAVS driving simulator is a full Nissan Maxima cab on a six-degrees-of-freedom, electrically-driven motion base. Three projection screens in front of the vehicle provide the driver with almost 180-degree field of view. Two LED screens and a projection screen behind the vehicle allow the driver to use the side and rear mirrors to see around the vehicle. A surround sound system in the vehicle cabin provides the sounds of the road and the vehicle’s engine. Force feedback on the steering wheel helps to recreate the feeling of driving the vehicle.

Nissan Simulator
CAVS Driving Sim

The driving simulator allows CAVS researchers to study driver performance and behavior in scenarios that would be difficult or dangerous to study in the real world. CAVS researchers have worked with law enforcement officers to study how officers communicate with dispatchers and with each other. We have also worked with both the Mississippi and U.S. Departments of Transportation to study how to help keep highway workers and pedestrians safe. CAVS researchers also develop new driving simulator technologies for the U.S. Army including models for simulating how pedestrians react when an Army vehicle drives through a neighborhood. CAVS researchers are now developing VR-based driving simulators to study how soldiers will work with semi-automated systems in military vehicles.

Mixed Reality Lab

The CAVS Mixed Reality Studio is the result of a partnership between CAVS and the Mitchell Memorial Library. The Mixed Reality Studio is located in the Mitchell Memorial Library’s MaxxSouth Digital Media Center. The studio includes two HTC Vive headsets, three Oculus Rift headsets, and two Microsoft HoloLens systems. The studio is equipped with a wide range of virtual reality software for education, entertainment, and creativity. In addition to VR experiences, the studio provides computer workstations equipped with software programs used to develop virtual reality applications. The CAVS Mixed Reality Studio makes virtual reality available for the public to experience and for students and faculty to build skills in virtual reality development and teaching.