The CAVS Off-Road Data Set:

An Open Source Dataset for Off-road Autonomous Vehicles

The CAVS Off-road Dataset is a collection of autonomous vehicle sensor data taken primarily in unstructured, or off-road, environments. The dataset includes data from sensors such as lidar, camera, radar and GPS, as well as vehicle proprioceptive sensors such as wheel speed. The dataset is open-source; we ask that researchers and developers appropriately reference the source when publishing or presenting work which makes use of this resource.

The published paper can be found at and the appropriate citation is:

Sharma, Suvash, Lalitha Dabbiru, Tyler Hannis, George Mason, Daniel W.Carruth, Matthew Doude, Chris Goodin et al. CaT: CAVS Traversability Dataset for Off-Road Autonomous Driving. IEEE Access 10 (2022): 24759-24768.

Dataset 1


This dataset comprises two sets of segmentation data specially targeted for the unstructured, off-road driving environment. The first set is a real-world semantic segmentation dataset with 500 images that were collected at the Center for Advanced Vehicular Systems (CAVS) proving grounds, which is a 55-acre area with trails, hills, obstacles, dense vegetation and trees (a forest environment). Data were collected during two different seasons: winter 2018 and summer 2019. The images are semantically labeled using classes such as smooth trail, rough trail, small vegetation, forest, sky, and obstacles. For the real-world dataset collection, a Sekonix SF3325-100 GMSL camera was mounted on the top of a Polaris Ranger crew XP100, capturing imagery at 10 frames per second. The second set is a synthetic dataset for semantic segmentation in unstructured driving environment generated using an in-house, physics-based, high-fidelity simulator, the Mississippi State University Autonomous Vehicle Simulator (MAVS). This set consists of 1,947 image pairs featuring different time of the day, lighting condition, plant density, cloud cover in the sky, and landscape.

Dataset 2


The off-road driving domain has many uncertainties such as uneven terrain structure, positive and negative obstacles, ditches, quagmires, hidden objects, and more that make it very unstructured. This dataset contains a new set of segmented images that are prepared considering such uncertainties and vehicle capability. We call it the “CaT” dataset. It contains 1812 two-dimensional camera images collected in three different locations around the CAVS proving ground. The images are annotated considering the traversing capability of three different vehicles: sedan, pickup, and off-road.