Open Source Datasets for Off-road Autonomous Vehicles

Researchers at CAVS have collected and labeled several datasets targeted towards off-road autonomous driving. These datasets consist of autonomous vehicle sensor data taken primarily in unstructured or off-road environments from sensors such as camera and GPS. These datasets are open source; we ask that researchers and developers appropriately reference the source when publishing or presenting work which makes use of this resource.

Off-road Autonomous Driving Segmentation Dataset

This dataset is comprised of two sets of segmentation data specially targeted for the unstructured, off-road driving environment.

  • The first set is a real-world semantic segmentation dataset with 1700 images that were collected at the CAVS proving grounds, which is a 55-acre area with trails, hills, obstacles, dense vegetation, similar to a forest environment. Data were collected during winter 2018 and summer 2019 using a Sekonix SF3325-100 GMSL camera mounted on the top of a Polaris Ranger crew XP100, capturing imagery at 10 frames per second. The images are semantically labeled using classes such as smooth trail, rough trail, small vegetation, forest, sky, and obstacles.

  • The second set is a synthetic dataset for semantic segmentation in unstructured driving environment generated using an in-house, physics-based, high-fidelity simulator, the Mississippi State University Autonomous Vehicle Simulator (MAVS). This set consists of 1,947 image pairs featuring different time of the day, lighting condition, plant density, cloud cover, and landscape.

CaT: CAVS Traversibility Dataset

The off-road driving domain has many uncertainties such as uneven terrain structure, positive and negative obstacles, ditches, hidden objects, and many more. This dataset contains a set of segmented images, which are annotated by vehicle drivers according to their perception of the vehicle’s ability to drive through the terrain. It contains 12,300 two-dimensional camera images collected in three different locations around the CAVS proving ground. The images are annotated considering the traversing capability of three different vehicles: sedan, pickup, and off-road.

The published paper can be found at and the appropriate citation is:

Sharma, Suvash, Lalitha Dabbiru, Tyler Hannis, George Mason, Daniel W.Carruth, Matthew Doude, Chris Goodin et al. CaT: CAVS Traversability Dataset for Off-Road Autonomous Driving. IEEE Access 10 (2022): 24759-24768.