Multi-Kernel Aggregation of Local and Global Features in Long Wave Infrared for Detection of SWAT Teams in Challenging Environments

Arya, A., Anderson, D., Bethel, C. L., & Carruth, D. W. (2013). Multi-Kernel Aggregation of Local and Global Features in Long Wave Infrared for Detection of SWAT Teams in Challenging Environments. SPIE Defense, Security, and Sensing Conference. Baltimore, MD.

Abstract

A vision system was designed for people detection to provide support to SWAT team members operating in challenging environments such as low-to-no light, smoke, etc. When the vision system is mounted on a mobile robot platform: it will enable the robot to function as an effective member of the SWAT team; to provide surveillance information; to make first contact with suspects; and provide safe entry for team members. The vision task is challenging because SWAT team members are typically concealed, carry various equipment such as shields, and perform tactical and stealthy maneuvers. Occlusion is a particular challenge because team members operate in close proximity to one another. An uncooled electro-optical/long wave infrared (EO/LWIR) camera, 7.5 to 13.5 m, was used. A unique thermal dataset was collected of SWAT team members from multiple teams performing tactical maneuvers during monthly training exercises. Our approach consisted of two stages: an object detector trained on people to find candidate windows, and a secondary feature extraction, multi-kernel (MK) aggregation and classification step to distinguish between SWAT team members and civilians. Two types of thermal features, local and global, are presented based on maximally stable extremal region (MSER) blob detection. Support vector machine (SVM) classification results of approximately [70,93]% for SWAT team member detection are reported based on the exploration of different combinations of visual information in terms of training data.