ADVANCED GESTURE RECOGNIZING SURVEILANCE SYSTEMS USING MICROSOFT KINECTS

Abstract

This research explores the possibility of implementing an advanced ges-ture recognizing surveillance system (A.G.R.S.S.) with the capability of mon-itoring and targeting a person who performs a threatening gesture within a designated area. By networking multiple Microsoft Kinects (gesture based video game controllers) together, we hypothesize that people can be moni-tored, tracked, and targeted based on the gestures they perform. The suc-cessful development of an A.G.R.S.S. can provide significant support in spot-ting individuals who pose a threat which can have civilian and military im-plementations. Since each Kinect can provide a spatial representation for twenty joints on a person, we developed code that links the aforementioned information from each Kinect into a single program. With two Kinects run-ning, we did trials of our program to simulate a trade-off of information be-tween the two Kinects. We also used these trials to analyze the effectiveness of the gesture recognition software. We found that multiple Kinects can be linked together to monitor and target a person based on the gestures they perform. The outcome of the project is a program that uses two Kinects to observe (live video stream), target, follow, and capture a picture of a person who has simulated firing a hand gun. These results unequivocally answer the question that we set out to investigate. Therefore, we can conclude that an A.G.R.S.S. can be developed using multiple Microsoft Kinects. This research paves the way for a future A.G.R.S.S. that monitors larger areas, looks for more gestures, and implements biometrics to identify individuals of interest.