Deep learning based crowd monitoring and person identification system

Abstract

In this paper, we propose a deep learning-based crowd monitoring and person identification system and a crowd-video dataset to address the challenges posed by the recent COVID-19 pandemic or future pandemics may occur, where maintaining social distance in public places is necessary. The system can be also implemented where crowd monitoring is necessary. For instance, public places like bank booths, airports, train stations, hospitals, tourist attractions, public transportation hubs, stadiums, arenas, etc. The system combines person tracking and social distance measurements to accurately detect individuals who may unintentionally violate the rules due to a lack of spatial awareness. To implement the system, a custom dataset was created to evaluate and tackle perspective correction and person-only detection issues. Three popular object detection models: YOLOv8, YOLOv7, and Faster R-CNN with and without the DeepSort tracking algorithm were used and a comparison of their performances is demonstrated. To build our system we took two different approaches. In the first approach, we used Faster R-CNN & YOLOv8 for person identification, and for tracking, we used the SORT tracking algorithm. In the second approach, we used YOLOv7 & YOLOv8 for person detection and DeepSortbased tracking algorithm which generates unique IDs and successfully tracks and reidentifies each person frame by frame using Kalman filter and Hungarian algorithm. The experimental results show that all models can accurately detect humans with 90% accuracy and estimate the distance between them. However, Faster R-CNN falls short in real-time human detection, whereas YOLOv8 outperforms YOLOv7 in terms of speed and detection accuracy. Still, YOLOv8 is relatively new and has less support for implementation. Thus, YOLOv7 is chosen for implementation in mobile, micro-controller-based, or IoT devices, as it offers better support for immediate implementation. The proposed system is efficient, accurate, and does not require human supervision. It includes a log system to track violations with frame rates and unique IDs. The system was tested using our custom dataset, and positive results were achieved, indicating its potential usefulness in crowd monitoring and social distance enforcement.