Lidar based people surveillance with tracking, biometric identification and activity recognition

We proposed new approaches for gait and activity analysis based on data streams of a Rotating Multi Beam (RMB) Lidar sensor. The introduced algorithms are embedded into an integrated 4D vision and visualization system, which is able to analyze and interactively display real scenarios in natural outdoor environments with walking pedestrians. The main focus of the investigations are gait based person re-identification during tracking, and recognition of specific activity patterns such as bending, waving, making phone calls and checking the time looking at wristwatches. The descriptors for training and recognition are observed and extracted from realistic outdoor surveillance scenarios, where multiple pedestrians are walking in the field of interest following possibly intersecting trajectories, thus the observations might often be affected by occlusions or background noise. Since there is no public database available for such scenarios, we created and published a new Lidar-based outdoor gait and activity dataset on our website, that contains point cloud sequences of 28 different persons extracted and aggregated from 35 minutes-long measurements. The presented results for Velodyne HDL64 [1,2,3,5], and Velodyne VLP16 [4] sensors confirm that both efficient gait-based identification and activity recognition is achievable in the sparse point clouds of a single RMB Lidar sensor. After extracting the people trajectories, we synthesized a free-viewpoint video, where moving avatar models follow the trajectories of the observed pedestrians in real time, ensuring that the leg movements of the animated avatars are synchronized with the real gait cycles observed in the Lidar stream.

 

Tracking multiple pedestrians in 3D [1], and silhouette projection to 2D [2]

 

Person re-identification using biometric gait features [2,3,4,5]

 

Recognizing various activities from point cloud sequences [4,5]

Benchmark dataset

Download Ground Truth data: SZTAKI-LGA Database

References:

[5]Cs. Benedek, B. Gálai, B. Nagy and Z. Jankó: ”Lidar-based Gait Analysis and Activity Recognition in a 4D Surveillance System,” IEEE Trans. on Circuits and Systems for Video Technology, vol. 28, no. 1, pp. 101-113, 2018, IF: 4.046

[4]B. Gálai, and Cs. Benedek: "Gait Recognition with Compact Lidar Sensors", International Conference on Computer Vision Theory and Applications (VISAPP), Porto, Portugal, 27 February - 1 March, 2017

[3] B. Gálai and C. Benedek: "Feature selection for Lidar-based gait recognition", International Workshop on Computational Intelligence for Multimedia Understanding (IWCIM), Prague, Czech Republic, October 29-30, 2015

[2] C. Benedek, B. Nagy, B. Gálai and Z. Jankó: "Lidar-based Gait Analysis in People Tracking and 4D Visualization", European Signal Processing Conference (EUSIPCO), Nice, France, August 31-September 4, 2015

[1] C. Benedek, "3D People Surveillance on Range Data Sequences of a Rotating Lidar," Pattern Recognition Letters, Special Issue on Depth Image Analysis vol. 50, pp. 149–158, 2014



Links

Geo-Information Computing @ Machine Perception Lab.

GeoComp Research Group

Machine Perception Laboratory

SZTAKI main page

GeoComp Demos:

Demo page

Publications

Contact

GeoComp Group leader: Dr. Csaba Benedek benedek.csaba@sztaki.hu

i4D project manager: Dr. Zsolt Jankó janko.zsolt@sztaki.hu

Head of MPLab: Prof. Tamás Szirányi

MPLab administration: Anikó Vágvölgyi

Address:

SZTAKI
Kende utca 13-17
H-1111 Budapest, Hungary
Tel: +36 1 279 6194
Fax: +36 1 279 6292

Flag Counter