PRedictive reasOning and multi-source fusion empowering AntiCipation of attacks and Terrorist actions In Urban EnVironmEnts
FP7-SEC-2011-1 STREP Project (2012-2015, 36 months)
Programme type: 7th Framework Programme
Subprogramme: Strategies for countering a terrorist attack in an urban environment - Capability Project
Contract type: Small or medium-scale focused research project
Subject index: Social Aspects, Security
- VITROCISET Vitrociset S.p.A. (Italy)
- AGH University of Science and Technology (POLAND)
- AIT Research and Education Laboratory in Information Technology (GREECE)
- CMR Consorzio Milano Ricerche (Italy)
- HWC HW Communications Ltd (UK)
- KEMEA Center for Security Studies (KEMEA), Ministry of Citizen Protection(GREECE)
- KU Kingston University (UK)
- ISIG Istituto di Sociologia Internazionale di Gorizia (ITALY)
- MTA SZTAKI Computer and Automation Research Institute, Hungarian Academy Of Sciences (HUNGARY)
- UNIBWM Universität der Bundeswehr München (GERMANY)
The main goal of PROACTIVE is to provide a holistic IoT framework for enhanced situation awareness in an urban environment, for the purpose of timely anticipating and effectively reacting to terrorist attacks. The PROACTIVE framework will be user driven and citizen friendly. From a technological perspective, the framework will integrate a host of novel technologies enabling information collection, filtering, analysis and fusion from multiple geographically dispersed devices (including cameras, microphone arrays, wireless sensor networks and UAVs (Unmanned Aerial Vehicles). At the same time, the framework will research and integrate advanced reasoning techniques (tailored to the security domain), in order to intelligently process and derive high level terrorist oriented semantics from a multitude of sensors streams.
Following the validation and evaluation of the framework, the project PROACTIVE endeavor to produce a set of relevant best practices and blueprints towards contributing to a common EU approach to terrorist prevention in the urban environment. PROACTIVE is supported by a rich set of end-users, which are either members of the consortium or members of a special user group that has been established for the purposes of the project.
Key research points:
- Behavioural Analysis
- Scene Analysis
- Person Tracking
- Context and situational awareness
- Self organizing networks
- Information Fusion
- UAV sensor node interaction and automation
- Internet-of-Things middleware
- Sensor modeling and ontologies
2.1 Monitoring activity on crowded scene
Sensor configuration was containing four cameras with overlapping FoV.
Scenario events specified for this location require the tracking of individual pedestrian in/across the crowd, including the crowd density estimation in order to avoid high computational load. Analyze the dynamic parameters of motion trajectories and send signal in case of running movement. Running pedestrian will find a place for hiding and observing the area. This activity will be detected utilizing sensor fusion technique.
Result of sensor registration and the aligned top view.
Results of object detection on multispectral data.
Object localization results from single frames, next, the object tracking outputs.
Alarm generated when the average speed higher than 2 m/s, object and its trajectory are highlighted with red.
Selected regions in two viewpoints and the highlighted "hidden" object.
2.2 Monitoring parking area
Sensor configuration is built from three cameras.
Objectives are to validate the waiting/parking time durations in different areas (e.g. parking is prohibited). Object interaction is also investigated: vehicle and driver connection is continuously checked and alarm sent when loitering (suspicious) movement was detected.
The processing steps are demonstrating the following components:
- Multispectral sensor registration, fusion and object detection, object localization and tracking.
- Object movement classification for state definition (move, stop, wait, parking). To measure the parking duration the behavior analyzer module follows the state changes/transitions and timestamps while the tracking method is stable.
- Object interaction detection, connect vehicle and its getting out driver (outcome of T7.3). Loitering is defined as a continuous movement around (near to) the vehicle and human approaches the car less than 1.5 meter.
Restricted area where parking is prohibited is highlighted. States are assigned for detected objects ("move" and "park").
Parking car in restricted area and the loitering human are marked with red.
2.3 Entrance surveillance
Event near to the security check is the verification of the incoming bodies. Thermal sensor was used to detect and analyze human shapes in order to identify cold(er) region where some metal object was covered with the clothes.
Processing steps are utilizing object detection and feature localization on human shapes: head and foot are aligned precisely. Cold blobs are located on the body and a temporal time constrain is applied to reduce false alarms.
The detected and marked cold region on the human body is shown as a red rectangle alert.
2.4 Object fence protection
Outdoor event was defined as a "suspicious" object was dropped over the fence. Video analysis techniques are applied to a) detect the human next to the fence, b) isolate flying object and c) track it frame by frame. Alarm is generated when the object trajectory is parabolic and it flies over the fence.
Left three images are consecutive frames of the sequence. The high object speed caused long displacement vectors. Right image contains both the motion trajectory (red) and the estimated parabolic curve (red).