Masonry image analysis
In this workpackage we introduce a novel end-to-end blind inpainting algorithm for masonry wall images, performing the automatic detection and virtual completion of occluded or damaged wall regions. For this purpose, we propose a three-stage deep neural network that comprises a U-Net-based sub-network for wall segmentation into brick, mortar and occluded regions, which is followed by a two-stage adversarial inpainting model.
The first adversarial network predicts the schematic mortar-brick pattern of the occluded areas based on the observed wall structure, providing in itself valuable structural information for archeological and architectural applications. Finally, the second adversarial network predicts the RGB pixel values yielding a realistic visual experience for the observer. While the three stages implement a sequential pipeline, they interact through dependencies of their loss functions admitting the consideration of hidden feature dependencies between the different network components. For training and testing the network a new dataset has been created, and an extensive qualitative and quantitative evaluation versus the state-of-the-art is given.
Publications:
Y. Ibrahim, B. Nagy and Cs. Benedek: ”Deep Learning-based Masonry Wall Image analysis,” Remote Sensing, vol. 12, no. 23, article 3918, 2020, IF: 4.848 
Open Access
Y. Ibrahim, B. Nagy and Cs. Benedek: "A GAN-based Blind Inpainting Method for Masonry Wall Images", International Conference on Pattern Recognition (ICPR), Milan, Italy (virtual conference), 10-15 January 2021 