WALL STONE EXTRACTION BASED ON STACKED CONDITIONAL GAN AND MULTISCALE IMAGE SEGMENTATION

The historical castles (castellated walls), which are cultural heritages in Japan, require regular maintenance, and it is necessary to record the arrangement of individual wall stones in the maintenance work. Recently, image processing techniques are practiced to optimize maintenance and management of the infrastructure assets. In the previous study, we proposed an automatic method for efficiently extracting individual wall stone polygons by improved multiscale image segmentation technique. However, the problem has remained that wall stone polygons could not be extracted properly when there were no clear gaps or boundaries between stones. To address this problem, we improved the multiscale image segmentation technique used in our previous studies. The first improvement is that in the region growing process, selecting the best combination of a plurality of objects instead of two. The second improvement is the modification of the shape criterion to be used. Besides, we proposed three-stage Stacked cGAN for wall stone edge detection that enables us to complement areas with weak or broken boundaries of stone edges. This approach is composed of a coarse-to-fine based image-to-edges translation network. The edge images derived from this method are used as the additional channel in multiscale image segmentation with a higher weight compared to the other RGB channels. It was confirmed that the separation performance of individual wall stone polygons was improved by the proposed method. Furthermore, the proposed method is highly effective to reduce the difficulty in setting of the scale parameter, which is usually sensitive to segmentation results and requires trial and error.

EXTENDED MULTISCALE IMAGE SEGMENTATION FOR CASTELLATED WALL MANAGEMENT

Castellated walls are positioned as tangible cultural heritage, which require regular maintenance to preserve their original state. For the demolition and repair work of the castellated wall, it is necessary to identify the individual stones constituting the wall. However, conventional approaches using laser scanning or integrated circuits (IC) tags were very time-consuming and cumbersome. Therefore, we herein propose an efficient approach for castellated wall management based on an extended multiscale image segmentation technique. In this approach, individual stone polygons are extracted from the castellated wall image and are associated with a stone management database. First, to improve the performance of the extraction of individual stone polygons having a convex shape, we developed a new shape criterion named <i>convex hull fitness</i> in the image segmentation process and confirmed its effectiveness. Next, we discussed the stone management database and its beneficial utilization in the repair work of castellated walls. Subsequently, we proposed irregular-shape indexes that are helpful for evaluating the stone shape and the stability of the stone arrangement state in castellated walls. Finally, we demonstrated an application of the proposed method for a typical castellated wall in Japan. Consequently, we confirmed that the stone polygons can be extracted with an acceptable level. Further, the condition of the shapes and the layout of the stones could be visually judged with the proposed irregular-shape indexes.