Explaining the Unique Behavioral Characteristics of Elderly and Adults Based on Deep Learning

In modern society, the population has been aging as the lifespan has increased owing to the advancement in medical technologies. This could pose a threat to the economic system and, in serious cases, to the ethics regarding the socially-weak elderly. An analysis of the behavioral characteristics of the elderly and young adults based on their physical conditions enables silver robots to provide customized services for the elderly to counter aging society problems, laying the groundwork for improving elderly welfare systems and automating elderly care systems. Accordingly, skeleton sequences modeling the changes of the human body are converted into pose evolution images (PEIs), and a convolutional neural network (CNN) is trained to classify the elderly and young adults for a single behavior. Then, a heatmap, which is a contributed portion of the inputs, is obtained using a gradient-weighted class activation map (Grad-CAM) for the classified results, and a skeleton-heatmap is obtained through a series of processes for the ease of analysis. Finally, the behavioral characteristics are derived through the difference matching analysis between the domains based on the skeleton-heatmap and RGB video matching analysis. In this study, we present the analysis of the behavioral characteristics of the elderly and young adults based on cognitive science using deep learning and discuss the examples of the analysis. Therefore, we have used the ETRI-Activity3D dataset, which is the largest of its kind among the datasets that have classified the behaviors of young adults and the elderly.

CT-Video Matching for Retrograde Intrarenal Surgery Based on Depth Prediction and Style Transfer

Retrograde intrarenal surgery (RIRS) is a minimally invasive endoscopic procedure for the treatment of kidney stones. Traditionally, RIRS is usually performed by reconstructing a 3D model of the kidney from preoperative CT images in order to locate the kidney stones; then, the surgeon finds and removes the stones with experience in endoscopic video. However, due to the many branches within the kidney, it can be difficult to relocate each lesion and to ensure that all branches are searched, which may result in the misdiagnosis of some kidney stones. To avoid this situation, we propose a convolutional neural network (CNN)-based method for matching preoperative CT images and intraoperative videos for the navigation of ureteroscopic procedures. First, a pair of synthetic images and depth maps reflecting preoperative information are obtained from a 3D model of the kidney. Then, a style transfer network is introduced to transfer the ureteroscopic images to the synthetic images, which can generate the associated depth maps. Finally, the fusion and matching of depth maps of preoperative images and intraoperative video images are realized based on semantic features. Compared with the traditional CT-video matching method, our method achieved a five times improvement in time performance and a 26% improvement in the top 10 accuracy.

Learning Implicit Temporal Alignment for Few-shot Video Classification

Few-shot video classification aims to learn new video categories with only a few labeled examples, alleviating the burden of costly annotation in real-world applications. However, it is particularly challenging to learn a class-invariant spatial-temporal representation in such a setting. To address this, we propose a novel matching-based few-shot learning strategy for video sequences in this work. Our main idea is to introduce an implicit temporal alignment for a video pair, capable of estimating the similarity between them in an accurate and robust manner. Moreover, we design an effective context encoding module to incorporate spatial and feature channel context, resulting in better modeling of intra-class variations. To train our model, we develop a multi-task loss for learning video matching, leading to video features with better generalization. Extensive experimental results on two challenging benchmarks, show that our method outperforms the prior arts with a sizable margin on Something-Something-V2 and competitive results on Kinetics.