Proceedings of the Joint Workshop of the 4th Workshop on Affective Social Multimedia Computing and first Multi-Modal Affective Computing of Large-Scale Multimedia Data

Social multimedia traffic is growing exponentially with the increased usage and continuous development of services and applications based on multimedia. Quality of Service (QoS), Quality of Information (QoI), scalability, reliability and such factors that are essential for social multimedia networks are realized by secure data transmission. For delivering actionable and timely insights in order to meet the growing demands of the user, multimedia analytics is performed by means of a trust-based paradigm. Efficient management and control of the network is facilitated by limiting certain capabilities such as energy-aware networking and runtime security in Software Defined Networks. In social multimedia context, suspicious flow detection is performed by a hybrid deep learning based anomaly detection scheme in order to enhance the SDN reliability. The entire process is divided into two modules namely – Abnormal activities detection using support vector machine based on Gradient descent and improved restricted Boltzmann machine which facilitates the anomaly detection module, and satisfying the strict requirements of QoS like low latency and high bandwidth in SDN using end-to-end data delivery module. In social multimedia, data delivery and anomaly detection services are essential in order to improve the efficiency and effectiveness of the system. For this purpose, we use benchmark datasets as well as real time evaluation to experimentally evaluate the proposed scheme. Detection of malicious events like confidential data collection, profile cloning and identity theft are performed to analyze the performance of the system using CMU-based insider threat dataset for large scale analysis.

Download Full-text

Affective Computing for Large-scale Heterogeneous Multimedia Data

ACM Transactions on Multimedia Computing Communications and Applications ◽

10.1145/3363560 ◽

2020 ◽

Vol 15 (3s) ◽

pp. 1-32

Author(s):

Sicheng Zhao ◽

Shangfei Wang ◽

Mohammad Soleymani ◽

Dhiraj Joshi ◽

Qiang Ji

Keyword(s):

Affective Computing ◽

Large Scale ◽

Multimedia Data

Download Full-text

Investigating the Relationship Between Emotion Recognition Software and Usability Metrics

i-com ◽

10.1515/icom-2020-0009 ◽

2020 ◽

Vol 19 (2) ◽

pp. 139-151

Author(s):

Thomas Schmidt ◽

Miriam Schlindwein ◽

Katharina Lichtner ◽

Christian Wolff

Keyword(s):

Emotion Recognition ◽

Regression Models ◽

Affective Computing ◽

Large Scale ◽

General Purpose ◽

Thinking Aloud ◽

Continue Research ◽

Usability Metrics ◽

Recognition Software ◽

The Relationship

AbstractDue to progress in affective computing, various forms of general purpose sentiment/emotion recognition software have become available. However, the application of such tools in usability engineering (UE) for measuring the emotional state of participants is rarely employed. We investigate if the application of sentiment/emotion recognition software is beneficial for gathering objective and intuitive data that can predict usability similar to traditional usability metrics. We present the results of a UE project examining this question for the three modalities text, speech and face. We perform a large scale usability test (N = 125) with a counterbalanced within-subject design with two websites of varying usability. We have identified a weak but significant correlation between text-based sentiment analysis on the text acquired via thinking aloud and SUS scores as well as a weak positive correlation between the proportion of neutrality in users’ voice and SUS scores. However, for the majority of the output of emotion recognition software, we could not find any significant results. Emotion metrics could not be used to successfully differentiate between two websites of varying usability. Regression models, either unimodal or multimodal could not predict usability metrics. We discuss reasons for these results and how to continue research with more sophisticated methods.

Download Full-text

Temporal Activity Path Based Character Correction in Heterogeneous Social Networks via Multimedia Sources

Advances in Multimedia ◽

10.1155/2018/2058670 ◽

2018 ◽

Vol 2018 ◽

pp. 1-16

Author(s):

Jun Long ◽

Lei Zhu ◽

Zhan Yang ◽

Chengyuan Zhang ◽

Xinpan Yuan

Keyword(s):

Social Networks ◽

Social Network ◽

Large Scale ◽

Multimedia Data ◽

Attribute Information ◽

Novel Method ◽

Relationship Network ◽

Heterogeneous Social Networks ◽

The Relationship ◽

Over Time

Vast amount of multimedia data contains massive and multifarious social information which is used to construct large-scale social networks. In a complex social network, a character should be ideally denoted by one and only one vertex. However, it is pervasive that a character is denoted by two or more vertices with different names; thus it is usually considered as multiple, different characters. This problem causes incorrectness of results in network analysis and mining. The factual challenge is that character uniqueness is hard to correctly confirm due to lots of complicated factors, for example, name changing and anonymization, leading to character duplication. Early, limited research has shown that previous methods depended overly upon supplementary attribute information from databases. In this paper, we propose a novel method to merge the character vertices which refer to the same entity but are denoted with different names. With this method, we firstly build the relationship network among characters based on records of social activities participating, which are extracted from multimedia sources. Then we define temporal activity paths (TAPs) for each character over time. After that, we measure similarity of the TAPs for any two characters. If the similarity is high enough, the two vertices should be considered as the same character. Based on TAPs, we can determine whether to merge the two character vertices. Our experiments showed that this solution can accurately confirm character uniqueness in large-scale social network.

Download Full-text