Mining the Semantics of Visual Concepts and Context

2005 ◽  
pp. 223-235
Author(s):  
Milind R. Naphade ◽  
John R. Smith
Keyword(s):  
2008 ◽  
Vol 52 (7) ◽  
pp. 808-823 ◽  
Author(s):  
L. Bai ◽  
S. Lao ◽  
A. F. Smeaton ◽  
N. E. O'Connor ◽  
D. Sadlier ◽  
...  

Author(s):  
Jose Alberto Raposo Pinheiro ◽  
Mirian Tavares

uTurn is a digital art installation that allows interaction inside a cinema-like environment or a similar public space — an exhibition system in the context of an audience, retrieving an elected media from the choices made by the majority of the public. The software in its core manages the selection — a meta-remediation that elects a media block, in the form of short-story movies (Vidbits) to be watched by a crowd. The interaction model assumes the need to find a preference in the viewing room in order to identify and choose the next Vidbit. The system allows navigation through media blocks in environments like a cinema room, a summer festival, or a public event. It can be configured to support visual concepts, or to integrate a narrative system in which other types of structures in the story demand that the content follows a segmentation of media. uTurn was exhibited during the 5th Artech International Conference, in 2015. The article addresses the creative process towards the production of the digital artifact using Apple's Quartz Composer.


2020 ◽  
Vol 2 (4) ◽  
pp. 397-413
Author(s):  
Pim Arendsen ◽  
Diego Marcos ◽  
Devis Tuia

In this paper, we study how to extract visual concepts to understand landscape scenicness. Using visual feature representations from a Convolutional Neural Network (CNN), we learn a number of Concept Activation Vectors (CAV) aligned with semantic concepts from ancillary datasets. These concepts represent objects, attributes or scene categories that describe outdoor images. We then use these CAVs to study their impact on the (crowdsourced) perception of beauty of landscapes in the United Kingdom. Finally, we deploy a technique to explore new concepts beyond those initially available in the ancillary dataset: Using a semi-supervised manifold alignment technique, we align the CNN image representation to a large set of word embeddings, therefore giving access to entire dictionaries of concepts. This allows us to obtain a list of new concept candidates to improve our understanding of the elements that contribute the most to the perception of scenicness. We do this without the need for any additional data by leveraging the commonalities in the visual and word vector spaces. Our results suggest that new and potentially useful concepts can be discovered by leveraging neighbourhood structures in the word vector spaces.


2019 ◽  
Vol 13 (01) ◽  
pp. 135-155
Author(s):  
Ye Zhang ◽  
Ryunosuke Tanishige ◽  
Ichiro Ide ◽  
Keisuke Doman ◽  
Yasutomo Kawanishi ◽  
...  

News videos are valuable multimedia information on real-world events. However, due to the incremental nature of the contents, a sequence of news videos on a related news topic could be redundant and lengthy. Thus, a number of methods have been proposed for their summarization. However, there is a problem that most of these methods do not consider the consistency between the auditory and visual contents. This becomes a problem in the case of news videos, since both contents do not always come from the same source. Considering this, in this paper, we propose a method for summarizing a sequence of news videos considering the consistency of auditory and visual contents. The proposed method first selects key-sentences from the auditory contents (Closed Caption) of each news story in the sequence, and next selects a shot in the news story whose “Visual Concepts” detected from the visual contents are the most consistent with the selected key-sentence. In the end, the audio segment corresponding to each key-sentence is synthesized with the selected shot, and then these clips are concatenated into a summarized video. Results from subjective experiments on summarized videos on several news topics show the effectiveness of the proposed method.


Sign in / Sign up

Export Citation Format

Share Document