Proceedings of the Brazilian Symposium on Multimedia and the Web最新文献_第9页

Multimodal intent classification with incomplete modalities using text embedding propagation 基于文本嵌入传播的不完全模态多模态意图分类

Proceedings of the Brazilian Symposium on Multimedia and the Web Pub Date : 2021-09-27 DOI: 10.1145/3470482.3479636

Victor Machado Gonzaga, Nils Murrugarra-Llerena, R. Marcacini

{"title":"Multimodal intent classification with incomplete modalities using text embedding propagation","authors":"Victor Machado Gonzaga, Nils Murrugarra-Llerena, R. Marcacini","doi":"10.1145/3470482.3479636","DOIUrl":"https://doi.org/10.1145/3470482.3479636","url":null,"abstract":"Determining the author's intent in a social media post is a challenging multimodal task and requires identifying complex relationships between image and text in the post. For example, the post image can represent an object, person, product, or company, while the text can be an ironic message about the image content. Similarly, a text can be a news headline, while the image represents a provocation, meme, or satire about the news. Existing approaches propose intent classification techniques combining both modalities. However, some posts may have missing textual annotations. Hence, we investigate a graph-based approach that propagates available text embedding data from complete multimodal posts to incomplete ones. This paper presents a text embedding propagation method, which transfers embeddings from BERT neural language models to image-only posts (i.e., posts with incomplete modality) considering the topology of a graph constructed from both visual and textual modalities available during the training step. By using this inference approach, our method provides competitive results when textual modality is available at different completeness levels, even compared to reference methods that require complete modalities.","PeriodicalId":350776,"journal":{"name":"Proceedings of the Brazilian Symposium on Multimedia and the Web","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125308901","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Learning Textual Representations from Multiple Modalities to Detect Fake News Through One-Class Learning 通过一堂课学习，学习多种形式的文本表示来检测假新闻

Proceedings of the Brazilian Symposium on Multimedia and the Web Pub Date : 2021-09-27 DOI: 10.1145/3470482.3479634

M. Gôlo, M. C. D. Souza, R. G. Rossi, S. O. Rezende, B. Nogueira, R. Marcacini

{"title":"Learning Textual Representations from Multiple Modalities to Detect Fake News Through One-Class Learning","authors":"M. Gôlo, M. C. D. Souza, R. G. Rossi, S. O. Rezende, B. Nogueira, R. Marcacini","doi":"10.1145/3470482.3479634","DOIUrl":"https://doi.org/10.1145/3470482.3479634","url":null,"abstract":"Fake news can rapidly spread through internet users. Approaches proposed in the literature for content classification usually learn models considering textual and contextual features from real and fake news to minimize the spread of disinformation. One of the prominent approaches to detect fake news is One-Class Learning (OCL), as it minimizes the data labeling effort, requiring only the labeling of fake news documents. The performance of these algorithms depends on the structured representation of the documents used in the learning process. Generally, a textual-based unimodal representation is used, such as bag-of-words or representations based on linguistic categories. We propose MVAE-FakeNews, a multimodal representation method to detect fake news in OCL. The proposed approach uses a Multimodal Variational Autoencoder, learns a new representation from the combination of two modalities considered promising for fake news detection: text embeddings and topic information. In the experiments, we used three datasets considering Portuguese and English languages. Results show that the MVAE-FakeNews obtained a better F1-Score for the class of interest, outperforming another nine methods in ten of twelve evaluated scenarios. MVAE-FakeNews presented a better average ranking and statistical difference from other representation models. The proposed method proved to be promising to represent the texts in the OCL scenario to detect fake news.","PeriodicalId":350776,"journal":{"name":"Proceedings of the Brazilian Symposium on Multimedia and the Web","volume":"83 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121905287","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 7

Towards Understanding the Use of Telegram by Political Groups in Brazil 了解巴西政治团体对电报的使用

Proceedings of the Brazilian Symposium on Multimedia and the Web Pub Date : 2021-09-27 DOI: 10.1145/3470482.3479640

Manoel Júnior, P. Melo, Ana P. C. Silva, Fabrício Benevenuto, J. Almeida

{"title":"Towards Understanding the Use of Telegram by Political Groups in Brazil","authors":"Manoel Júnior, P. Melo, Ana P. C. Silva, Fabrício Benevenuto, J. Almeida","doi":"10.1145/3470482.3479640","DOIUrl":"https://doi.org/10.1145/3470482.3479640","url":null,"abstract":"Instant messaging platforms such as Telegram and WhatsApp became one of the main means of communication used by people all over the world. In most of these services, communities are created around the so called groups and channels, allowing easy, encrypted and instantaneous information exchange. With the political debate gaining a widespread attention from the public and permeated with intense discussion and polarization, specially in a context in which far right communities are being banned from maistream social networks like Twitter, Youtube, and Facebook, alternative platforms, like Telegram become very popular as they start to be seeked as a \"free space to discussion\" and abused for dissemination of misinformation and hate speech. This work consists in a data analysis for Brazilian public groups and channels for political discussion on Telegram, observing the network created in the platform as well as a closer look in the dynamics of messages and members in this platform. Our findings show that political mobilization increased substantially on Telegram in recent years, suggesting a mass migration from other mainstream platforms. We find the large groups structure of Telegram are effective in spreading the messages through the network, with the content being viewed by numerous users and forwarded multiple times. Looking at the messages, we find an expressive interplay between Telegram and external web pages, notably for Youtube and other social networks. Furthermore, we observed a relevant amount of messages attacking political personalities and spreading unchecked content about COVID-19 pandemic. Taken all together, we perform an extense study in how political discussion advanced on Telegram in Brazil and how they operate in this alternative messaging application.","PeriodicalId":350776,"journal":{"name":"Proceedings of the Brazilian Symposium on Multimedia and the Web","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127881755","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 11

Identifying Criminal Suspects on Social Networks: A Vocabulary-Based Method 基于词汇的社交网络犯罪嫌疑人识别方法

Proceedings of the Brazilian Symposium on Multimedia and the Web Pub Date : 2020-11-30 DOI: 10.1145/3428658.3431091

Érick S. Florentino, R. Goldschmidt, M. C. Cavalcanti

{"title":"Identifying Criminal Suspects on Social Networks: A Vocabulary-Based Method","authors":"Érick S. Florentino, R. Goldschmidt, M. C. Cavalcanti","doi":"10.1145/3428658.3431091","DOIUrl":"https://doi.org/10.1145/3428658.3431091","url":null,"abstract":"Identifying suspects of crimes on social networks is one of the most relevant tasks in the analysis of this type of network. Most of the computational methods focused on this task involve supervised machine learning, and, therefore, require previously labeled datasets that inform, among the registered people, messages and/or conversations, which ones are suspects. However, in practice, this type of information is not available, for several reasons, among which, it is rare or even protected by secrecy guaranteed by law. This limitation makes it very difficult to effectively use these methods in real situations. Hence, the present work raises the hypothesis that the use of a controlled vocabulary on the field of application can make it possible the identification of suspects in social networks, without the need for previously labeled datasets. In order to search for experimental evidence that points to the validity of the hypothesis raised, this article proposes a generic method that uses a controlled vocabulary with categorized terms, according to a certain domain (e.g., pedophilia, cyberbullying, terrorism, etc.), to analyze messages exchanged on social networks, in order to identify criminal suspects. The results obtained in a preliminary experiment in pedophilia domain showed signs of adequacy of the proposed method.","PeriodicalId":350776,"journal":{"name":"Proceedings of the Brazilian Symposium on Multimedia and the Web","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121822902","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Evaluating Early Fusion Operators at Mid-Level Feature Space 中期特征空间早期融合算子评估

Proceedings of the Brazilian Symposium on Multimedia and the Web Pub Date : 2020-11-30 DOI: 10.1145/3428658.3431079

Antonio A. R. Beserra, R. M. Kishi, R. Goularte

{"title":"Evaluating Early Fusion Operators at Mid-Level Feature Space","authors":"Antonio A. R. Beserra, R. M. Kishi, R. Goularte","doi":"10.1145/3428658.3431079","DOIUrl":"https://doi.org/10.1145/3428658.3431079","url":null,"abstract":"Early fusion techniques have been proposed in video analysis tasks as a way to improve efficacy by generating compact data models capable of keeping semantic clues present on multimodal data. First attempts to fuse multimodal data employed fusion operators at low-level feature space, losing data representativeness. This drove later research efforts to evolve simple operators to complex operations, which became, in general, inseparable of the multimodal semantic clues processing. In this paper, we investigate the application of early multimodal fusion operators at the mid-level feature space. Five different operators (Concatenation, Sum, Gram, Average and Maximum) were employed to fuse mid-level multimodal video features. Fused data derived from each operator were then used as input for two different video analysis tasks: Temporal Video Scene Segmentation and Video Classification. For each task, we performed a comparative analysis among the operators and related work techniques designed for these tasks using complex fusion operations. The efficacy results reached by the operators were very close to those reached by the techniques, pointing out strong evidence that working on a more homogeneous feature space can reduce known low-level fusion drawbacks. In addition, operators make data fusion separable, allowing researchers to keep the focus on developing semantic clues representations.","PeriodicalId":350776,"journal":{"name":"Proceedings of the Brazilian Symposium on Multimedia and the Web","volume":"95 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122127260","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Using Data Augmentation and Neural Networks to Improve the Emotion Analysis of Brazilian Portuguese Texts 使用数据增强和神经网络改进巴西葡萄牙语文本的情感分析

Proceedings of the Brazilian Symposium on Multimedia and the Web Pub Date : 2020-11-30 DOI: 10.1145/3428658.3431080

Vinícius Veríssimo, Rostand E. O. Costa

{"title":"Using Data Augmentation and Neural Networks to Improve the Emotion Analysis of Brazilian Portuguese Texts","authors":"Vinícius Veríssimo, Rostand E. O. Costa","doi":"10.1145/3428658.3431080","DOIUrl":"https://doi.org/10.1145/3428658.3431080","url":null,"abstract":"Information and Communication Technologies present as an interesting alternative for the mitigation of barriers that arise in the context of communication of information, mainly as technologies aimed at the machine translation of content in oral language into sign language. After years, despite the improvement of these technologies, the use of them still divides the opinions of the Deaf Community, due to the low emotional expressiveness of 3D avatars. Therefore, as a way to assist the machine translation of texts in oral language to sign language, this study aims to evaluate the influence of the parameters of a data augmentation method in a textual dataset and the use of neural networks for emotion analysis of Bazilian Portuguese texts. The analysis of emotions in texts presents a relevant challenge in diversity due to the nuances and different forms of expression that the human language uses. In this context, the use of deep neural networks has gained enough space as a way to deal with these challenges, mainly with the use of algorithms that deal with emotion analysis as a textual classification task, such as the MultiFiT approach. To circumvent the scarcity of data in Brazilian Portuguese aimed at this task, some strategies for increasing data were evaluated and applied to improve the database used in training. The results of the emotion analysis experiments with Transfer Learning pointed to accuracy above 94% in the best case.","PeriodicalId":350776,"journal":{"name":"Proceedings of the Brazilian Symposium on Multimedia and the Web","volume":"86 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117271297","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

An Approach for Improving DBpedia as a Research Data Hub 一个改进DBpedia作为研究数据中心的方法

Proceedings of the Brazilian Symposium on Multimedia and the Web Pub Date : 2020-11-30 DOI: 10.1145/3428658.3431075

Jean Gabriel Nguema Ngomo, G. R. Lopes, M. Campos, M. C. Cavalcanti

{"title":"An Approach for Improving DBpedia as a Research Data Hub","authors":"Jean Gabriel Nguema Ngomo, G. R. Lopes, M. Campos, M. C. Cavalcanti","doi":"10.1145/3428658.3431075","DOIUrl":"https://doi.org/10.1145/3428658.3431075","url":null,"abstract":"Extracted from Wikipedia content, DBpedia is considered one of the most important knowledge bases of the Semantic Web, which has editions in several languages, among which those in English (DBpedia EN) and Portuguese (DBpedia PT). All DBpedia editions are subject to quality issues, more especially DBpedia PT suffers from inconsistencies and lack of data in several domains. This paper describes a semi-automatic and incremental process for publishing data on DBpedia, coming from reliable external sources, while seeking to improve aspects of its quality. In an open science context, the proposal aims at consolidating DBpedia as a reference hub for research data, so that research from any area supported by the Semantic Web data can use its data reliably. Although the approach is independent from a specific DBpedia edition, the supporting prototype tool, named ETL4DBpedia, was built for DBpedia PT, based on ETL workflows (Extract, Transform, Load). This paper also describes the assessment of the approach, applying the tool in a real-usage scenario involving data from the field of botany. This application resulted in an increase by 127% in the completeness of species of medicinal plants in DBpedia PT, besides showing satisfactory performance for ETL4Bpedia components.","PeriodicalId":350776,"journal":{"name":"Proceedings of the Brazilian Symposium on Multimedia and the Web","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114221808","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

An Ontology-based Information Model for Multi-Domain Semantic Modeling and Analysis of Smart City Data 基于本体的智慧城市数据多领域语义建模与分析信息模型

Proceedings of the Brazilian Symposium on Multimedia and the Web Pub Date : 2020-11-30 DOI: 10.1145/3428658.3430973

B. Rocha, Larysse Silva, T. Batista, Everton Cavalcante, Porfírio Gomes

{"title":"An Ontology-based Information Model for Multi-Domain Semantic Modeling and Analysis of Smart City Data","authors":"B. Rocha, Larysse Silva, T. Batista, Everton Cavalcante, Porfírio Gomes","doi":"10.1145/3428658.3430973","DOIUrl":"https://doi.org/10.1145/3428658.3430973","url":null,"abstract":"Smart city services are typically defined according to domains (e.g., health, education, safety) and supported by different systems. Consequently, the analysis of smart city data is often domain-specific, thus limiting the capabilities of the offered services and hampering decision-making that relies on isolated domain information. To support a suitable analysis across multiple domains, it is necessary having a unified data model able to handle the inherent heterogeneity of smart city data and take into account both geographic and citizen information. This paper presents an ontology-based information model to support multi-domain analysis in smart cities to foster interoperability and powerful automated reasoning upon unambiguous information. The proposed information model follows Linked Data principles and takes advantage of ontologies to define information semantically. The semantic relationships and properties defined in the model also allow inferring new pieces of information that improve accuracy when analyzing multiple city domains. This paper reports an evaluation of the information model through ontological metrics and competence questions.","PeriodicalId":350776,"journal":{"name":"Proceedings of the Brazilian Symposium on Multimedia and the Web","volume":"70 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134262183","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 8

Analyzing the Use of COVID-19 Ads on Facebook 分析Facebook上COVID-19广告的使用情况

Proceedings of the Brazilian Symposium on Multimedia and the Web Pub Date : 2020-11-30 DOI: 10.1145/3428658.3431088

Márcio Silva, Fabrício Benevenuto

引用次数: 2

Collaborative Filtering Strategy for Product Recommendation Using Personality Characteristics of Customers 基于顾客个性特征的产品推荐协同过滤策略

Proceedings of the Brazilian Symposium on Multimedia and the Web Pub Date : 2020-11-30 DOI: 10.1145/3428658.3430969

J. J. B. Aguiar, J. Fechine, E. Costa

引用次数: 6