{"title":"基于多模态深度学习的仇恨模因预测模型","authors":"Md. Rekib Ahmed, Neeraj Bhadani, I. Chakraborty","doi":"10.1109/CCGE50943.2021.9776440","DOIUrl":null,"url":null,"abstract":"With the emergence of deep neural networks along with high-end computers that can process deep architectures, there has been a lot of research when Computer Vision and Natural Language Processing has been fused into a single problem. To enable students and researchers to deep dive into multimodal deep learning Facebook AI Research team published a dataset on hateful meme classification “The Hateful Meme Challenge Dataset” in May 2020 that gave us the motivation to test ourselves and an opportunity to learn more about the dataset. The rise of communication on the internet with memes as a medium, they have been used to convey incorrect information, political agendas and also has led to cyberbullying, trolling etc. This results in the need of creating an automated tool that can detect such hateful content published on the internet and remove it at the root level before it does any harm. This paper intends to adopt Unimodal Text and Image models using Bert, LSTM and VGG16, Resnet50, SE-Resnet50, XSE-Resnet architectures and combining them into Multimodal models for effective prediction of a hateful meme. The paper compares various architectures both unimodal models and multimodal models on the evaluation metrics AUC-ROC score, F1 score and accuracy score.)","PeriodicalId":130452,"journal":{"name":"2021 International Conference on Computing, Communication and Green Engineering (CCGE)","volume":"33 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2021-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":"{\"title\":\"Hateful Meme Prediction Model Using Multimodal Deep Learning\",\"authors\":\"Md. Rekib Ahmed, Neeraj Bhadani, I. Chakraborty\",\"doi\":\"10.1109/CCGE50943.2021.9776440\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"With the emergence of deep neural networks along with high-end computers that can process deep architectures, there has been a lot of research when Computer Vision and Natural Language Processing has been fused into a single problem. To enable students and researchers to deep dive into multimodal deep learning Facebook AI Research team published a dataset on hateful meme classification “The Hateful Meme Challenge Dataset” in May 2020 that gave us the motivation to test ourselves and an opportunity to learn more about the dataset. The rise of communication on the internet with memes as a medium, they have been used to convey incorrect information, political agendas and also has led to cyberbullying, trolling etc. This results in the need of creating an automated tool that can detect such hateful content published on the internet and remove it at the root level before it does any harm. This paper intends to adopt Unimodal Text and Image models using Bert, LSTM and VGG16, Resnet50, SE-Resnet50, XSE-Resnet architectures and combining them into Multimodal models for effective prediction of a hateful meme. The paper compares various architectures both unimodal models and multimodal models on the evaluation metrics AUC-ROC score, F1 score and accuracy score.)\",\"PeriodicalId\":130452,\"journal\":{\"name\":\"2021 International Conference on Computing, Communication and Green Engineering (CCGE)\",\"volume\":\"33 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2021-09-23\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"1\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2021 International Conference on Computing, Communication and Green Engineering (CCGE)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/CCGE50943.2021.9776440\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2021 International Conference on Computing, Communication and Green Engineering (CCGE)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/CCGE50943.2021.9776440","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
Hateful Meme Prediction Model Using Multimodal Deep Learning
With the emergence of deep neural networks along with high-end computers that can process deep architectures, there has been a lot of research when Computer Vision and Natural Language Processing has been fused into a single problem. To enable students and researchers to deep dive into multimodal deep learning Facebook AI Research team published a dataset on hateful meme classification “The Hateful Meme Challenge Dataset” in May 2020 that gave us the motivation to test ourselves and an opportunity to learn more about the dataset. The rise of communication on the internet with memes as a medium, they have been used to convey incorrect information, political agendas and also has led to cyberbullying, trolling etc. This results in the need of creating an automated tool that can detect such hateful content published on the internet and remove it at the root level before it does any harm. This paper intends to adopt Unimodal Text and Image models using Bert, LSTM and VGG16, Resnet50, SE-Resnet50, XSE-Resnet architectures and combining them into Multimodal models for effective prediction of a hateful meme. The paper compares various architectures both unimodal models and multimodal models on the evaluation metrics AUC-ROC score, F1 score and accuracy score.)