AI Open Pub Date : 2024-01-01 DOI: 10.1016/j.aiopen.2024.01.006

Zongtao Liu , Wei Dong , Chaoliang Wang , Haoqingzi Shen , Gang Sun , Qun jiang , Quanjin Tao , Yang Yang

{"title":"Boosting graph search with attention network for solving the general orienteering problem","authors":"Zongtao Liu , Wei Dong , Chaoliang Wang , Haoqingzi Shen , Gang Sun , Qun jiang , Quanjin Tao , Yang Yang","doi":"10.1016/j.aiopen.2024.01.006","DOIUrl":"https://doi.org/10.1016/j.aiopen.2024.01.006","url":null,"abstract":"<div><p>Recently, several studies explore to use neural networks(NNs) to solve different routing problems, which is an auspicious direction. These studies usually design an encoder–decoder based framework that uses encoder embeddings of nodes and the problem-specific context to iteratively generate node sequence(path), and further optimize the produced result on top, such as a beam search. However, these models are limited to accepting only the coordinates of nodes as input, disregarding the self-referential nature of the studied routing problems, and failing to account for the low reliability of node selection in the initial stages, thereby posing challenges for real-world applications.</p><p>In this paper, we take the orienteering problem as an example to tackle these limitations in the previous studies. We propose a novel combination of a variant beam search algorithm and a learned heuristic for solving the general orienteering problem. We acquire the heuristic with an attention network that takes the distances among nodes as input, and learn it via a reinforcement learning framework. The empirical studies show that our method can surpass a wide range of baselines and achieve results iteratively generate the optimal or highly specialized approach.</p></div>","PeriodicalId":100068,"journal":{"name":"AI Open","volume":"5 ","pages":"Pages 46-54"},"PeriodicalIF":0.0,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S266665102400007X/pdfft?md5=4bd44cc9b0d6326c8e34b456fa017774&pid=1-s2.0-S266665102400007X-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139936464","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Wave2Graph: Integrating spectral features and correlations for graph-based learning in sound waves Wave2Graph：整合频谱特征和相关性，实现基于图谱的声波学习

AI Open Pub Date : 2024-01-01 DOI: 10.1016/j.aiopen.2024.08.004

Van-Truong Hoang , Khanh-Tung Tran , Xuan-Son Vu , Duy-Khuong Nguyen , Monowar Bhuyan , Hoang D. Nguyen

{"title":"Wave2Graph: Integrating spectral features and correlations for graph-based learning in sound waves","authors":"Van-Truong Hoang , Khanh-Tung Tran , Xuan-Son Vu , Duy-Khuong Nguyen , Monowar Bhuyan , Hoang D. Nguyen","doi":"10.1016/j.aiopen.2024.08.004","DOIUrl":"10.1016/j.aiopen.2024.08.004","url":null,"abstract":"<div><p>This paper investigates a novel graph-based representation of sound waves inspired by the physical phenomenon of correlated vibrations. We propose a Wave2Graph framework for integrating multiple acoustic representations, including the spectrum of frequencies and correlations, into various neural computing architectures to achieve new state-of-the-art performances in sound classification. The capability and reliability of our end-to-end framework are evidently demonstrated in voice pathology for low-cost and non-invasive mass-screening of medical conditions, including respiratory illnesses and Alzheimer’s Dementia. We conduct extensive experiments on multiple public benchmark datasets (ICBHI and ADReSSo) and our real-world dataset (IJSound: Respiratory disease detection using coughs and breaths). Wave2Graph framework consistently outperforms previous state-of-the-art methods with a large magnitude, up to 7.65% improvement, promising the usefulness of graph-based representation in signal processing and machine learning.</p></div>","PeriodicalId":100068,"journal":{"name":"AI Open","volume":"5 ","pages":"Pages 115-125"},"PeriodicalIF":0.0,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2666651024000147/pdfft?md5=39354e1c8fc8f37b3f91eb3d652b379f&pid=1-s2.0-S2666651024000147-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142158467","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

How to generate popular post headlines on social media? 如何在社交媒体上生成受欢迎的帖子标题？

AI Open Pub Date : 2023-12-16 DOI: 10.1016/j.aiopen.2023.12.002

Zhouxiang Fang , Min Yu , Zhendong Fu , Boning Zhang , Xuanwen Huang , Xiaoqi Tang , Yang Yang

{"title":"How to generate popular post headlines on social media?","authors":"Zhouxiang Fang , Min Yu , Zhendong Fu , Boning Zhang , Xuanwen Huang , Xiaoqi Tang , Yang Yang","doi":"10.1016/j.aiopen.2023.12.002","DOIUrl":"https://doi.org/10.1016/j.aiopen.2023.12.002","url":null,"abstract":"<div><p>Posts, as important containers of user-generated-content on social media, are of tremendous social influence and commercial value. As an integral component of post, headline has decisive influence on post’s popularity. However, the current mainstream method for headline generation is still manually writing, which is unstable and requires extensive human effort. This drives us to explore a novel research question: Can we automate the generation of popular headlines on social media? We collect more than 1 million posts of 42,447 thousand celebrities from public data of Xiaohongshu, which is a well-known social media platform in China. We then conduct careful observations on the headlines of these posts. Observation results demonstrate that trends and personal styles are widespread in headlines on social medias and have significant contribution to posts’ popularity. Motivated by these insights, we present MEBART, which combines <strong>M</strong>ultiple preference-<strong>E</strong>xtractors with <strong>B</strong>idirectional and <strong>A</strong>uto-Regressive Transformers (BART), capturing trends and personal styles to generate popular headlines on social medias. We perform extensive experiments on real-world datasets and achieve SOTA performance compared with advanced baselines. In addition, ablation and case studies demonstrate that <em>MEBART</em> advances in capturing trends and personal styles.</p></div>","PeriodicalId":100068,"journal":{"name":"AI Open","volume":"5 ","pages":"Pages 1-9"},"PeriodicalIF":0.0,"publicationDate":"2023-12-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2666651023000244/pdfft?md5=77f6189a8605961caeb7262aab78dbf9&pid=1-s2.0-S2666651023000244-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139050352","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Language as a latent sequence: Deep latent variable models for semi-supervised paraphrase generation 作为潜在序列的语言：半监督转述生成的深层潜在变量模型

AI Open Pub Date : 2023-01-01 DOI: 10.1016/j.aiopen.2023.05.001

Jialin Yu , Alexandra I. Cristea , Anoushka Harit , Zhongtian Sun , Olanrewaju Tahir Aduragba , Lei Shi , Noura Al Moubayed

{"title":"Language as a latent sequence: Deep latent variable models for semi-supervised paraphrase generation","authors":"Jialin Yu , Alexandra I. Cristea , Anoushka Harit , Zhongtian Sun , Olanrewaju Tahir Aduragba , Lei Shi , Noura Al Moubayed","doi":"10.1016/j.aiopen.2023.05.001","DOIUrl":"https://doi.org/10.1016/j.aiopen.2023.05.001","url":null,"abstract":"<div><p>This paper explores deep latent variable models for semi-supervised paraphrase generation, where the missing target pair for unlabelled data is modelled as a latent paraphrase sequence. We present a novel unsupervised model named <em>variational sequence auto-encoding reconstruction</em> (<strong>VSAR</strong>), which performs latent sequence inference given an observed text. To leverage information from text pairs, we additionally introduce a novel supervised model we call <em>dual directional learning</em> (<strong>DDL</strong>), which is designed to integrate with our proposed VSAR model. Combining VSAR with DDL (<strong>DDL+VSAR</strong>) enables us to conduct semi-supervised learning. Still, the combined model suffers from a cold-start problem. To further combat this issue, we propose an improved weight initialisation solution, leading to a novel two-stage training scheme we call <em>knowledge-reinforced-learning</em> (<strong>KRL</strong>). Our empirical evaluations suggest that the combined model yields competitive performance against the state-of-the-art supervised baselines on complete data. Furthermore, in scenarios where only a fraction of the labelled pairs are available, our combined model consistently outperforms the strong supervised model baseline (<strong>DDL</strong>) by a significant margin (<span><math><mrow><mi>p</mi><mo><</mo><mo>.</mo><mn>05</mn></mrow></math></span>; Wilcoxon test). Our code is publicly available at https://github.com/jialin-yu/latent-sequence-paraphrase.</p></div>","PeriodicalId":100068,"journal":{"name":"AI Open","volume":"4 ","pages":"Pages 19-32"},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49710554","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

UPRec: User-aware Pre-training for sequential Recommendation UPRec：顺序推荐的用户感知预培训

AI Open Pub Date : 2023-01-01 DOI: 10.1016/j.aiopen.2023.08.008

Chaojun Xiao , Ruobing Xie , Yuan Yao , Zhiyuan Liu , Maosong Sun , Xu Zhang , Leyu Lin

{"title":"UPRec: User-aware Pre-training for sequential Recommendation","authors":"Chaojun Xiao , Ruobing Xie , Yuan Yao , Zhiyuan Liu , Maosong Sun , Xu Zhang , Leyu Lin","doi":"10.1016/j.aiopen.2023.08.008","DOIUrl":"https://doi.org/10.1016/j.aiopen.2023.08.008","url":null,"abstract":"<div><p>Recent years witness the success of pre-trained models to alleviate the data sparsity problem in recommender systems. However, existing pre-trained models for recommendation mainly focus on leveraging universal sequence patterns from user behavior sequences and item information, whereas ignore heterogeneous user information to capture personalized interests, which has been shown to contribute to the personalized recommendation. In this paper, we propose a simple yet effective model, called <strong>U</strong>ser-aware <strong>P</strong>re-training for <strong>Rec</strong>ommendation (UPRec), which could flexibly encode heterogeneous user information into the sequential modeling of user behaviors. Specifically, UPRec first encodes the sequential behavior to generate user embeddings, and then jointly optimizes the model with the sequential objective and user-aware objective constructed from the user attributes and structured social graphs. Comprehensive experimental results on two real-world large-scale recommendation datasets demonstrate that UPRec can effectively enrich the user representations with user attributes and social relations and thus provide more appropriate recommendations for users.</p></div>","PeriodicalId":100068,"journal":{"name":"AI Open","volume":"4 ","pages":"Pages 137-144"},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49710673","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Learning fair representations via an adversarial framework 通过对抗性框架学习公平表征

AI Open Pub Date : 2023-01-01 DOI: 10.1016/j.aiopen.2023.08.003

Huadong Qiu , Rui Feng , Ruoyun Hu , Xiao Yang , Shaowa Lin , Quanjin Tao , Yang Yang

{"title":"Learning fair representations via an adversarial framework","authors":"Huadong Qiu , Rui Feng , Ruoyun Hu , Xiao Yang , Shaowa Lin , Quanjin Tao , Yang Yang","doi":"10.1016/j.aiopen.2023.08.003","DOIUrl":"https://doi.org/10.1016/j.aiopen.2023.08.003","url":null,"abstract":"<div><p>Fairness has become a central issue for our research community as classification algorithms are adopted in societally critical domains such as recidivism prediction and loan approval. In this work, we consider the potential bias based on protected attributes (e.g., race and gender), and tackle this problem by learning latent representations of individuals that are statistically indistinguishable between protected groups while sufficiently preserving other information for classification. To do that, we develop a minimax adversarial framework with a <em>generator</em> to capture the data distribution and generate latent representations, and a <em>critic</em> to ensure that the distributions across different protected groups are similar. Our framework provides theoretical guarantee with respect statistical parity and individual fairness. Empirical results on four real-world datasets also show that the learned representation can effectively be used for classification tasks such as credit risk prediction while obstructing information related to protected groups, especially when removing protected attributes is not sufficient for fair classification.</p></div>","PeriodicalId":100068,"journal":{"name":"AI Open","volume":"4 ","pages":"Pages 91-97"},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49761371","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4

Associating multiple vision transformer layers for fine-grained image representation 关联多个视觉转换层以实现细粒度图像表示

AI Open Pub Date : 2023-01-01 DOI: 10.1016/j.aiopen.2023.09.001

Fayou Sun , Hea Choon Ngo , Yong Wee Sek , Zuqiang Meng

{"title":"Associating multiple vision transformer layers for fine-grained image representation","authors":"Fayou Sun , Hea Choon Ngo , Yong Wee Sek , Zuqiang Meng","doi":"10.1016/j.aiopen.2023.09.001","DOIUrl":"https://doi.org/10.1016/j.aiopen.2023.09.001","url":null,"abstract":"<div><p>- Accurate discriminative region proposal has an important effect for fine-grained image recognition. The vision transformer (ViT) brings about a striking effect in computer vision due to its innate multi-head self-attention mechanism. However, the attention maps are gradually similar after certain layers, and since ViT used a classification token to achieve classification, it is unable to effectively select discriminative image patches for fine-grained image classification. To accurately detect discriminative regions, we propose a novel network AMTrans, which efficiently increases layers to learn diverse features and utilizes integrated raw attention maps to capture more salient features. Specifically, we employ DeepViT as backbone to solve the attention collapse issue. Then, we fuse each head attention weight within each layer to produce an attention weight map. After that, we alternatively use recurrent residual refinement blocks to promote salient feature and then utilize the semantic grouping method to propose the discriminative feature region. A lot of experiments prove that AMTrans acquires the SOTA performance on four widely used fine-grained datasets under the same settings, involving Stanford-Cars, Stanford-Dogs, CUB-200-2011, and ImageNet.</p></div>","PeriodicalId":100068,"journal":{"name":"AI Open","volume":"4 ","pages":"Pages 130-136"},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49732632","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

MOTT: A new model for multi-object tracking based on green learning paradigm MOTT:基于绿色学习范式的多目标跟踪新模型

AI Open Pub Date : 2023-01-01 DOI: 10.1016/j.aiopen.2023.09.002

Shan Wu , Amnir Hadachi , Chaoru Lu , Damien Vivet

引用次数: 0

Semantic graph based topic modelling framework for multilingual fake news detection 基于语义图的多语言假新闻检测主题建模框架

AI Open Pub Date : 2023-01-01 DOI: 10.1016/j.aiopen.2023.08.004

Rami Mohawesh , Xiao Liu , Hilya Mudrika Arini , Yutao Wu , Hui Yin

{"title":"Semantic graph based topic modelling framework for multilingual fake news detection","authors":"Rami Mohawesh , Xiao Liu , Hilya Mudrika Arini , Yutao Wu , Hui Yin","doi":"10.1016/j.aiopen.2023.08.004","DOIUrl":"https://doi.org/10.1016/j.aiopen.2023.08.004","url":null,"abstract":"<div><p>Fake news detection is one of the most alluring problems that has grabbed the interest of Machine Learning (ML) and Natural Language Processing (NLP) experts in recent years. The majority of existing studies on detecting fake news are written in English, restricting its application outside the English-speaking population. The lack of annotated corpora and technologies makes it difficult to identify false news in the scenario of low-resource languages, despite the growth in multilingual web content. Moreover, existing works cannot collect more semantic and contextual characteristics from documents in a particular multilingual text corpus. To bridge up these challenges and deal with the multilingual fake news detection challenge, we develop a new semantic graph attention-based representation learning framework to extract structural and semantic representations of texts. Our experiments on TALLIP fake news datasets showed that the classification performance had been significantly enhanced, ranging from 1% to 7% in terms of accuracy metric, and our proposed framework outperformed the state-of-the-art techniques for the multilingual fake news detection task.</p></div>","PeriodicalId":100068,"journal":{"name":"AI Open","volume":"4 ","pages":"Pages 33-41"},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49710400","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

AdaDS: Adaptive data selection for accelerating pre-trained language model knowledge distillation 加速预训练语言模型知识升华的自适应数据选择

AI Open Pub Date : 2023-01-01 DOI: 10.1016/j.aiopen.2023.08.005

Qinhong Zhou , Peng Li , Yang Liu , Yuyang Guan , Qizhou Xing , Ming Chen , Maosong Sun , Yang Liu

{"title":"AdaDS: Adaptive data selection for accelerating pre-trained language model knowledge distillation","authors":"Qinhong Zhou , Peng Li , Yang Liu , Yuyang Guan , Qizhou Xing , Ming Chen , Maosong Sun , Yang Liu","doi":"10.1016/j.aiopen.2023.08.005","DOIUrl":"https://doi.org/10.1016/j.aiopen.2023.08.005","url":null,"abstract":"<div><p>Knowledge distillation (KD) is a widely used method for transferring knowledge from large teacher models to computationally efficient student models. Unfortunately, the computational cost of KD becomes unaffordable as pre-trained language models (PLMs) grow larger. Computing KD loss on only part of the training set is a promising way to accelerate KD. However, existing works heuristically leverage only one static data selection strategy during the KD process, demonstrating inconsistent improvements across different distillation scenarios. In this work, we conduct a thorough study on various typical data selection strategies for KD, and show that this problem is due to the fact that the best data selection strategy is specific to various factors, including task, selected data size, and training stage. To automatically adapt to these factors, we propose a framework named AdaDS to learn to choose the data selection strategy adaptively during the KD process. Experimental results show that our proposed method is effective for various tasks and selected data sizes under both fine-tuning and pre-training stages, achieving comparable performance to DistilBERT with only 10% amount of queries to the teacher model.</p></div>","PeriodicalId":100068,"journal":{"name":"AI Open","volume":"4 ","pages":"Pages 56-63"},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49732904","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

AI Open最新文献