arXiv - CS - Information Retrieval最新文献_第8页

RegNLP in Action: Facilitating Compliance Through Automated Information Retrieval and Answer Generation RegNLP 在行动：通过自动信息检索和答案生成促进合规性

arXiv - CS - Information Retrieval Pub Date : 2024-09-09 DOI: arxiv-2409.05677

Tuba Gokhan, Kexin Wang, Iryna Gurevych, Ted Briscoe

引用次数: 0

Benchmarking Chinese Knowledge Rectification in Large Language Models 大型语言模型中的中文知识校正基准测试

arXiv - CS - Information Retrieval Pub Date : 2024-09-09 DOI: arxiv-2409.05806

Tianhe Lu, Jizhan Fang, Yunzhi Yao, Xin Xu, Ningyu Zhang, Huajun Chen

{"title":"Benchmarking Chinese Knowledge Rectification in Large Language Models","authors":"Tianhe Lu, Jizhan Fang, Yunzhi Yao, Xin Xu, Ningyu Zhang, Huajun Chen","doi":"arxiv-2409.05806","DOIUrl":"https://doi.org/arxiv-2409.05806","url":null,"abstract":"While Large Language Models (LLMs) exhibit remarkable generative\u0000capabilities, they are not without flaws, particularly in the form of\u0000hallucinations. This issue is even more pronounced when LLMs are applied to\u0000specific languages and domains. For example, LLMs may generate nonsense\u0000information when handling Chinese ancient poetry, proverbs, or idioms, owing to\u0000the lack of specific knowledge. To this end, this paper introduces a benchmark\u0000for rectifying Chinese knowledge in LLMs via knowledge editing. Specifically,\u0000we introduce a new Chinese dataset, CKnowEdit, by collecting seven type of\u0000knowledge from various sources, including classical texts, idioms, and content\u0000from Baidu Tieba Ruozhiba, thereby accounting for the unique polyphony,\u0000antithesis, and logical constructs inherent in the Chinese language. Through\u0000the analysis of this dataset, we uncover the challenges faced by current LLMs\u0000in mastering Chinese. Furthermore, our evaluation of state-of-the-art knowledge\u0000editing techniques on this dataset unveil the substantial scope for advancement\u0000in the rectification of Chinese knowledge. Code and dataset are available at\u0000https://github.com/zjunlp/EasyEdit.","PeriodicalId":501281,"journal":{"name":"arXiv - CS - Information Retrieval","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142205357","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

RBoard: A Unified Platform for Reproducible and Reusable Recommender System Benchmarks RBoard：可重复使用的推荐系统基准统一平台

arXiv - CS - Information Retrieval Pub Date : 2024-09-09 DOI: arxiv-2409.05526

Tri Kurniawan Wijaya, Edoardo D'Amico, Gabor Fodor, Manuel V. Loureiro

{"title":"RBoard: A Unified Platform for Reproducible and Reusable Recommender System Benchmarks","authors":"Tri Kurniawan Wijaya, Edoardo D'Amico, Gabor Fodor, Manuel V. Loureiro","doi":"arxiv-2409.05526","DOIUrl":"https://doi.org/arxiv-2409.05526","url":null,"abstract":"Recommender systems research lacks standardized benchmarks for\u0000reproducibility and algorithm comparisons. We introduce RBoard, a novel\u0000framework addressing these challenges by providing a comprehensive platform for\u0000benchmarking diverse recommendation tasks, including CTR prediction, Top-N\u0000recommendation, and others. RBoard's primary objective is to enable fully\u0000reproducible and reusable experiments across these scenarios. The framework\u0000evaluates algorithms across multiple datasets within each task, aggregating\u0000results for a holistic performance assessment. It implements standardized\u0000evaluation protocols, ensuring consistency and comparability. To facilitate\u0000reproducibility, all user-provided code can be easily downloaded and executed,\u0000allowing researchers to reliably replicate studies and build upon previous\u0000work. By offering a unified platform for rigorous, reproducible evaluation\u0000across various recommendation scenarios, RBoard aims to accelerate progress in\u0000the field and establish a new standard for recommender systems benchmarking in\u0000both academia and industry. The platform is available at https://rboard.org and\u0000the demo video can be found at https://bit.ly/rboard-demo.","PeriodicalId":501281,"journal":{"name":"arXiv - CS - Information Retrieval","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142205350","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

End-to-End Learnable Item Tokenization for Generative Recommendation 用于生成式推荐的端到端可学习项目标记化

arXiv - CS - Information Retrieval Pub Date : 2024-09-09 DOI: arxiv-2409.05546

Enze Liu, Bowen Zheng, Cheng Ling, Lantao Hu, Han Li, Wayne Xin Zhao

{"title":"End-to-End Learnable Item Tokenization for Generative Recommendation","authors":"Enze Liu, Bowen Zheng, Cheng Ling, Lantao Hu, Han Li, Wayne Xin Zhao","doi":"arxiv-2409.05546","DOIUrl":"https://doi.org/arxiv-2409.05546","url":null,"abstract":"Recently, generative recommendation has emerged as a promising new paradigm\u0000that directly generates item identifiers for recommendation. However, a key\u0000challenge lies in how to effectively construct item identifiers that are\u0000suitable for recommender systems. Existing methods typically decouple item\u0000tokenization from subsequent generative recommendation training, likely\u0000resulting in suboptimal performance. To address this limitation, we propose\u0000ETEGRec, a novel End-To-End Generative Recommender by seamlessly integrating\u0000item tokenization and generative recommendation. Our framework is developed\u0000based on the dual encoder-decoder architecture, which consists of an item\u0000tokenizer and a generative recommender. In order to achieve mutual enhancement\u0000between the two components, we propose a recommendation-oriented alignment\u0000approach by devising two specific optimization objectives: sequence-item\u0000alignment and preference-semantic alignment. These two alignment objectives can\u0000effectively couple the learning of item tokenizer and generative recommender,\u0000thereby fostering the mutual enhancement between the two components. Finally,\u0000we further devise an alternating optimization method, to facilitate stable and\u0000effective end-to-end learning of the entire framework. Extensive experiments\u0000demonstrate the effectiveness of our proposed framework compared to a series of\u0000traditional sequential recommendation models and generative recommendation\u0000baselines.","PeriodicalId":501281,"journal":{"name":"arXiv - CS - Information Retrieval","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142205349","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Replicability Measures for Longitudinal Information Retrieval Evaluation 纵向信息检索评估的可复制性措施

arXiv - CS - Information Retrieval Pub Date : 2024-09-09 DOI: arxiv-2409.05417

Jüri Keller, Timo Breuer, Philipp Schaer

引用次数: 0

DatAasee -- A Metadata-Lake as Metadata Catalog for a Virtual Data-Lake DatAasee -- 作为虚拟数据湖元数据目录的元数据湖

arXiv - CS - Information Retrieval Pub Date : 2024-09-09 DOI: arxiv-2409.05512

Christian Himpe

引用次数: 0

Assessing SPARQL capabilities of Large Language Models 评估大型语言模型的 SPARQL 功能

arXiv - CS - Information Retrieval Pub Date : 2024-09-09 DOI: arxiv-2409.05925

Lars-Peter Meyer, Johannes Frey, Felix Brei, Natanael Arndt

{"title":"Assessing SPARQL capabilities of Large Language Models","authors":"Lars-Peter Meyer, Johannes Frey, Felix Brei, Natanael Arndt","doi":"arxiv-2409.05925","DOIUrl":"https://doi.org/arxiv-2409.05925","url":null,"abstract":"The integration of Large Language Models (LLMs) with Knowledge Graphs (KGs)\u0000offers significant synergistic potential for knowledge-driven applications. One\u0000possible integration is the interpretation and generation of formal languages,\u0000such as those used in the Semantic Web, with SPARQL being a core technology for\u0000accessing KGs. In this paper, we focus on measuring out-of-the box capabilities\u0000of LLMs to work with SPARQL and more specifically with SPARQL SELECT queries\u0000applying a quantitative approach. We implemented various benchmarking tasks in the LLM-KG-Bench framework for\u0000automated execution and evaluation with several LLMs. The tasks assess\u0000capabilities along the dimensions of syntax, semantic read, semantic create,\u0000and the role of knowledge graph prompt inclusion. With this new benchmarking tasks, we evaluated a selection of GPT, Gemini,\u0000and Claude models. Our findings indicate that working with SPARQL SELECT\u0000queries is still challenging for LLMs and heavily depends on the specific LLM\u0000as well as the complexity of the task. While fixing basic syntax errors seems\u0000to pose no problems for the best of the current LLMs evaluated, creating\u0000semantically correct SPARQL SELECT queries is difficult in several cases.","PeriodicalId":501281,"journal":{"name":"arXiv - CS - Information Retrieval","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142205347","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

OneEdit: A Neural-Symbolic Collaboratively Knowledge Editing System OneEdit：神经符号协作知识编辑系统

arXiv - CS - Information Retrieval Pub Date : 2024-09-09 DOI: arxiv-2409.07497

Ningyu Zhang, Zekun Xi, Yujie Luo, Peng Wang, Bozhong Tian, Yunzhi Yao, Jintian Zhang, Shumin Deng, Mengshu Sun, Lei Liang, Zhiqiang Zhang, Xiaowei Zhu, Jun Zhou, Huajun Chen

{"title":"OneEdit: A Neural-Symbolic Collaboratively Knowledge Editing System","authors":"Ningyu Zhang, Zekun Xi, Yujie Luo, Peng Wang, Bozhong Tian, Yunzhi Yao, Jintian Zhang, Shumin Deng, Mengshu Sun, Lei Liang, Zhiqiang Zhang, Xiaowei Zhu, Jun Zhou, Huajun Chen","doi":"arxiv-2409.07497","DOIUrl":"https://doi.org/arxiv-2409.07497","url":null,"abstract":"Knowledge representation has been a central aim of AI since its inception.\u0000Symbolic Knowledge Graphs (KGs) and neural Large Language Models (LLMs) can\u0000both represent knowledge. KGs provide highly accurate and explicit knowledge\u0000representation, but face scalability issue; while LLMs offer expansive coverage\u0000of knowledge, but incur significant training costs and struggle with precise\u0000and reliable knowledge manipulation. To this end, we introduce OneEdit, a\u0000neural-symbolic prototype system for collaborative knowledge editing using\u0000natural language, which facilitates easy-to-use knowledge management with KG\u0000and LLM. OneEdit consists of three modules: 1) The Interpreter serves for user\u0000interaction with natural language; 2) The Controller manages editing requests\u0000from various users, leveraging the KG with rollbacks to handle knowledge\u0000conflicts and prevent toxic knowledge attacks; 3) The Editor utilizes the\u0000knowledge from the Controller to edit KG and LLM. We conduct experiments on two\u0000new datasets with KGs which demonstrate that OneEdit can achieve superior\u0000performance.","PeriodicalId":501281,"journal":{"name":"arXiv - CS - Information Retrieval","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142205315","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Federated Transfer Learning Based Cooperative Wideband Spectrum Sensing with Model Pruning 基于联合迁移学习的合作式宽带频谱传感与模型剪枝

arXiv - CS - Information Retrieval Pub Date : 2024-09-09 DOI: arxiv-2409.05462

Jibin Jia, Peihao Dong, Fuhui Zhou, Qihui Wu

引用次数: 0

Recommender Systems Algorithm Selection for Ranking Prediction on Implicit Feedback Datasets 在隐式反馈数据集上进行排名预测的推荐系统算法选择

arXiv - CS - Information Retrieval Pub Date : 2024-09-09 DOI: arxiv-2409.05461

Lukas Wegmeth, Tobias Vente, Joeran Beel

{"title":"Recommender Systems Algorithm Selection for Ranking Prediction on Implicit Feedback Datasets","authors":"Lukas Wegmeth, Tobias Vente, Joeran Beel","doi":"arxiv-2409.05461","DOIUrl":"https://doi.org/arxiv-2409.05461","url":null,"abstract":"The recommender systems algorithm selection problem for ranking prediction on\u0000implicit feedback datasets is under-explored. Traditional approaches in\u0000recommender systems algorithm selection focus predominantly on rating\u0000prediction on explicit feedback datasets, leaving a research gap for ranking\u0000prediction on implicit feedback datasets. Algorithm selection is a critical\u0000challenge for nearly every practitioner in recommender systems. In this work,\u0000we take the first steps toward addressing this research gap. We evaluate the\u0000NDCG@10 of 24 recommender systems algorithms, each with two hyperparameter\u0000configurations, on 72 recommender systems datasets. We train four optimized\u0000machine-learning meta-models and one automated machine-learning meta-model with\u0000three different settings on the resulting meta-dataset. Our results show that\u0000the predictions of all tested meta-models exhibit a median Spearman correlation\u0000ranging from 0.857 to 0.918 with the ground truth. We show that the median\u0000Spearman correlation between meta-model predictions and the ground truth\u0000increases by an average of 0.124 when the meta-model is optimized to predict\u0000the ranking of algorithms instead of their performance. Furthermore, in terms\u0000of predicting the best algorithm for an unknown dataset, we demonstrate that\u0000the best optimized traditional meta-model, e.g., XGBoost, achieves a recall of\u000048.6%, outperforming the best tested automated machine learning meta-model,\u0000e.g., AutoGluon, which achieves a recall of 47.2%.","PeriodicalId":501281,"journal":{"name":"arXiv - CS - Information Retrieval","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142226275","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0