Proceedings. IEEE Conference on Multimedia Information Processing and Retrieval最新文献

筛选

英文中文

LitAI: Enhancing Multimodal Literature Understanding and Mining with Generative AI. LitAI：利用生成式人工智能加强多模态文学理解和挖掘。

Proceedings. IEEE Conference on Multimedia Information Processing and Retrieval Pub Date : 2024-08-01 Epub Date: 2024-10-15 DOI: 10.1109/mipr62202.2024.00080

Gowtham Medisetti, Zacchaeus Compson, Heng Fan, Huaxiao Yang, Yunhe Feng

{"title":"LitAI: Enhancing Multimodal Literature Understanding and Mining with Generative AI.","authors":"Gowtham Medisetti, Zacchaeus Compson, Heng Fan, Huaxiao Yang, Yunhe Feng","doi":"10.1109/mipr62202.2024.00080","DOIUrl":"10.1109/mipr62202.2024.00080","url":null,"abstract":"Information processing and retrieval in literature are critical for advancing scientific research and knowledge discovery. The inherent multimodality and diverse literature formats, including text, tables, and figures, present significant challenges in literature information retrieval. This paper introduces LitAI, a novel approach that employs readily available generative AI tools to enhance multimodal information retrieval from literature documents. By integrating tools such as optical character recognition (OCR) with generative AI services, LitAI facilitates the retrieval of text, tables, and figures from PDF documents. We have developed specific prompts that leverage in-context learning and prompt engineering within Generative AI to achieve precise information extraction. Our empirical evaluations, conducted on datasets from the ecological and biological sciences, demonstrate the superiority of our approach over several established baselines including Tesseract-OCR and GPT-4. The implementation of LitAI is accessible at https://github.com/ResponsibleAILab/LitAI.","PeriodicalId":520274,"journal":{"name":"Proceedings. IEEE Conference on Multimedia Information Processing and Retrieval","volume":"2024 ","pages":"471-476"},"PeriodicalIF":0.0,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11526646/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142560310","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0