Optimizing Performance: How Compact Models Match or Exceed GPT's Classification Capabilities through Fine-Tuning

Baptiste Lefort, Eric Benhamou, Jean-Jacques Ohana, David Saltiel, Beatrice Guez
{"title":"Optimizing Performance: How Compact Models Match or Exceed GPT's Classification Capabilities through Fine-Tuning","authors":"Baptiste Lefort, Eric Benhamou, Jean-Jacques Ohana, David Saltiel, Beatrice Guez","doi":"arxiv-2409.11408","DOIUrl":null,"url":null,"abstract":"In this paper, we demonstrate that non-generative, small-sized models such as\nFinBERT and FinDRoBERTa, when fine-tuned, can outperform GPT-3.5 and GPT-4\nmodels in zero-shot learning settings in sentiment analysis for financial news.\nThese fine-tuned models show comparable results to GPT-3.5 when it is\nfine-tuned on the task of determining market sentiment from daily financial\nnews summaries sourced from Bloomberg. To fine-tune and compare these models,\nwe created a novel database, which assigns a market score to each piece of news\nwithout human interpretation bias, systematically identifying the mentioned\ncompanies and analyzing whether their stocks have gone up, down, or remained\nneutral. Furthermore, the paper shows that the assumptions of Condorcet's Jury\nTheorem do not hold suggesting that fine-tuned small models are not independent\nof the fine-tuned GPT models, indicating behavioural similarities. Lastly, the\nresulted fine-tuned models are made publicly available on HuggingFace,\nproviding a resource for further research in financial sentiment analysis and\ntext classification.","PeriodicalId":501139,"journal":{"name":"arXiv - QuantFin - Statistical Finance","volume":"17 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - QuantFin - Statistical Finance","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2409.11408","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

Abstract

In this paper, we demonstrate that non-generative, small-sized models such as FinBERT and FinDRoBERTa, when fine-tuned, can outperform GPT-3.5 and GPT-4 models in zero-shot learning settings in sentiment analysis for financial news. These fine-tuned models show comparable results to GPT-3.5 when it is fine-tuned on the task of determining market sentiment from daily financial news summaries sourced from Bloomberg. To fine-tune and compare these models, we created a novel database, which assigns a market score to each piece of news without human interpretation bias, systematically identifying the mentioned companies and analyzing whether their stocks have gone up, down, or remained neutral. Furthermore, the paper shows that the assumptions of Condorcet's Jury Theorem do not hold suggesting that fine-tuned small models are not independent of the fine-tuned GPT models, indicating behavioural similarities. Lastly, the resulted fine-tuned models are made publicly available on HuggingFace, providing a resource for further research in financial sentiment analysis and text classification.
优化性能:紧凑型模型如何通过微调匹配或超越 GPT 的分类能力
在本文中,我们证明了非生成的小型模型,如FinBERT和FinDRoBERTa,经过微调后,可以在金融新闻情感分析的零点学习设置中优于GPT-3.5和GPT-4模型。这些经过微调的模型在对GPT-3.5进行微调后,在从彭博社的每日金融新闻摘要中判断市场情感的任务上显示出与GPT-3.5相当的结果。为了对这些模型进行微调和比较,我们创建了一个新颖的数据库,在没有人为解读偏差的情况下,为每条新闻分配一个市场得分,系统地识别被提及的公司,分析其股票是上涨、下跌还是保持中立。此外,本文还表明,孔多塞评判定理的假设并不成立,这表明微调小模型与微调 GPT 模型并不独立,这表明了行为上的相似性。最后,本文在 HuggingFace 上公开了微调模型的结果,为进一步研究金融情感分析和文本分类提供了资源。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信