Baptiste Lefort, Eric Benhamou, Jean-Jacques Ohana, David Saltiel, Beatrice Guez
{"title":"优化性能:紧凑型模型如何通过微调匹配或超越 GPT 的分类能力","authors":"Baptiste Lefort, Eric Benhamou, Jean-Jacques Ohana, David Saltiel, Beatrice Guez","doi":"arxiv-2409.11408","DOIUrl":null,"url":null,"abstract":"In this paper, we demonstrate that non-generative, small-sized models such as\nFinBERT and FinDRoBERTa, when fine-tuned, can outperform GPT-3.5 and GPT-4\nmodels in zero-shot learning settings in sentiment analysis for financial news.\nThese fine-tuned models show comparable results to GPT-3.5 when it is\nfine-tuned on the task of determining market sentiment from daily financial\nnews summaries sourced from Bloomberg. To fine-tune and compare these models,\nwe created a novel database, which assigns a market score to each piece of news\nwithout human interpretation bias, systematically identifying the mentioned\ncompanies and analyzing whether their stocks have gone up, down, or remained\nneutral. Furthermore, the paper shows that the assumptions of Condorcet's Jury\nTheorem do not hold suggesting that fine-tuned small models are not independent\nof the fine-tuned GPT models, indicating behavioural similarities. Lastly, the\nresulted fine-tuned models are made publicly available on HuggingFace,\nproviding a resource for further research in financial sentiment analysis and\ntext classification.","PeriodicalId":501139,"journal":{"name":"arXiv - QuantFin - Statistical Finance","volume":"17 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Optimizing Performance: How Compact Models Match or Exceed GPT's Classification Capabilities through Fine-Tuning\",\"authors\":\"Baptiste Lefort, Eric Benhamou, Jean-Jacques Ohana, David Saltiel, Beatrice Guez\",\"doi\":\"arxiv-2409.11408\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"In this paper, we demonstrate that non-generative, small-sized models such as\\nFinBERT and FinDRoBERTa, when fine-tuned, can outperform GPT-3.5 and GPT-4\\nmodels in zero-shot learning settings in sentiment analysis for financial news.\\nThese fine-tuned models show comparable results to GPT-3.5 when it is\\nfine-tuned on the task of determining market sentiment from daily financial\\nnews summaries sourced from Bloomberg. To fine-tune and compare these models,\\nwe created a novel database, which assigns a market score to each piece of news\\nwithout human interpretation bias, systematically identifying the mentioned\\ncompanies and analyzing whether their stocks have gone up, down, or remained\\nneutral. Furthermore, the paper shows that the assumptions of Condorcet's Jury\\nTheorem do not hold suggesting that fine-tuned small models are not independent\\nof the fine-tuned GPT models, indicating behavioural similarities. Lastly, the\\nresulted fine-tuned models are made publicly available on HuggingFace,\\nproviding a resource for further research in financial sentiment analysis and\\ntext classification.\",\"PeriodicalId\":501139,\"journal\":{\"name\":\"arXiv - QuantFin - Statistical Finance\",\"volume\":\"17 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-08-22\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - QuantFin - Statistical Finance\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2409.11408\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - QuantFin - Statistical Finance","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2409.11408","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
Optimizing Performance: How Compact Models Match or Exceed GPT's Classification Capabilities through Fine-Tuning
In this paper, we demonstrate that non-generative, small-sized models such as
FinBERT and FinDRoBERTa, when fine-tuned, can outperform GPT-3.5 and GPT-4
models in zero-shot learning settings in sentiment analysis for financial news.
These fine-tuned models show comparable results to GPT-3.5 when it is
fine-tuned on the task of determining market sentiment from daily financial
news summaries sourced from Bloomberg. To fine-tune and compare these models,
we created a novel database, which assigns a market score to each piece of news
without human interpretation bias, systematically identifying the mentioned
companies and analyzing whether their stocks have gone up, down, or remained
neutral. Furthermore, the paper shows that the assumptions of Condorcet's Jury
Theorem do not hold suggesting that fine-tuned small models are not independent
of the fine-tuned GPT models, indicating behavioural similarities. Lastly, the
resulted fine-tuned models are made publicly available on HuggingFace,
providing a resource for further research in financial sentiment analysis and
text classification.