{"title":"L3iTC at the FinLLM Challenge Task: Quantization for Financial Text Classification & Summarization","authors":"Elvys Linhares Pontes, Carlos-Emiliano González-Gallardo, Mohamed Benjannet, Caryn Qu, Antoine Doucet","doi":"arxiv-2408.03033","DOIUrl":"https://doi.org/arxiv-2408.03033","url":null,"abstract":"This article details our participation (L3iTC) in the FinLLM Challenge Task\u00002024, focusing on two key areas: Task 1, financial text classification, and\u0000Task 2, financial text summarization. To address these challenges, we\u0000fine-tuned several large language models (LLMs) to optimize performance for\u0000each task. Specifically, we used 4-bit quantization and LoRA to determine which\u0000layers of the LLMs should be trained at a lower precision. This approach not\u0000only accelerated the fine-tuning process on the training data provided by the\u0000organizers but also enabled us to run the models on low GPU memory. Our\u0000fine-tuned models achieved third place for the financial classification task\u0000with an F1-score of 0.7543 and secured sixth place in the financial\u0000summarization task on the official test datasets.","PeriodicalId":501309,"journal":{"name":"arXiv - CS - Computational Engineering, Finance, and Science","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141936370","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"SETN: Stock Embedding Enhanced with Textual and Network Information","authors":"Takehiro Takayanagi, Hiroki Sakaji, Kiyoshi Izumi","doi":"arxiv-2408.02899","DOIUrl":"https://doi.org/arxiv-2408.02899","url":null,"abstract":"Stock embedding is a method for vector representation of stocks. There is a\u0000growing demand for vector representations of stock, i.e., stock embedding, in\u0000wealth management sectors, and the method has been applied to various tasks\u0000such as stock price prediction, portfolio optimization, and similar fund\u0000identifications. Stock embeddings have the advantage of enabling the\u0000quantification of relative relationships between stocks, and they can extract\u0000useful information from unstructured data such as text and network data. In\u0000this study, we propose stock embedding enhanced with textual and network\u0000information (SETN) using a domain-adaptive pre-trained transformer-based model\u0000to embed textual information and a graph neural network model to grasp network\u0000information. We evaluate the performance of our proposed model on related\u0000company information extraction tasks. We also demonstrate that stock embeddings\u0000obtained from the proposed model perform better in creating thematic funds than\u0000those obtained from baseline methods, providing a promising pathway for various\u0000applications in the wealth management industry.","PeriodicalId":501309,"journal":{"name":"arXiv - CS - Computational Engineering, Finance, and Science","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141936371","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jiaxing Lu, Heran Li, Fangwei Ning, Yixuan Wang, Xinze Li, Yan Shi
{"title":"Constructing Mechanical Design Agent Based on Large Language Models","authors":"Jiaxing Lu, Heran Li, Fangwei Ning, Yixuan Wang, Xinze Li, Yan Shi","doi":"arxiv-2408.02087","DOIUrl":"https://doi.org/arxiv-2408.02087","url":null,"abstract":"Since ancient times, mechanical design aids have been developed to assist\u0000human users, aimed at improving the efficiency and effectiveness of design.\u0000However, even with the widespread use of contemporary Computer-Aided Design\u0000(CAD) systems, there are still high learning costs, repetitive work, and other\u0000challenges. In recent years, the rise of Large Language Models (LLMs) has\u0000introduced new productivity opportunities to the field of mechanical design.\u0000Yet, it remains unrealistic to rely on LLMs alone to complete mechanical design\u0000tasks directly. Through a series of explorations, we propose a method for\u0000constructing a comprehensive Mechanical Design Agent (MDA) by guiding LLM\u0000learning. To verify the validity of our proposed method, we conducted a series\u0000of experiments and presented relevant cases.","PeriodicalId":501309,"journal":{"name":"arXiv - CS - Computational Engineering, Finance, and Science","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141936373","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Bilateral Trade Flow Prediction by Gravity-informed Graph Auto-encoder","authors":"Naoto Minakawa, Kiyoshi Izumi, Hiroki Sakaji","doi":"arxiv-2408.01938","DOIUrl":"https://doi.org/arxiv-2408.01938","url":null,"abstract":"The gravity models has been studied to analyze interaction between two\u0000objects such as trade amount between a pair of countries, human migration\u0000between a pair of countries and traffic flow between two cities. Particularly\u0000in the international trade, predicting trade amount is instrumental to industry\u0000and government in business decision making and determining economic policies.\u0000Whereas the gravity models well captures such interaction between objects, the\u0000model simplifies the interaction to extract essential relationships or needs\u0000handcrafted features to drive the models. Recent studies indicate the\u0000connection between graph neural networks (GNNs) and the gravity models in\u0000international trade. However, to our best knowledge, hardly any previous\u0000studies in the this domain directly predicts trade amount by GNNs. We propose\u0000GGAE (Gravity-informed Graph Auto-encoder) and its surrogate model, which is\u0000inspired by the gravity model, showing trade amount prediction by the gravity\u0000model can be formulated as an edge weight prediction problem in GNNs and solved\u0000by GGAE and its surrogate model. Furthermore, we conducted experiments to\u0000indicate GGAE with GNNs can improve trade amount prediction compared to the\u0000traditional gravity model by considering complex relationships.","PeriodicalId":501309,"journal":{"name":"arXiv - CS - Computational Engineering, Finance, and Science","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141936375","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Md. Saikat Islam Khan, Aparna Gupta, Oshani Seneviratne, Stacy Patterson
{"title":"Fed-RD: Privacy-Preserving Federated Learning for Financial Crime Detection","authors":"Md. Saikat Islam Khan, Aparna Gupta, Oshani Seneviratne, Stacy Patterson","doi":"arxiv-2408.01609","DOIUrl":"https://doi.org/arxiv-2408.01609","url":null,"abstract":"We introduce Federated Learning for Relational Data (Fed-RD), a novel\u0000privacy-preserving federated learning algorithm specifically developed for\u0000financial transaction datasets partitioned vertically and horizontally across\u0000parties. Fed-RD strategically employs differential privacy and secure\u0000multiparty computation to guarantee the privacy of training data. We provide\u0000theoretical analysis of the end-to-end privacy of the training algorithm and\u0000present experimental results on realistic synthetic datasets. Our results\u0000demonstrate that Fed-RD achieves high model accuracy with minimal degradation\u0000as privacy increases, while consistently surpassing benchmark results.","PeriodicalId":501309,"journal":{"name":"arXiv - CS - Computational Engineering, Finance, and Science","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141936377","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Impact of Major Health Events on Pharmaceutical Stocks: A Comprehensive Analysis Using Macroeconomic and Market Indicators","authors":"Morteza Maleki, SeyedAli Ghahari","doi":"arxiv-2408.01883","DOIUrl":"https://doi.org/arxiv-2408.01883","url":null,"abstract":"This study investigates the impact of significant health events on\u0000pharmaceutical stock performance, employing a comprehensive analysis\u0000incorporating macroeconomic and market indicators. Using Ordinary Least Squares\u0000(OLS) regression, we evaluate the effects of thirteen major health events since\u00002000, including the Anthrax attacks, SARS outbreak, H1N1 pandemic, and COVID-19\u0000pandemic, on the pharmaceutical sector. The analysis covers different phases of\u0000each event beginning, peak, and ending to capture their temporal influence on\u0000stock prices. Our findings reveal distinct patterns in stock performance,\u0000driven by market reactions to the initial news, peak impact, and eventual\u0000resolution of these crises. We also examine scenarios with and without key\u0000macroeconomic (MA) and market (MI) indicators to isolate their contributions.\u0000This detailed examination provides valuable insights for investors,\u0000policymakers, and stakeholders in understanding the interplay between major\u0000health events and health market dynamics, guiding better decision-making during\u0000future health related disruptions.","PeriodicalId":501309,"journal":{"name":"arXiv - CS - Computational Engineering, Finance, and Science","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141936376","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Wenyan Xu, Rundong Wang, Chen Li, Yonghong Hu, Zhonghua Lu
{"title":"HRFT: Mining High-Frequency Risk Factor Collections End-to-End via Transformer","authors":"Wenyan Xu, Rundong Wang, Chen Li, Yonghong Hu, Zhonghua Lu","doi":"arxiv-2408.01271","DOIUrl":"https://doi.org/arxiv-2408.01271","url":null,"abstract":"In quantitative trading, it is common to find patterns in short term volatile\u0000trends of the market. These patterns are known as High Frequency (HF) risk\u0000factors, serving as key indicators of future stock price volatility.\u0000Traditionally, these risk factors were generated by financial models relying\u0000heavily on domain-specific knowledge manually added rather than extensive\u0000market data. Inspired by symbolic regression (SR), which infers mathematical\u0000laws from data, we treat the extraction of formulaic risk factors from\u0000high-frequency trading (HFT) market data as an SR task. In this paper, we\u0000challenge the manual construction of risk factors and propose an end-to-end\u0000methodology, Intraday Risk Factor Transformer (IRFT), to directly predict\u0000complete formulaic factors, including constants. We use a hybrid\u0000symbolic-numeric vocabulary where symbolic tokens represent operators/stock\u0000features and numeric tokens represent constants. We train a Transformer model\u0000on the HFT dataset to generate complete formulaic HF risk factors without\u0000relying on a predefined skeleton of operators. It determines the general shape\u0000of the stock volatility law up to a choice of constants. We refine the\u0000predicted constants (a, b) using the Broyden Fletcher Goldfarb Shanno algorithm\u0000(BFGS) to mitigate non-linear issues. Compared to the 10 approaches in SRBench,\u0000a living benchmark for SR, IRFT gains a 30% excess investment return on the\u0000HS300 and SP500 datasets, with inference times orders of magnitude faster than\u0000theirs in HF risk factor mining tasks.","PeriodicalId":501309,"journal":{"name":"arXiv - CS - Computational Engineering, Finance, and Science","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141936378","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yuxiang Wei, Anees Abrol, James Lah, Deqiang Qiu, Vince D. Calhoun
{"title":"A deep spatio-temporal attention model of dynamic functional network connectivity shows sensitivity to Alzheimer's in asymptomatic individuals","authors":"Yuxiang Wei, Anees Abrol, James Lah, Deqiang Qiu, Vince D. Calhoun","doi":"arxiv-2408.00378","DOIUrl":"https://doi.org/arxiv-2408.00378","url":null,"abstract":"Alzheimer's disease (AD) progresses from asymptomatic changes to clinical\u0000symptoms, emphasizing the importance of early detection for proper treatment.\u0000Functional magnetic resonance imaging (fMRI), particularly dynamic functional\u0000network connectivity (dFNC), has emerged as an important biomarker for AD.\u0000Nevertheless, studies probing at-risk subjects in the pre-symptomatic stage\u0000using dFNC are limited. To identify at-risk subjects and understand alterations\u0000of dFNC in different stages, we leverage deep learning advancements and\u0000introduce a transformer-convolution framework for predicting at-risk subjects\u0000based on dFNC, incorporating spatial-temporal self-attention to capture brain\u0000network dependencies and temporal dynamics. Our model significantly outperforms\u0000other popular machine learning methods. By analyzing individuals with diagnosed\u0000AD and mild cognitive impairment (MCI), we studied the AD progression and\u0000observed a higher similarity between MCI and asymptomatic AD. The interpretable\u0000analysis highlights the cognitive-control network's diagnostic importance, with\u0000the model focusing on intra-visual domain dFNC when predicting asymptomatic AD\u0000subjects.","PeriodicalId":501309,"journal":{"name":"arXiv - CS - Computational Engineering, Finance, and Science","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141884035","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jonathan Stollberg, Tarun Gangwar, Oliver Weeger, Dominik Schillinger
{"title":"Multiscale topology optimization of functionally graded lattice structures based on physics-augmented neural network material models","authors":"Jonathan Stollberg, Tarun Gangwar, Oliver Weeger, Dominik Schillinger","doi":"arxiv-2408.00510","DOIUrl":"https://doi.org/arxiv-2408.00510","url":null,"abstract":"We present a new framework for the simultaneous optimiziation of both the\u0000topology as well as the relative density grading of cellular structures and\u0000materials, also known as lattices. Due to manufacturing constraints, the\u0000optimization problem falls into the class of NP-complete mixed-integer\u0000nonlinear programming problems. To tackle this difficulty, we obtain a relaxed\u0000problem from a multiplicative split of the relative density and a penalization\u0000approach. The sensitivities of the objective function are derived such that any\u0000gradient-based solver might be applied for the iterative update of the design\u0000variables. In a next step, we introduce a material model that is parametric in\u0000the design variables of interest and suitable to describe the isotropic\u0000deformation behavior of quasi-stochastic lattices. For that, we derive and\u0000implement further physical constraints and enhance a physics-augmented neural\u0000network from the literature that was formulated initially for rhombic\u0000materials. Finally, to illustrate the applicability of the method, we\u0000incorporate the material model into our computational framework and exemplary\u0000optimize two-and three-dimensional benchmark structures as well as a complex\u0000aircraft component.","PeriodicalId":501309,"journal":{"name":"arXiv - CS - Computational Engineering, Finance, and Science","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141884034","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Qionghua Liao, Guilong Li, Jiajie Yu, Ziyuan Gu, Wei Ma
{"title":"Online Prediction-Assisted Safe Reinforcement Learning for Electric Vehicle Charging Station Recommendation in Dynamically Coupled Transportation-Power Systems","authors":"Qionghua Liao, Guilong Li, Jiajie Yu, Ziyuan Gu, Wei Ma","doi":"arxiv-2407.20679","DOIUrl":"https://doi.org/arxiv-2407.20679","url":null,"abstract":"With the proliferation of electric vehicles (EVs), the transportation network\u0000and power grid become increasingly interdependent and coupled via charging\u0000stations. The concomitant growth in charging demand has posed challenges for\u0000both networks, highlighting the importance of charging coordination. Existing\u0000literature largely overlooks the interactions between power grid security and\u0000traffic efficiency. In view of this, we study the en-route charging station\u0000(CS) recommendation problem for EVs in dynamically coupled transportation-power\u0000systems. The system-level objective is to maximize the overall traffic\u0000efficiency while ensuring the safety of the power grid. This problem is for the\u0000first time formulated as a constrained Markov decision process (CMDP), and an\u0000online prediction-assisted safe reinforcement learning (OP-SRL) method is\u0000proposed to learn the optimal and secure policy by extending the PPO method. To\u0000be specific, we mainly address two challenges. First, the constrained\u0000optimization problem is converted into an equivalent unconstrained optimization\u0000problem by applying the Lagrangian method. Second, to account for the uncertain\u0000long-time delay between performing CS recommendation and commencing charging,\u0000we put forward an online sequence-to-sequence (Seq2Seq) predictor for state\u0000augmentation to guide the agent in making forward-thinking decisions. Finally,\u0000we conduct comprehensive experimental studies based on the Nguyen-Dupuis\u0000network and a large-scale real-world road network, coupled with IEEE 33-bus and\u0000IEEE 69-bus distribution systems, respectively. Results demonstrate that the\u0000proposed method outperforms baselines in terms of road network efficiency,\u0000power grid safety, and EV user satisfaction. The case study on the real-world\u0000network also illustrates the applicability in the practical context.","PeriodicalId":501309,"journal":{"name":"arXiv - CS - Computational Engineering, Finance, and Science","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-07-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141863329","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}