Information Systems最新文献

{"title":"Hands-on analysis of using large language models for the auto evaluation of programming assignments","authors":"Kareem Mohamed ,&nbsp;Mina Yousef ,&nbsp;Walaa Medhat ,&nbsp;Ensaf Hussein Mohamed ,&nbsp;Ghada Khoriba ,&nbsp;Tamer Arafa","doi":"10.1016/j.is.2024.102473","DOIUrl":"10.1016/j.is.2024.102473","url":null,"abstract":"<div><div>The increasing adoption of programming education necessitates efficient and accurate methods for evaluating students’ coding assignments. Traditional manual grading is time-consuming, often inconsistent, and prone to subjective biases. This paper explores the application of large language models (LLMs) for the automated evaluation of programming assignments. LLMs can use advanced natural language processing capabilities to assess code quality, functionality, and adherence to best practices, providing detailed feedback and grades. We demonstrate the effectiveness of LLMs through experiments comparing their performance with human evaluators across various programming tasks. Our study evaluates the performance of several LLMs for automated grading. Gemini 1.5 Pro achieves an exact match accuracy of 86% and a <span><math><mrow><mo>±</mo><mn>1</mn></mrow></math></span> accuracy of 98%. GPT-4o also demonstrates strong performance, with exact match and <span><math><mrow><mo>±</mo><mn>1</mn></mrow></math></span> accuracies of 69% and 97%, respectively. Both models correlate highly with human evaluations, indicating their potential for reliable automated grading. However, models such as Llama 3 70B and Mixtral 8 <span><math><mo>×</mo></math></span> 7B exhibit low accuracy and alignment with human grading, particularly in problem-solving tasks. These findings suggest that advanced LLMs are instrumental in scalable, automated educational assessment. Additionally, LLMs enhance the learning experience by offering personalized, instant feedback, fostering an iterative learning process. The findings suggest that LLMs could play a pivotal role in the future of programming education, ensuring scalability and consistency in evaluation.</div></div>","PeriodicalId":50363,"journal":{"name":"Information Systems","volume":"128 ","pages":"Article 102473"},"PeriodicalIF":3.0,"publicationDate":"2024-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142529772","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Influence maximization based on discrete particle swarm optimization on multilayer network","authors":"Saiwei Wang ,&nbsp;Wei Liu ,&nbsp;Ling Chen ,&nbsp;Shijie Zong","doi":"10.1016/j.is.2024.102466","DOIUrl":"10.1016/j.is.2024.102466","url":null,"abstract":"<div><div>Influence maximization (IM) aims to strategically select influential users to maximize information propagation in social networks. Most of the existing studies focus on IM in single-layer networks. However, we have observed that individuals often engage in multiple social platforms to fulfill various social needs. To make better use of this observation, we consider an extended problem of how to maximize influence spread in multilayer networks. The Multilayer Influence Maximization (MLIM) problem is different from the IM problem because information propagation behaves differently in multilayer networks compared to single-layer networks: users influenced on one layer may trigger the propagation of information on another layer. Our work successfully models the information propagation process as a Multilayer Independent Cascade model in multilayer networks. Based on the characteristics of this model, we introduce an approximation function called Multilayer Expected Diffusion Value (MLEDV) for it. However, the NP-hardness of the MLIM problem has posed significant challenges to our work. To tackle the issue, we devise a novel algorithm based on Discrete Particle Swarm Optimization. Our algorithm consists of two stages: 1) the candidate node selection, where we devise a novel centrality metric called Random connectivity Centrality to select candidate nodes, which assesses the importance of nodes from a connectivity perspective. 2)the seed selection, where we employ a discrete particle swarm algorithm to find seed nodes from the candidate nodes. We use MLEDV as a fitness function to measure the spreading power of candidate solutions in our algorithm. Additionally, we introduce a Neighborhood Optimization strategy to increase the convergence of the algorithm. We conduct experiments on four real-world networks and two self-built networks and demonstrate that our algorithms are effective for the MLIM problem.</div></div>","PeriodicalId":50363,"journal":{"name":"Information Systems","volume":"127 ","pages":"Article 102466"},"PeriodicalIF":3.0,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142420321","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Runtime integration of machine learning and simulation for business processes: Time and decision mining predictions","authors":"Francesca Meneghello ,&nbsp;Chiara Di Francescomarino ,&nbsp;Chiara Ghidini ,&nbsp;Massimiliano Ronzani","doi":"10.1016/j.is.2024.102472","DOIUrl":"10.1016/j.is.2024.102472","url":null,"abstract":"<div><div>Recent research in Computer Science has investigated the use of Deep Learning (DL) techniques to complement outcomes or decisions within a Discrete Event Simulation (DES) model. The main idea of this combination is to maintain a white box simulation model complement it with information provided by DL models to overcome the unrealistic or oversimplified assumptions of traditional DESs. State-of-the-art techniques in BPM combine Deep Learning and Discrete Event Simulation in a post-integration fashion: first an entire simulation is performed, and then a DL model is used to add waiting times and processing times to the events produced by the simulation model.</div><div>In this paper, we aim at taking a step further by introducing <span>Rims</span> (Runtime Integration of Machine Learning and Simulation). Instead of complementing the outcome of a complete simulation with the results of predictions a posteriori, <span>Rims</span> provides a tight integration of the predictions of the DL model <em>at runtime</em> during the simulation. This runtime-integration enables us to fully exploit the specific predictions while respecting simulation execution, thus enhancing the performance of the overall system both w.r.t. the single techniques (Business Process Simulation and DL) separately and the post-integration approach. In particular, the runtime integration ensures the accuracy of intercase features for time prediction, such as the number of ongoing traces at a given time, by calculating them during directly the simulation, where all traces are executed in parallel. Additionally, it allows for the incorporation of online queue information in the DL model and enables the integration of other predictive models into the simulator to enhance decision point management within the process model. These enhancements improve the performance of <span>Rims</span> in accurately simulating the real process in terms of control flow, as well as in terms of time and congestion dimensions. Especially in process scenarios with significant congestion – when a limited availability of resources leads to significant event queues for their allocation – the ability of <span>Rims</span> to use queue features to predict waiting times allows it to surpass the state-of-the-art. We evaluated our approach with real-world and synthetic event logs, using various metrics to assess the simulation model’s quality in terms of control-flow, time, and congestion dimensions.</div></div>","PeriodicalId":50363,"journal":{"name":"Information Systems","volume":"128 ","pages":"Article 102472"},"PeriodicalIF":3.0,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142442017","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Business process simulation: Probabilistic modeling of intermittent resource availability and multitasking behavior","authors":"Orlenys López-Pintado,&nbsp;Marlon Dumas","doi":"10.1016/j.is.2024.102471","DOIUrl":"10.1016/j.is.2024.102471","url":null,"abstract":"<div><div>In business process simulation, resource availability is typically modeled by assigning a calendar to each resource, e.g., Monday–Friday, 9:00–18:00. Resources are assumed to be always available during each time slot in their availability calendar. This assumption often becomes invalid due to interruptions, breaks, or time-sharing across processes. In other words, existing approaches fail to capture intermittent availability. Another limitation of existing approaches is that they either do not consider multitasking behavior, or if they do, they assume that resources always multitask (up to a maximum capacity) whenever available. However, studies have shown that the multitasking patterns vary across days. This paper introduces a probabilistic approach to model resource availability and multitasking behavior for business process simulation. In this approach, each time slot in a resource calendar has an associated availability probability and a multitasking probability per multitasking level. For example, a resource may be available on Fridays between 14:00–15:00 with 90% probability, and given that they are performing one task during this slot, they may take on a second concurrent task with 60% probability. We propose algorithms to discover probabilistic calendars and probabilistic multitasking capacities from event logs. An evaluation shows that, with these enhancements, simulation models discovered from event logs better replicate the distribution of activities and cycle times, relative to approaches with crisp calendars and monotasking assumptions.</div></div>","PeriodicalId":50363,"journal":{"name":"Information Systems","volume":"127 ","pages":"Article 102471"},"PeriodicalIF":3.0,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142432610","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Modeling higher-order social influence using multi-head graph attention autoencoder","authors":"Elnaz Meydani ,&nbsp;Christoph Duesing ,&nbsp;Matthias Trier","doi":"10.1016/j.is.2024.102474","DOIUrl":"10.1016/j.is.2024.102474","url":null,"abstract":"<div><div>Recommender systems are powerful tools developed to mitigate information overload in e-commerce platforms. Social recommender systems leverage social relations among users to predict their preferences. Recently, graph neural networks have been utilized for social recommendations, modeling user-user social relations and user–item interactions as graph-structured data. Despite their improvement over traditional systems, most existing social recommender systems exploit only first-order social relations and overlook the importance of social influence diffusion from higher-order neighbors in social networks. Additionally, these techniques often treat all neighboring nodes equally, without highlighting the most influential ones. To address these challenges, we introduce GATE-SR, a novel model that leverages a multi-head graph attention autoencoder to capture indirect social influence from higher-order neighbors while emphasizing the most relevant users. Moreover, we incorporate implicit social connections derived from coherent communities within the network. While GATE-SR performs comparably to baseline models in rich data environments, its strength lies in excelling at cold-start scenarios—where other models often fall short. This focus on cold-start performance aligns with our goal of building a robust recommender system for real-world challenges. Through extensive experiments on three real-world datasets, we demonstrate that GATE-SR outperforms several state-of-the-art baselines in cold-start scenarios. These results highlight the crucial role of accentuating the most influential neighbors, both explicit and implicit, when modeling higher-order social connections for more accurate recommendations.</div></div>","PeriodicalId":50363,"journal":{"name":"Information Systems","volume":"128 ","pages":"Article 102474"},"PeriodicalIF":3.0,"publicationDate":"2024-10-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142445834","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Exploiting explicit item–item correlations from knowledge graphs for enhanced sequential recommendation","authors":"Yanlin Zhang ,&nbsp;Yuchen Shi ,&nbsp;Deqing Yang ,&nbsp;Xiaodong Gu","doi":"10.1016/j.is.2024.102470","DOIUrl":"10.1016/j.is.2024.102470","url":null,"abstract":"<div><div>In recent years, the research of employing knowledge graphs (KGs) in sequential recommendation (SR) has received a lot of attention, since the side information extracted from KGs, especially the information of the correlations between items, indeed helps the SR models achieve better performance. However, many previous KG-based SR models tend to introduce some noise information when learning item embeddings, or insufficiently fuse item–item correlations into their sequential modeling, thus limiting their performance improvements. In this paper, we propose a <strong>D</strong>istance-<strong>A</strong>ware <strong>K</strong>nowledge-based <strong>S</strong>equential <strong>R</strong>ecommendation model (<strong>DAKSR</strong>), which exploits the explicit item–item correlations from KGs to achieve enhanced SR. Specifically, as one critical component in our DAKSR, the <em>distance score matrix</em> (DSM) is first obtained to indicate the correlations between items, and then leveraged in the following three major modules of DAKSR. First, in the Item-Set Embedding layer (ISE) all item embeddings are learned based on DSM, in which the noise information is eliminated effectively. Meanwhile, the Knowledge-Infused Transformer (KIT) incorporates DSM into its attention mechanism to improve the feature extraction. Furthermore, the Knowledge Contrastive Learning module (KCL) also leverages the item–item correlations presented in DSM to generate two credible sequence views, which are used to refine sample representations through a contrastive learning strategy, and thus improve the model’s robustness. Our extensive experiments on three SR benchmarks obviously demonstrate our DAKSR’s superior performance over the state-of-the-art (SOTA) KG-based recommendation models. The implementation of our DAKSR is available at <span><span>https://github.com/Easonsi/DAKSR</span><svg><path></path></svg></span> for reproducing our experiment results conveniently.</div></div>","PeriodicalId":50363,"journal":{"name":"Information Systems","volume":"128 ","pages":"Article 102470"},"PeriodicalIF":3.0,"publicationDate":"2024-10-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142529775","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Advances on data management systems","authors":"Ladjel Bellatreche,&nbsp;Marlon Dumas,&nbsp;Panagiotis Karras,&nbsp;Raimundas Matulevičius,&nbsp;Silvia Chiusano,&nbsp;Tania Cerquitelli,&nbsp;Robert Wrembel","doi":"10.1016/j.is.2024.102467","DOIUrl":"10.1016/j.is.2024.102467","url":null,"abstract":"","PeriodicalId":50363,"journal":{"name":"Information Systems","volume":"127 ","pages":"Article 102467"},"PeriodicalIF":3.0,"publicationDate":"2024-10-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142444592","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Special Issue of CAiSE 2023 Best Papers","authors":"Iris Reinhartz-Berger ,&nbsp;Marta Indulska","doi":"10.1016/j.is.2024.102469","DOIUrl":"10.1016/j.is.2024.102469","url":null,"abstract":"","PeriodicalId":50363,"journal":{"name":"Information Systems","volume":"127 ","pages":"Article 102469"},"PeriodicalIF":3.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142444620","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Finding meaningful paths in heterogeneous graphs with PathWays","authors":"Nelly Barret ,&nbsp;Antoine Gauquier ,&nbsp;Jia-Jean Law ,&nbsp;Ioana Manolescu","doi":"10.1016/j.is.2024.102463","DOIUrl":"10.1016/j.is.2024.102463","url":null,"abstract":"<div><div>Graphs, and notably RDF graphs, are a prominent way of sharing data. As data usage democratizes, users need help figuring out the useful content of a graph dataset. In particular, journalists with whom we collaborate are interested in identifying, in a graph, the <em>connections between entities</em>, e.g., people, organizations, emails, etc. We present a novel method for exploring data graphs through <em>their data paths connecting Named Entities</em> (NEs, in short); each data path leads to a tabular-looking set of results. NEs are extracted from the data through dedicated Information Extraction modules. Our method builds upon the pre-existing ConnectionLens platform and follow-up work in the Abstra project, which builds simple, visual ER-style summaries of semi-structured data. The contribution of the present work, and its novelty, is twofold. First, we propose a novel analysis of entity-to-entity paths contained in datasets of any nature, and propose a new method for ranking paths, leveraging a novel Information Extraction (IE) module we built on top of ChatGPT. Second, we present an efficient approach to enumerate and compute NE paths, based on an algorithm which automatically recommends sub-paths to materialize, and rewrites the path queries using these subpaths. Our experiments demonstrate the interest of NE paths and the efficiency of our method for computing and ranking them.</div></div>","PeriodicalId":50363,"journal":{"name":"Information Systems","volume":"127 ","pages":"Article 102463"},"PeriodicalIF":3.0,"publicationDate":"2024-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142420464","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Using AI explainable models and handwriting/drawing tasks for psychological well-being","authors":"Francesco Prinzi ,&nbsp;Pietro Barbiero ,&nbsp;Claudia Greco ,&nbsp;Terry Amorese ,&nbsp;Gennaro Cordasco ,&nbsp;Pietro Liò ,&nbsp;Salvatore Vitabile ,&nbsp;Anna Esposito","doi":"10.1016/j.is.2024.102465","DOIUrl":"10.1016/j.is.2024.102465","url":null,"abstract":"<div><div>This study addresses the increasing threat to Psychological Well-Being (PWB) posed by Depression, Anxiety, and Stress conditions. Machine learning methods have shown promising results for several psychological conditions. However, the lack of transparency in existing models impedes practical application. The study aims to develop explainable machine learning models for depression, anxiety and stress prediction, focusing on features extracted from tasks involving handwriting and drawing.</div><div>Two hundred patients completed the Depression, Anxiety, and Stress Scale (DASS-21) and performed seven tasks related to handwriting and drawing. Extracted features, encompassing pressure, stroke pattern, time, space, and pen inclination, were used to train the explainable-by-design Entropy-based Logic Explained Network (e-LEN) model, employing first-order logic rules for explanation. Performance comparison was performed with XGBoost, enhanced by the SHAP explanation method.</div><div>The trained models achieved notable accuracy in predicting depression (0.749 ±0.089), anxiety (0.721 ±0.088), and stress (0.761 ±0.086) through 10-fold cross-validation (repeated 20 times). The e-LEN model’s logic rules facilitated clinical validation, uncovering correlations with existing clinical literature. While performance remained consistent for depression and anxiety on an independent test dataset, a slight degradation was observed for stress prediction in the test task.</div></div>","PeriodicalId":50363,"journal":{"name":"Information Systems","volume":"127 ","pages":"Article 102465"},"PeriodicalIF":3.0,"publicationDate":"2024-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142420465","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}