Journal of Big Data最新文献

{"title":"Customizing and fine-tuning opioid overdose machine learning models for community-specific dashboards.","authors":"Daniel R Harris, Aaron Mullen, Peter Rock, Carmen Canedo, Katherine Thompson, Michelle R Lofwall, Cody Bumgardner, Jeffery Talbert, Svetla Slavova","doi":"10.1186/s40537-026-01507-4","DOIUrl":"10.1186/s40537-026-01507-4","url":null,"abstract":"<p><p>Drug overdose is a leading cause of death in the United States. Targeted community interventions and allocation of resources for reducing overdose deaths require timely and actionable data to make a meaningful public health impact. We present our work on tuning geographic-specific machine learning forecast models to predict the number of opioid-involved overdoses involving Emergency Medical Services (EMS) in Kentucky, with a focus on optimizing covariates representing measurable opioid overdose risk and protective factors. We aggregated suspected opioid-involved overdoses in EMS data into monthly totals by county between January 2017 and September 2025. We further grouped these counties into larger regions, representing Area Development Districts (ADDs), service areas for Community Mental Health Centers (CMHCs), and local health departments (LHDs). We constructed long-term time-series forecasting models (NLinear LTSF) to predict opioid-involved overdoses three months into the future. We tested the impact of using eight covariate data sources: Kentucky State Police drug seizures, Medicaid claims, community naloxone distribution, Kentucky Department of Corrections intakes and releases, social determinants of health data, location of treatment venues, and weather. We evaluated our models using both an unseen 10% test set and an expanding window strategy, where we simulated predicting historical data and retrained the model using data available for every three-month window. We embedded predictions into community-specific dashboards where end-users can select individual counties, or larger regions covered by distinct ADDs, CMHCs, or LHDs. The dashboards are integrated in Kentucky's Rapid Actionable Data for Opioid Response system designed to deliver ongoing near-real time analytics and forecasting at state and local levels.</p>","PeriodicalId":15158,"journal":{"name":"Journal of Big Data","volume":" ","pages":""},"PeriodicalIF":10.8,"publicationDate":"2026-07-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13618779/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149074766","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Multi-omics Mendelian randomization revealing SEMA7A as potential drug target for facial skin aging","authors":"Xueyao Cai, Weidong Li, Wenjun Shi, Yuchen Cai, Jianda Zhou","doi":"10.1186/s40537-025-01355-8","DOIUrl":"https://doi.org/10.1186/s40537-025-01355-8","url":null,"abstract":"The study of facial skin aging has attracted significant attention not only for its esthetic implications but also for its potential to shed light on the mechanisms underlying the development of age-related diseases. The purpose of this research was to systematically investigate potential drug targets for facial skin aging through multi-omics genetic methods. We utilized expression quantitative trait loci (eQTLs) and protein quantitative trait loci (pQTLs) as instruments to explore the therapeutic potential for facial skin aging. This was supplemented with transcriptome analysis and in vitro experiments, including siRNA knockdown in human HaCaT and NIH 3T3 cells, which confirmed the genetic results. Additionally, we conducted two-sample Mendelian randomization (MR) to investigate the causal effect of facial aging on skin neoplasms and performed metabolome-wide MR to screen for metabolic biomarkers associated with facial aging. Through integrated analysis of eQTLs and colocalization, five potential druggable genes were identified, including BRSK2, IL20RB, NEK10, RAB35, and SEMA7A. Of these, SEMA7A showed strong evidence with facial skin aging through subsequent pQTL and colocalization analysis. Additionally, genetically determined facial skin aging exhibited a significant causal effect on basal cell carcinoma (BCC) (OR = 3.973, 95% CI: 1.284, 12.290, p = 0.016 [inverse-variance weighted, IVW]), while no causal effect was found on malignant melanoma and squamous cell carcinoma. Subsequent transcriptome and in vitro experiment suggested that SEMA7A played an important role in the pathogenesis of skin aging. The metabolome-wide MR identified several circulating metabolic traits as biomarkers for facial skin aging, implicating lipoprotein subfractions (HDL, LDL, and VLDL) and fatty acid profiles. This study demonstrates the genetic significance of targeting SEMA7A for facial skin aging, providing new insights into the disease mechanism and aiding the development of future drug therapies.","PeriodicalId":15158,"journal":{"name":"Journal of Big Data","volume":"13 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2026-01-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147898555","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Comprehensive representation of health-related phenotypes in one million dogs using topic modelling of electronic health records.","authors":"Peter-John Mäntylä Noble, Sean Oliver Farrell, Noura Al-Moubayed, Alan David Radford","doi":"10.1186/s40537-026-01365-0","DOIUrl":"10.1186/s40537-026-01365-0","url":null,"abstract":"<p><p>Historically, veterinary studies screening for breed, age and sex predisposition to disease have relied on collating small-scale studies of clinical datasets. The availability of larger datasets through groups such as the Small Animal Veterinary Surveillance Network (SAVSNET) promise access to information regarding a wide range of clinical presentations at scale, however, methodological limitations surrounding the extraction of specific disease information or screening for disease predispositions result in a substantial reduction in the number of animals studied. These studies often address very focused hypotheses - only leveraging a small fraction of the intrinsic value of the data at any one time. Here, we implemented an unsupervised machine learning methodology, creating a representation of a large volume of clinical notes collected by SAVSNET from veterinary practices across the UK. We utilise BERTopic, a topic-modelling tool based on Bidirectional Encoder Representations using Transformers (BERT) architecture, and show it is able to surface known phenotypes, such as breed predispositions to hypoadrenocorticism, diabetes mellitus and mitral valve disease, as well as potential novel patterns of disease phenotypes. This scalable and granular modelling technique facilitates the rapid interrogation of large clinical datasets, enabling the identification of a broad range of phenotypes within the population and the early detection of temporal changes indicative of emerging infectious or environmental diseases.</p><p><strong>Supplementary information: </strong>The online version contains supplementary material available at 10.1186/s40537-026-01365-0.</p>","PeriodicalId":15158,"journal":{"name":"Journal of Big Data","volume":"13 1","pages":"50"},"PeriodicalIF":10.8,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13035608/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147592339","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"CardiaTics: An explainable AI integrated heart disease diagnosis model with feature engineering and stacked ensemble approach.","authors":"Partho Ghose, Hasan Jamil","doi":"10.1186/s40537-026-01395-8","DOIUrl":"10.1186/s40537-026-01395-8","url":null,"abstract":"<p><p>Heart disease is a leading global cause of morbidity and mortality. Accurate and prompt diagnoses are crucial for its effective prevention and management. Integrating multiple machine learning algorithms, this research introduces a stacked ensemble machine learning model, called CardiaTics (stands for Cardiac DiagnosTics), toward improving heart disease detection. We detect outliers and remove them as a first-step to ensure data quality and maintain integrity. Ten distinct machine learning algorithms are then individually applied, culminating in the creation of a stacked ensemble model. We use feature engineering to refine the model further applying three well-known techniques -Pearson correlation, Chi-Square Test (Chi-2), and Recursive Feature Elimination. The implementation of these techniques on the benchmark dataset results in an optimized feature set. Experimental results show that CardiaTics delivers 89.3% accuracy on raw data, and significantly improves its accuracy after feature selection to 93.3%, outperforming the individual classifiers. However, can human professionals rely on algorithms for prediction when the underlying process is not fully understood? To address concerns regarding interpretability, trust, and transparency in black-box predictions, we propose utilizing SHapley Additive exPlanations (SHAP) and Explain Like I'm 5 (ELI5) in the second phase to elucidate feature importance in our model. The SHAP summary plots of CardiaTics reveal that the positive and negative contributors to heart disease are comparable, thereby enhancing the model's interpretability and reliability and helping refine the decision-making process.</p>","PeriodicalId":15158,"journal":{"name":"Journal of Big Data","volume":"13 1","pages":"59"},"PeriodicalIF":10.8,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13068695/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147673668","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Advancing hospital healthcare: achieving IoT-based secure health monitoring through multilayer machine learning","authors":"Ke Qi","doi":"10.1186/s40537-024-01038-w","DOIUrl":"https://doi.org/10.1186/s40537-024-01038-w","url":null,"abstract":"Abstract Background Data based clinical decision support system is a boon for health care monitoring. Smart healthcare monitoring systems play a vital role in the early diagnosis and detection of the physical and mental health of patients. The smart clinical IoT (C-IoT) systems are data-driven and provide efficient support for this purpose. Purpose There is a need to have a secure, accurate, and efficient HCM system that is capable of processing large amounts of patient data for timely diagnosis and detection of various health complications. Traditional ways of migration are imprecise, less secure, and do not cover all angles necessary in the contemporary healthcare environment. Because of this, the conceptual IoT-based secure health monitoring system employs machine learning algorithms for enhanced accuracy. Method This study presents the conjugate applications of machine learning algorithms with the cloud-based C-IoT model systems. This model is a lightweight encryption block model that maintains provisional security for health and clinical data. It assists in patient’s health issues which are diagnosed with the existing database of the history of that patient and proper measures are taken with proper diagnosis and using this prediction model. The health status is diagnosed from the pre-historical database of the patient’s database. Results This cloud-based smart C-IoT system shows the results approximately with 91% accuracy while using Artificial Neural Network (ANN) algorithms. This smart C-IoT-based health issue diagnostic model is one step ahead toward the modernization of society 5.0. Future prospects The proposed IoT-based secure health monitoring system expands the surgeries of health care by achieving a high diagnostic accuracy of 91% employing ANN algorithms, the excellence of which is founded on data intensity with prior patient data, and the data security by lightweight encryption algorithms. Aligned with Society 5.0, it brings new, friendly, and efficient features to healthcare that replace many existing methods with better ones in terms of precision, security, and coverage.","PeriodicalId":15158,"journal":{"name":"Journal of Big Data","volume":"12 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-01-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://journalofbigdata.springeropen.com/counter/pdf/10.1186/s40537-024-01038-w","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147916340","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"<i>F</i>u<i>n</i>Da: scalable serverless data analytics and in situ query processing.","authors":"Elyes Lounissi, Suvam Kumar Das, Ronnit Peter, Xiaozheng Zhang, Suprio Ray, Lianyin Jia","doi":"10.1186/s40537-025-01141-6","DOIUrl":"10.1186/s40537-025-01141-6","url":null,"abstract":"<p><p>The pay-what-you-use model of serverless Cloud computing (or serverless, for short) offers significant benefits to the users. This computing paradigm is ideal for short running ephemeral tasks, however, it is not suitable for stateful long running tasks, such as complex data analytics and query processing. We propose <i>F</i>u<i>n</i>Da, an on-premises serverless data analytics framework, which extends our previously proposed system for unified data analytics and in situ SQL query processing called DaskDB. Unlike existing serverless solutions, which struggle with stateful and long running data analytics tasks, <i>F</i>u<i>n</i>Da overcomes their limitations. Our ongoing research focuses on developing a robust architecture for <i>F</i>u<i>n</i>Da, enabling true serverless in on-premises environments, while being able to operate on a public Cloud, such as AWS Cloud. We have evaluated our system on several benchmarks with different scale factors. Our experimental results in both on-premises and AWS Cloud settings demonstrate <i>F</i>u<i>n</i>Da's ability to support automatic scaling, low-latency execution of data analytics workloads, and more flexibility to serverless users.</p>","PeriodicalId":15158,"journal":{"name":"Journal of Big Data","volume":"12 1","pages":"116"},"PeriodicalIF":10.8,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12064580/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143991480","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"UniqueNOSD: a novel framework for NoSQL over SQL databases.","authors":"Abdulrauf A Gidado, C I Ezeife","doi":"10.1186/s40537-025-01307-2","DOIUrl":"https://doi.org/10.1186/s40537-025-01307-2","url":null,"abstract":"<p><p>To date, most large corporations still have their core solutions on relational databases but only use non-relational (i.e. NoSQL) database management systems (DBMS) for their non-core systems that favour availability and scalability through partitioning while trading off consistency. NoSQL systems are built based on the CAP (i.e., Consistency, Availability and Partitioning) database theorem, which trades off one of these features while maintaining the others. The need for systems availability and scalability drives the use of NoSQL, while the lack of consistency and robust query engines as obtainable in relational databases, impede their usage. To mitigate these drawbacks, researchers and companies like Amazon, Google, and Facebook run 'SQL over NoSQL' systems such as Dynamo, Google's Spanner, Memcache, Zidian, Apache Hive and SparkSQL. These systems create a query engine layer over NoSQL systems but suffer from data redundancy and lack consistency obtainable in relational DBMS. Also, their query engine is not relational complete because they cannot process all relational algebra-based queries as obtainable in a relational database. In this paper, we present a 'Unique NoSQL over SQL Database' (UniqueNOSD) system, an extension of NOSD and an inverse of existing approaches. This approach is motivated by the need for existing systems to fully deploy NoSQL data store functionalities without the limitation of building an extra SQL layer for querying. To allow appropriate storage and retrieval of data on document-based NoSQL databases without data redundancy and inconsistency while encouraging both horizontal and vertical partitioning, we propose NoSQL over SQL Block as a Value ([Formula: see text]) data storage strategy. Unlike relational database model where a relation is represented as [Formula: see text], with a key attribute [Formula: see text] and [Formula: see text] is the primary key to the set of attributes [Formula: see text] of the relation, in [Formula: see text] (represented as a tuple (<i>K</i>, <i>B</i>) where <i>K</i> means key and <i>B</i> means block). We represent a relation as [Formula: see text] with a key attribute <i>K</i> and a set of <i>n</i> relations (i.e., <i>r</i>) called blocks <i>B</i> and each <i>r</i> [Formula: see text] contains a set of its own attributes and is denoted as [Formula: see text] with a key attribute <i>k</i> and a set of <i>n</i> attributes typical to a relational model. The relations [Formula: see text] in <i>R</i> of [Formula: see text] are related through foreign key relationships. Using existing benchmark systems of 'SQL over NoSQL', relational databases and real-life datasets for our experiments, we demonstrated that our NoSQL over SQL system outperforms existing relational databases, SQL over NoSQL systems and is novel in ensuring data consistency, scalability, query execution and improving data storage and retrieval in large database systems without data loss and enhancing improved performan","PeriodicalId":15158,"journal":{"name":"Journal of Big Data","volume":"12 1","pages":"255"},"PeriodicalIF":6.4,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12628391/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145563910","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Optimizing poultry audio signal classification with deep learning and burn layer fusion","authors":"Esraa Hassan, Samar Elbedwehy, Mahmoud Y. Shams, Tarek Abd El-Hafeez, Nora El-Rashidy","doi":"10.1186/s40537-024-00985-8","DOIUrl":"https://doi.org/10.1186/s40537-024-00985-8","url":null,"abstract":"<p>This study introduces a novel deep learning-based approach for classifying poultry audio signals, incorporating a custom Burn Layer to enhance model robustness. The methodology integrates digital audio signal processing, convolutional neural networks (CNNs), and the innovative Burn Layer, which injects controlled random noise during training to reinforce the model's resilience to input signal variations. The proposed architecture is streamlined, with convolutional blocks, densely connected layers, dropout, and an additional Burn Layer to fortify robustness. The model demonstrates efficiency by reducing trainable parameters to 191,235, compared to traditional architectures with over 1.7 million parameters. The proposed model utilizes a Burn Layer with burn intensity as a parameter and an Adamax optimizer to optimize and address the overfitting problem. Thorough evaluation using six standard classification metrics showcases the model's superior performance, achieving exceptional sensitivity (96.77%), specificity (100.00%), precision (100.00%), negative predictive value (NPV) (95.00%), accuracy (98.55%), F1 score (98.36%), and Matthew’s correlation coefficient (MCC) (95.88%). This research contributes valuable insights into the fields of audio signal processing, animal health monitoring, and robust deep-learning classification systems. The proposed model presents a systematic approach for developing and evaluating a deep learning-based poultry audio classification system. It processes raw audio data and labels to generate digital representations, utilizes a Burn Layer for training variability, and constructs a CNN model with convolutional blocks, pooling, and dense layers. The model is optimized using the Adamax algorithm and trained with data augmentation and early-stopping techniques. Rigorous assessment on a test dataset using standard metrics demonstrates the model's robustness and efficiency, with the potential to significantly advance animal health monitoring and disease detection through audio signal analysis.</p>","PeriodicalId":15158,"journal":{"name":"Journal of Big Data","volume":"23 1","pages":""},"PeriodicalIF":8.1,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142253832","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Machine learning and deep learning models based grid search cross validation for short-term solar irradiance forecasting","authors":"Doaa El-Shahat, Ahmed Tolba, Mohamed Abouhawwash, Mohamed Abdel-Basset","doi":"10.1186/s40537-024-00991-w","DOIUrl":"https://doi.org/10.1186/s40537-024-00991-w","url":null,"abstract":"<p>In late 2023, the United Nations conference on climate change (COP28), which was held in Dubai, encouraged a quick move from fossil fuels to renewable energy. Solar energy is one of the most promising forms of energy that is both sustainable and renewable. Generally, photovoltaic systems transform solar irradiance into electricity. Unfortunately, instability and intermittency in solar radiation can lead to interruptions in electricity production. The accurate forecasting of solar irradiance guarantees sustainable power production even when solar irradiance is not present. Batteries can store solar energy to be used during periods of solar absence. Additionally, deterministic models take into account the specification of technical PV systems and may be not accurate for low solar irradiance. This paper presents a comparative study for the most common Deep Learning (DL) and Machine Learning (ML) algorithms employed for short-term solar irradiance forecasting. The dataset was gathered in Islamabad during a five-year period, from 2015 to 2019, at hourly intervals with accurate meteorological sensors. Furthermore, the Grid Search Cross Validation (GSCV) with five folds is introduced to ML and DL models for optimizing the hyperparameters of these models. Several performance metrics are used to assess the algorithms, such as the <i>Adjusted R</i>^<i>2</i><i> score</i>, <i>Normalized Root Mean Square Error</i> (NRMSE), <i>Mean Absolute Deviation</i> (MAD), <i>Mean Absolute Error</i> (MAE) and <i>Mean Square Error</i> (MSE). The statistical analysis shows that CNN-LSTM outperforms its counterparts of nine well-known DL models with <i>Adjusted R</i>^<i>2</i><i> score</i> value of 0.984. For ML algorithms, gradient boosting regression is an effective forecasting method with <i>Adjusted R</i>^<i>2</i><i> score</i> value of 0.962, beating its rivals of six ML models. Furthermore, SHAP and LIME are examples of explainable Artificial Intelligence (XAI) utilized for understanding the reasons behind the obtained results.</p>","PeriodicalId":15158,"journal":{"name":"Journal of Big Data","volume":"13 1","pages":""},"PeriodicalIF":8.1,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142253831","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Shielding networks: enhancing intrusion detection with hybrid feature selection and stack ensemble learning","authors":"Ali Mohammed Alsaffar, Mostafa Nouri-Baygi, Hamed M. Zolbanin","doi":"10.1186/s40537-024-00994-7","DOIUrl":"https://doi.org/10.1186/s40537-024-00994-7","url":null,"abstract":"<p>The frequent usage of computer networks and the Internet has made computer networks vulnerable to numerous attacks, highlighting the critical need to enhance the precision of security mechanisms. One of the most essential measures to safeguard networking resources and infrastructures is an intrusion detection system (IDS). IDSs are widely used to detect, identify, and track malicious threats. Although various machine learning algorithms have been used successfully in IDSs, they are still suffering from low prediction performances. One reason behind the low accuracy of IDSs is that existing network traffic datasets have high computational complexities that are mainly caused by redundant, incomplete, and irrelevant features. Furthermore, standalone classifiers exhibit restricted classification performance and typically fail to produce satisfactory outcomes when dealing with imbalanced, multi-category traffic data. To address these issues, we propose an efficient intrusion detection model, which is based on hybrid feature selection and stack ensemble learning. Our hybrid feature selection method, called MI-Boruta, combines mutual information (MI) as a filter method and the Boruta algorithm as a wrapper method to determine optimal features from our datasets. Then, we apply stacked ensemble learning by using random forest (RF), Catboost, and XGBoost algorithms as base learners with multilayer perceptron (MLP) as meta-learner. We test our intrusion detection model on two widely recognized benchmark datasets, namely UNSW-NB15 and CICIDS2017. We show that our proposed IDS outperforms existing IDSs in almost all performance criteria, including accuracy, recall, precision, F1-Score, false positive rate, true positive rate, and error rate.</p>","PeriodicalId":15158,"journal":{"name":"Journal of Big Data","volume":"19 1","pages":""},"PeriodicalIF":8.1,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142253792","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}