Amina Shahzadi, Ting Wang, Matthew Parry, Mark Bebbington
{"title":"Modelling time-inhomogeneous incomplete records of point processes using variants of hidden Markov models","authors":"Amina Shahzadi, Ting Wang, Matthew Parry, Mark Bebbington","doi":"10.1007/s11634-025-00632-x","DOIUrl":"10.1007/s11634-025-00632-x","url":null,"abstract":"<div><p>Many point processes such as earthquakes or volcanic eruptions have incomplete records with the degree of incompleteness varying over time. For these point processes, the number of missing events between each pair of consecutively observed events can be a random variable that may depend on time, effecting the estimation of parameters or hazard. Such incomplete point processes can be modelled by compound renewal processes where the sum of renewal processes is a random variable because of random variable number of missing events. We propose shifted compound Poisson-Gamma and time-dependent shifted compound Poisson-Gamma renewal processes. Since the number of missing events can be regarded as an unobserved process, the proposed renewal processes are introduced to use in the framework of different types of homogeneous and inhomogeneous hidden Markov models to model the time-dependent variable number of missing events between each pair of consecutively observed events of incomplete point processes. Simulation experiments are employed to check the performance of proposed renewal processes with hidden Markov models. We apply the proposed models to the large magnitude explosive volcanic eruptions database to analyze the time-dependent incompleteness and demonstrate how we estimate the completeness of the record and the future hazard rate.</p></div>","PeriodicalId":49270,"journal":{"name":"Advances in Data Analysis and Classification","volume":"20 2","pages":"663 - 691"},"PeriodicalIF":2.1,"publicationDate":"2025-04-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s11634-025-00632-x.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148314724","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Maria Giovanna Ranalli, Fulvia Pennoni, Francesco Bartolucci, Antonietta Mira
{"title":"When non-response makes estimates from a census a small area estimation problem: the case of the survey on graduates’ employment status in Italy","authors":"Maria Giovanna Ranalli, Fulvia Pennoni, Francesco Bartolucci, Antonietta Mira","doi":"10.1007/s11634-025-00630-z","DOIUrl":"10.1007/s11634-025-00630-z","url":null,"abstract":"<div><p>Since 1998, AlmaLaurea—a consortium of 80 Italian universities and a member of the Italian National Statistical System—has conducted an annual census on graduates’ employment status. The survey provides estimates of descriptive indicators at both the population level and for specific subpopulations (domains) of interest, such as degree programmes. Some domains have very few observations due to a small population size and non-response. In this paper, we address this estimation problem within a Small Area Estimation framework. Specifically, we propose using generalized linear mixed models that incorporate two variables as proxies for graduates’ response propensity, making the assumption of non-informative non-response more plausible. Degree programme estimates of employment rates are derived as (semi-parametric) empirical best predictions using a finite mixture of logistic regression models, with their mean squared error estimated via a second-order, bias-corrected, analytical estimator. Sensitivity analysis is conducted to assess the explanatory power of variables modelling response propensity and to evaluate potential correlations between area-specific random effects and observed heterogeneity.</p></div>","PeriodicalId":49270,"journal":{"name":"Advances in Data Analysis and Classification","volume":"19 classification and related methods”","pages":"515 - 543"},"PeriodicalIF":1.3,"publicationDate":"2025-04-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s11634-025-00630-z.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145163429","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Statistical jump model for mixed-type data with missing data imputation","authors":"Federico P. Cortese, Antonio Pievatolo","doi":"10.1007/s11634-025-00631-y","DOIUrl":"10.1007/s11634-025-00631-y","url":null,"abstract":"<div><p>In this paper, we address the challenge of clustering mixed-type data with temporal evolution by introducing the statistical jump model for mixed-type data. This novel framework incorporates regime persistence, enhancing interpretability and reducing the frequency of state switches, and efficiently handles missing data. The model is easily interpretable through its state-conditional medians and modes, making it accessible to practitioners and policymakers. We validate our approach through extensive simulation studies and an empirical application to air quality data, demonstrating its superiority in inferring persistent air quality regimes compared to the traditional air quality index. Our contributions include a robust method for mixed-type temporal clustering, effective missing data management, and practical insights for environmental monitoring.</p></div>","PeriodicalId":49270,"journal":{"name":"Advances in Data Analysis and Classification","volume":"20 2","pages":"635 - 659"},"PeriodicalIF":2.1,"publicationDate":"2025-03-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s11634-025-00631-y.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148314719","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Parametric models for distributional data","authors":"Paula Brito, A. Pedro Duarte Silva","doi":"10.1007/s11634-025-00624-x","DOIUrl":"10.1007/s11634-025-00624-x","url":null,"abstract":"<div><p>We present parametric probabilistic models for numerical distributional variables. The proposed models are based on the representation of each distribution by a location measure and inter-quantile ranges, for given quantiles, thereby characterizing the underlying empirical distributions in a flexible way. Multivariate Normal distributions are assumed for the whole set of indicators, considering alternative structures of the variance–covariance matrix. For all cases, maximum likelihood estimators of the corresponding parameters are derived. This modelling allows for hypothesis testing and multivariate parametric analysis. The proposed framework is applied to Analysis of Variance and parametric Discriminant Analysis of distributional data. A simulation study examines the performance of the proposed models in classification problems under different data conditions. Applications to Internet traffic data and Portuguese official data illustrate the relevance of the proposed approach.</p></div>","PeriodicalId":49270,"journal":{"name":"Advances in Data Analysis and Classification","volume":"19 4","pages":"1119 - 1146"},"PeriodicalIF":1.3,"publicationDate":"2025-03-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s11634-025-00624-x.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145612827","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A low-rank non-convex norm method for multiview graph clustering","authors":"Alaeddine Zahir, Khalide Jbilou, Ahmed Ratnani","doi":"10.1007/s11634-025-00628-7","DOIUrl":"10.1007/s11634-025-00628-7","url":null,"abstract":"<div><p>This study addresses the challenge of multiview clustering by integrating information from multiple data sources to improve clustering accuracy. We propose CGMVC-NC, a novel Consensus Graph-Based Multi-View Clustering method Using Low-Rank Non-Convex Norm, which effectively captures correlations across views. Unlike traditional methods, CGMVC-NC introduces a non-convex low-rank tensor norm to enhance the representation of shared structures while reducing noise and redundancy. By constructing a consensus graph that preserves essential multiview relationships, our approach ensures more reliable clustering results. Extensive experiments on benchmark datasets confirm its superiority over existing techniques, demonstrating improved clustering performance and robustness in handling complex multiview data.</p></div>","PeriodicalId":49270,"journal":{"name":"Advances in Data Analysis and Classification","volume":"20 2","pages":"615 - 634"},"PeriodicalIF":2.1,"publicationDate":"2025-03-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148314723","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Editorial for ADAC issue 1 of volume 19 (2025)","authors":"Maurizio Vichi, Andrea Cerioli, Hans A. Kestler","doi":"10.1007/s11634-025-00629-6","DOIUrl":"10.1007/s11634-025-00629-6","url":null,"abstract":"","PeriodicalId":49270,"journal":{"name":"Advances in Data Analysis and Classification","volume":"19 1","pages":"1 - 4"},"PeriodicalIF":1.4,"publicationDate":"2025-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143707011","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Weighted embedding and outlier detection of metric space data","authors":"Lauri Heinonen, Henri Nyberg, Joni Virta","doi":"10.1007/s11634-025-00627-8","DOIUrl":"10.1007/s11634-025-00627-8","url":null,"abstract":"<div><p>This work discusses weighted kernel point projection (WKPP), a new method for embedding metric space or kernel data. WKPP is based on an iteratively weighted generalization of multidimensional scaling and kernel principal component analysis, and one of its main uses is outlier detection. After a detailed derivation of the method and its algorithm, we give theoretical guarantees regarding its convergence and outlier detection capabilities. Additionally, as one of our mathematical contributions, we give a novel characterization of kernelizability, connecting it also to the classical kernel literature. In our empirical examples, WKPP is benchmarked with respect to several competing outlier detection methods, using various different datasets. The obtained results show that WKPP is computationally fast, while simultaneously achieving performance comparable to state-of-the-art methods.</p></div>","PeriodicalId":49270,"journal":{"name":"Advances in Data Analysis and Classification","volume":"20 2","pages":"577 - 613"},"PeriodicalIF":2.1,"publicationDate":"2025-02-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s11634-025-00627-8.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148314748","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Maurizio Romano, Claudio Conversano, Roberta Siciliano, Antonio D’Ambrosio
{"title":"Particle swarm optimization for preference rankings","authors":"Maurizio Romano, Claudio Conversano, Roberta Siciliano, Antonio D’Ambrosio","doi":"10.1007/s11634-025-00626-9","DOIUrl":"10.1007/s11634-025-00626-9","url":null,"abstract":"<div><p>Preference learning, or the analysis of preference rankings, is gaining more and more importance in various scientific disciplines. Preference learning methods allow predicting preferences on a set of alternatives. The ingredients are a pool of evaluators and a set of objects or items to be ranked in order of preference. The rank aggregation problem must be solved in order to aggregate preferences or rankings with the aim to find a consensus or collective decision. Branch-and-bound-like procedures are usable up to problems involving a relatively small number of objects, say less than 200. When the number of items becomes very large, the rank aggregation problem becomes increasingly difficult to approach so that it is universally recognized as an NP-hard problem. Several heuristic methods have been proposed to provide increasingly accurate solutions. These assume the Kemeny axiomatic approach that better deals with tied rankings. In this paper, we adopt a strategy based on Particle Swarm Optimization by adapting procedures born to solve optimization problems in continuous spaces to discrete combinatorial optimization problems. A simulation study shows the performance of the proposed algorithm in a controlled environment. A benchmarking complex data set and two real world data sets with large number of items are considered. As a result, the proposed algorithm provides significant savings in computational time and comparable accuracy with respect to other recent algorithms.</p></div>","PeriodicalId":49270,"journal":{"name":"Advances in Data Analysis and Classification","volume":"20 2","pages":"551 - 575"},"PeriodicalIF":2.1,"publicationDate":"2025-02-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s11634-025-00626-9.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148314747","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Random models for adjusting fuzzy rand index extensions","authors":"Ryan DeWolfe, Jeffrey L. Andrews","doi":"10.1007/s11634-025-00625-w","DOIUrl":"10.1007/s11634-025-00625-w","url":null,"abstract":"<div><p>The adjusted Rand index (ARI) is a widely used method for comparing hard clusterings, but requires a choice of random model that is often left implicit. Several recent works have extended the Rand index to fuzzy clusterings and adjusted for chance agreement with the permutation model, but the assumptions of this random model are difficult to justify for fuzzy clusterings. Previous work on random models for hard clusterings has shown that different random models can impact similarity rankings, so matching the assumptions of the random model to the algorithm is essential. We propose a single framework computing the ARI with three new random models that are intuitive and explainable for both hard and fuzzy clusterings. The theory and assumptions of the proposed models are contrasted with the existing permutation model, and computations on synthetic and benchmark data show that each model has distinct behaviour, meaning accurate model selection is important for the reliability of results.</p></div>","PeriodicalId":49270,"journal":{"name":"Advances in Data Analysis and Classification","volume":"19 classification and related methods”","pages":"361 - 385"},"PeriodicalIF":1.3,"publicationDate":"2025-02-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145165581","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ganesh Babu, Aoife Gowen, Michael Fop, Isobel Claire Gormley
{"title":"A consensus-constrained parsimonious Gaussian mixture model for clustering hyperspectral images","authors":"Ganesh Babu, Aoife Gowen, Michael Fop, Isobel Claire Gormley","doi":"10.1007/s11634-025-00623-y","DOIUrl":"10.1007/s11634-025-00623-y","url":null,"abstract":"<div><p>The use of hyperspectral imaging to investigate food samples has grown due to the improved performance and lower cost of instrumentation. Food engineers use hyperspectral images to classify the type and quality of a food sample, typically using classification methods. In order to train these methods, every pixel in each training image needs to be labelled. Typically, computationally cheap threshold-based approaches are used to label the pixels, and classification methods are trained based on those labels. However, threshold-based approaches are subjective and cannot be generalized across hyperspectral images taken in different conditions and of different foods. Here a consensus-constrained parsimonious Gaussian mixture model (ccPGMM) is proposed to label pixels in hyperspectral images using a model-based clustering approach. The ccPGMM utilizes information that is available on some pixels and specifies constraints on those pixels belonging to the same or different clusters while clustering the rest of the pixels in the image. A latent variable model is used to represent the high-dimensional data in terms of a small number of underlying latent factors. To ensure computational feasibility, a consensus clustering approach is employed, where the data are divided into multiple randomly selected subsets of variables and constrained clustering is applied to each data subset; the clustering results are then consolidated across all data subsets to provide a consensus clustering solution. The ccPGMM approach is applied to simulated datasets and real hyperspectral images of three types of puffed cereal, corn, rice, and wheat. Improved clustering performance and computational efficiency are demonstrated when compared to other current state-of-the-art approaches.</p></div>","PeriodicalId":49270,"journal":{"name":"Advances in Data Analysis and Classification","volume":"19 classification and related methods”","pages":"323 - 359"},"PeriodicalIF":1.3,"publicationDate":"2025-02-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s11634-025-00623-y.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145162774","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}