{"title":"Semiparametric Latent Topic Modeling on Consumer-Generated Corpora","authors":"Dominic B. Dayta, Erniel B. Barrios","doi":"10.1007/s40745-025-00587-y","DOIUrl":"10.1007/s40745-025-00587-y","url":null,"abstract":"<div><p>Common methods used for topic modeling have generally suffered problems of overfitting, leading to diminished predictive performance, as well as a weakness towards reconstructing sparse topic structures that involve only a few critical words to aid in interpretation. Considering the text typically contained in customer feedback, this paper proposes a semiparametric topic model utilizing a two-step approach: (1) makes use of nonnegative matrix factorization to recover topic distributions based on word co-occurrences and; (2) use semiparametric regression to identify factors driving the expression of particular topics in the documents given additional auxiliary information such as location, time of writing, and other features of the author. This approach provides a generative model that can be useful for predicting topics in new documents based on these auxiliary variables, and is demonstrated to accurately identify topics even for documents limited in length or size of vocabulary. In an application to real customer feedback, the topics provided by our model are shown to be as interpretable and useful for downstream analysis tasks as with those produced by current legacy methods.</p></div>","PeriodicalId":36280,"journal":{"name":"Annals of Data Science","volume":"12 6","pages":"1941 - 1963"},"PeriodicalIF":0.0,"publicationDate":"2025-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s40745-025-00587-y.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145537720","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Identifying the Intents Behind Website Visits by Employing Unsupervised Machine Learning Models","authors":"Judah Soobramoney, Retius Chifurira, Temesgen Zewotir, Knowledge Chinhamu","doi":"10.1007/s40745-024-00586-5","DOIUrl":"10.1007/s40745-024-00586-5","url":null,"abstract":"<div><p>With digitisation globally on the rise, corporates are compelled to better understand the usage of their websites. In doing so, corporates will be empowered to better understand consumers, and make necessary adjustments to ultimately improve the corporate’s stance in the competitive global landscape of this modern age. However, the online website visit data has proven to be highly complex, big in data volume, and highly transactional with users expressing unique behaviours. Thus, extracting insight can be a complex problem to solve. This study aimed to employ unsupervised machine learning models to identify the intentions behind the visits on the observed website. The data studied was sourced from the Google Analytics tracking tool that was deployed on a corporate informative website. The study employed a k-means, hierarchical and dbscan unsupervised machine learning models to understand the intents behind visitors on the studied website. All three models detected five major intents that were expressed within the observed data. The intents identified were labelled as “accidentals”, “drop-offs”, “engrossed”, “get-in-touch” and “seekers”. On the observed data, all three unsupervised machine learning methods have performed well. However, in the context of the study, which investigated the intents that drove online visits, the hierarchical clustering method yielded superior results by maintaining the best balance between cluster homogeneity (stronger silhouette coefficients) and cluster size.</p></div>","PeriodicalId":36280,"journal":{"name":"Annals of Data Science","volume":"12 1","pages":"413 - 437"},"PeriodicalIF":0.0,"publicationDate":"2025-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s40745-024-00586-5.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143521572","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Liliya Kurmasheva, Ildar Kurmashev, Vladimir Kulikov, Valentina Kulikova, Askar Tajigitov
{"title":"The Use of Data Mining in the Management of the Career Guidance Work of the University","authors":"Liliya Kurmasheva, Ildar Kurmashev, Vladimir Kulikov, Valentina Kulikova, Askar Tajigitov","doi":"10.1007/s40745-024-00585-6","DOIUrl":"10.1007/s40745-024-00585-6","url":null,"abstract":"<div><p>This study explores the application of data mining techniques to analyse factors influencing university choice and predict enrolment trends in Kazakhstan. For this purpose, methods of analysis (multiple correlation and regression analysis, factor analysis), Brown’s prediction model (Brown’s method), synthesis, concretization, comparison, generalization and survey were used. A survey of 192 first-year students was conducted to identify information sources used by applicants and key factors influencing university choice. Four main factors were revealed through principal component analysis: organization of educational activities, recommendations, availability of state grants, and unwillingness to serve in the army. Enrolment forecasting was conducted using demographic and economic variables in a time series regression model. Results indicated birth rate and migration as significant drivers of enrolment demand. The findings provide insights into applicant decision-making that can inform career guidance and enrolment management strategies. This exploratory study demonstrates the potential of data mining to improve the career counselling and strategic planning capacity of universities.</p></div>","PeriodicalId":36280,"journal":{"name":"Annals of Data Science","volume":"12 6","pages":"1923 - 1940"},"PeriodicalIF":0.0,"publicationDate":"2024-12-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145537725","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Generalized Alpha Power Inverted Weibull Distribution: Application of Air Pollution in Kathmandu, Nepal","authors":"Govinda Prasad Dhungana, Arun Kumar Chaudhary, Ramesh Prasad Tharu, Vijay Kumar","doi":"10.1007/s40745-024-00581-w","DOIUrl":"10.1007/s40745-024-00581-w","url":null,"abstract":"<div><p>A novel probability distribution, the Generalized Alpha Power Inverted Weibull (GAPIW) distribution, is derived from the generalization of the <span>(alpha)</span><i>-</i>power family and compounded with the inverted Weibull distribution. The researchers looked into a lot of different sub-models and found important properties of the GAPIW distribution such as, quantile function, median, mode, moments, mean residual lifetime, and stress-strength reliability. The estimation of distribution parameters was carried out through maximum likelihood estimation methods.</p><p>To gain insights into the characteristics of the GAPIW distribution, the study applied it to the analysis of air pollution data, specifically PM2.5, PM10, and TSP data from multiple stations in the Kathmandu Valley. Notably, the findings indicate that air quality in these areas was significantly worse during winter than in other seasons. Also, the ratio (PM2.5/PM10) of particulate matter is higher, indicating air pollution from anthropogenesis particles in the Valley<i>.</i></p><p>The results demonstrate that the GAPIW distribution is validated through different diagrammatic representations, such as P-P plots, Q-Q plots, and mathematical calculations like the K-S test. The findings reveal that, on average, only three days per month or one month per year predict air pollution levels below the threshold in the Kathmandu Valley. Furthermore, compared to others <span>(alpha)</span><i>-</i>power family of distribution available in the literature, the proposed GAPIW distribution stands as a viable alternative model for assessing and understanding air pollution data and related environmental data. This research has the potential to make valuable contributions to the field of environmental science and air quality monitoring.</p></div>","PeriodicalId":36280,"journal":{"name":"Annals of Data Science","volume":"12 5","pages":"1691 - 1715"},"PeriodicalIF":0.0,"publicationDate":"2024-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144905057","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Sentiment-Based Hierarchical Deep Learning Framework Using Hybrid Optimization for Course Recommendation in E-learning","authors":"A. Madhavi, A. Nagesh, A. Govardhan","doi":"10.1007/s40745-024-00580-x","DOIUrl":"10.1007/s40745-024-00580-x","url":null,"abstract":"<div><p>Course recommendation (CD) is essential for success in a student’s educational journey. Due to the variations in student’s knowledge system, it might be difficult to select the course content from online educational platforms. This problem is overcome by introducing the Political Jellyfish search optimization (PJSO) based Hierarchical Deep Learning for Text (HDLTex) model for sentiment classification (SC) in CD. Here, the input data is taken from the E-khool database, which is subjected to the learner/course agglomerative matrix calculation. Then, the course is grouped by utilizing Bayesian Fuzzy clustering (BFC). When the query is given, bi-level matching is performed. The learner retrieves the preferred items after the best course group is found. Furthermore, course review data is applied to the tokenization process employing <i>Bidirectional Encoder Representations from Transformers (</i>BERT). Finally, the feature extraction is carried out and SC is performed by using HDLTex, which is trained by the proposed PJSO. Moreover, the PJSO is the incorporation of Political Optimizer (PO) and Jellyfish Search Optimization (JSO). The devised PJSO-based HDLTex has a superior assessment for maximum precision of 0.904, maximum recall of 0.915 and maximum F-Measure of 0.904 respectively.</p></div>","PeriodicalId":36280,"journal":{"name":"Annals of Data Science","volume":"12 5","pages":"1661 - 1690"},"PeriodicalIF":0.0,"publicationDate":"2024-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144905056","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Masoud Bazari Jamkhaneh, S. M. T. K. MirMostafaee, Marziye Jadidi
{"title":"On the xgamma k-record values and associated inference","authors":"Masoud Bazari Jamkhaneh, S. M. T. K. MirMostafaee, Marziye Jadidi","doi":"10.1007/s40745-024-00582-9","DOIUrl":"10.1007/s40745-024-00582-9","url":null,"abstract":"<div><p>The xgamma distribution was first introduced by Sen et al. [1] as an alternative distribution to the exponential model. The xgamma distribution exhibits a bathtub-shaped hazard rate function, so it is suitable for many lifetime phenomena. In this paper, we consider the upper <i>k</i>-record values from the xgamma distribution. We obtain exact explicit expressions for the moments of <i>k</i>-record values. We compute the means, variances, and covariances of the upper <i>k</i>-records. Using these computed values, we can find the best linear unbiased estimators (BLUEs) and the best linear invariant estimators (BLIEs) of the location and scale parameters of the xgamma model. In addition, we work on the prediction of a future <i>k</i>-record value. We find the best linear unbiased predictor (BLUP) and the best linear invariant predictor (BLIP) of a future <i>k</i>-record value. Another linear predictor is also discussed. A simulation study is performed to assess the proposed estimators and predictors. We also present a real data example in order to illustrate the application of the theoretical results of the paper. At the end of the paper, we will provide several concluding remarks.</p></div>","PeriodicalId":36280,"journal":{"name":"Annals of Data Science","volume":"12 5","pages":"1717 - 1745"},"PeriodicalIF":0.0,"publicationDate":"2024-11-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144905131","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Deep Enhancement in Supplychain Management with Adaptive Serial Cascaded Autoencoder with Long Short Term Memory and Multi-layered Perceptron Framework","authors":"Ashok Kumar Sarkar, Anupam Das","doi":"10.1007/s40745-024-00576-7","DOIUrl":"10.1007/s40745-024-00576-7","url":null,"abstract":"<div><p>Recognizing and reducing risk is a major part of Supply Chain Management (SCM). Several companies are invested in Supply Chain Risk Management (SCRM) and they have the knowledge about the procurement occupancies within their companies and take steps to ensure this potent source of strategic value. Additionally, these types of companies yield the highest returns with the lowest amount of financial risk. Moreover, reducing financial risk in the SCM network requires thoughtful analysis and a proactive strategy. Hence, this task aims to make a financial risk assessment in SCM with deep learning techniques based on big data. Financial risk-related big data is collected from the Kaggle database and utilized in the data transformation phase. The transformed data is employed for evaluating the financial risk with the support of an Adaptive Serial Cascaded Autoencoder with Long Short-Term Memory and Multi-Layered Perceptron (ASCALSMLP). Here, the parameters for the deep learning techniques like LSTM and MLP were tuned by the hybrid Sandpiper Galactic Swarm Optimization (SGSO) algorithm to enhance the efficacy of the offered approach. From the results analysis, the accuracy of the developed model is 91.12% better than DHOA, 92.5% more than COA, 93.75% improved than GSO, and 94.62% superior to SOA models. Therefore, the results from the developed approach demonstrate effective prediction of financial risks.</p></div>","PeriodicalId":36280,"journal":{"name":"Annals of Data Science","volume":"12 5","pages":"1577 - 1606"},"PeriodicalIF":0.0,"publicationDate":"2024-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144905011","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Statistical Data-Driven Modelling and Forecasting: An Application to COVID-19 Pandemic","authors":"Shalabh, Subhra Sankar Dhar, Sabara Parshad Rajeshbhai","doi":"10.1007/s40745-024-00583-8","DOIUrl":"10.1007/s40745-024-00583-8","url":null,"abstract":"<div><p>One of the key objectives of statistics is to provide a model compatible with the data generated by an unknown random process. Often, it happens that the unknown process is intractable, and no prior data or information associated with the unknown process is available. Under such circumstances, well-known techniques like regression modelling techniques may not work. As a result, an alternative approach may be to observe the general features of the process from the available data. Afterward, a suitable statistical distribution, like a mixture of certain distributions, can be fitted to the existing available data, and future observations can be predicted using this fitting. For example, one may consider the prediction related to the COVID-19 pandemic. As it occurred for the first time, no prior data was available to apprehend the behaviour and progression of the COVID-19 pandemic. For such cases, a data-based statistical modelling procedure can be adopted to predict future occurrences based on a small data set. This article presents such an application-oriented, data-based statistical modelling procedure with an implementation on the COVID-19 data. The proposed procedure can be used for a wide range of modelling and forecasting of future events.</p></div>","PeriodicalId":36280,"journal":{"name":"Annals of Data Science","volume":"12 5","pages":"1747 - 1770"},"PeriodicalIF":0.0,"publicationDate":"2024-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144905012","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Sentiment Analysis of Hate Speech on Women in Social Media Platform Using Multi Label Classification","authors":"Praveena Pillala, Kondapalli Venkata Ramana","doi":"10.1007/s40745-024-00584-7","DOIUrl":"10.1007/s40745-024-00584-7","url":null,"abstract":"<div><p>We live in a world where everything is connected to online social media platforms, and the person uses social media networks like Face book, Twitter, Instagram, Whatsapp, etc. In the present scenario, working women, celebrities, sports persons, and social living women are facing all kinds of comments and negative tweets on social media platforms. Even in many situations including online harassment, using abusive words, scolding toxic words, or hate speech increasing daily basis on social media platforms. In consideration of abused comments or words found on online forums, websites, blogs, YouTube, and social media networks must be detected using efficient automatic detection tools. Searching for each toxic word from comments is inefficient and requires a great deal of time. The study explored toxic comment datasets from Kaggle, using natural language processing; machine learning techniques with the DistilBERT model are used in sentiment analysis to identify the toxic text comments. Here DistilBERT model outperformed with the best accuracy of 97% results and machine learning technique with SVM using Multi classification obtained in Threat label with Precision of 0.99, Recall of 1.0, and F1Score of 0.99 and model accuracy is 0.99.</p></div>","PeriodicalId":36280,"journal":{"name":"Annals of Data Science","volume":"12 6","pages":"1903 - 1921"},"PeriodicalIF":0.0,"publicationDate":"2024-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145537724","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pedro Luiz Ramos, Ana Paula Silva Figueiredo, Diego Carvalho do Nascimento, Fernando Moala, Edilson Flores
{"title":"Beyond Regular SPC: Bridging the (C_{pk}) Capability Index for (a)Symmetric Data","authors":"Pedro Luiz Ramos, Ana Paula Silva Figueiredo, Diego Carvalho do Nascimento, Fernando Moala, Edilson Flores","doi":"10.1007/s40745-024-00577-6","DOIUrl":"10.1007/s40745-024-00577-6","url":null,"abstract":"<div><p>The advancement of technology has increased competitiveness, especially in the manufacturing industry. Alongside Statistical Process Control (SPC), capacity indices are tools used to measure the quality of processes and are useful for establishing standards in manufacturing products. This study was motivated to propose a new control chart based on the capability index <span>(C_{pk})</span>, which is particularly useful for real-time monitoring with respect to short time frames and longitudinal studies. Our methodology proposes a graphical monitoring tools that is obtained by utilizing the rolling capability index with standard distributions (Normal, Gamma, or Weibull) and bootstrap intervals based on closed-form estimators. Simulations and real-world applications demonstrated the utility of our framework, which is computationally inexpensive and applicable to real-time monitoring (useful for longitudinal or time-varying processes), showing that modified-Cpk for asymmetric processes is more accurate than point estimation based on normality. Moreover, the exemplification data from a Chocolate Factory showed an acceptable process trend in 25% of the observed rolling-windows (from the first modified-Cpk estimations), versus the normal-based that barely detected this pattern (only one in-sample period). That means, a reduction of <span>(approx 22%)</span> on the quality improvement interventions (translated as false alarms).</p></div>","PeriodicalId":36280,"journal":{"name":"Annals of Data Science","volume":"12 5","pages":"1607 - 1633"},"PeriodicalIF":0.0,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144905039","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}