Doklady Mathematics最新文献_第3页

Zero Order Algorithm for Decentralized Optimization Problems

IF 0.5 4区数学

Doklady Mathematics Pub Date : 2025-03-22 DOI: 10.1134/S1064562424602336

A. S. Veprikov, E. D. Petrov, G. V. Evseev, A. N. Beznosikov

{"title":"Zero Order Algorithm for Decentralized Optimization Problems","authors":"A. S. Veprikov, E. D. Petrov, G. V. Evseev, A. N. Beznosikov","doi":"10.1134/S1064562424602336","DOIUrl":"10.1134/S1064562424602336","url":null,"abstract":"In this paper we consider a distributed optimization problem in the black-box formulation. This means that the target function f is decomposed into the sum of (n) functions ({{f}_{i}}), where (n) is the number of workers, it is assumed that each worker has access only to the zero-order noisy oracle, i.e., only to the values of ({{f}_{i}}(x)) with added noise. In this paper, we propose a new method ZO-MARINA based on the state-of-the-art distributed optimization algorithm MARINA. In particular, the following modifications are made to solve the problem in the black-box formulation: (i) we use approximations of the gradient instead of the true value, (ii) the difference of two approximated gradients at some coordinates is used instead of the compression operator. In this paper, a theoretical convergence analysis is provided for non-convex functions and functions satisfying the PL condition. The convergence rate of the proposed algorithm is correlated with the results for the algorithm that uses the first-order oracle. The theoretical results are validated in computational experiments to find optimal hyperparameters for the Resnet-18 neural network, that is trained on the CIFAR-10 dataset and the SVM model on the LibSVM library dataset and on the Mnist-784 dataset.","PeriodicalId":531,"journal":{"name":"Doklady Mathematics","volume":"110 1 supplement","pages":"S261 - S277"},"PeriodicalIF":0.5,"publicationDate":"2025-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1134/S1064562424602336.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143676487","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Heterogeneous Computational Scheduling Using Adaptive Neural Hyper-Heuristic

IF 0.5 4区数学

Doklady Mathematics Pub Date : 2025-03-22 DOI: 10.1134/S106456242460221X

A. Allahverdyan, A. Zhadan, I. Kondratov, O. Petrosian, A. Romanovskii, V. Kharin, Yin Li

{"title":"Heterogeneous Computational Scheduling Using Adaptive Neural Hyper-Heuristic","authors":"A. Allahverdyan, A. Zhadan, I. Kondratov, O. Petrosian, A. Romanovskii, V. Kharin, Yin Li","doi":"10.1134/S106456242460221X","DOIUrl":"10.1134/S106456242460221X","url":null,"abstract":"In heterogeneous computing environments, efficiently scheduling tasks, especially those forming Directed Acyclic Graphs (DAGs), is critical. This is particularly true for various Cloud and Edge computing tasks, as well as training Large Language Models (LLMs). This paper introduces a new scheduling approach using an Adaptive Neural Hyper-heuristic. By integrating a neural network trained with genetic algorithms, our method aims to minimize makespan. The approach uses a two-level algorithm: the first level prioritizes tasks using adaptive heuristic and the second level assigns resources based on the Earliest Finish Time (EFT) algorithm. Our tests show that this method significantly improves over traditional scheduling heuristics and other machine learning-based approaches. It reduces the makespan by 6.7% for small-scale DAGs and 28.49% for large-scale DAGs compared to the leading DONF algorithm. Additionally, it achieves a proximity of 84.08% to 96.43% to the optimal solutions found using Mixed-Integer Linear Programming (MILP), demonstrating its effectiveness in diverse computational settings.","PeriodicalId":531,"journal":{"name":"Doklady Mathematics","volume":"110 1 supplement","pages":"S151 - S161"},"PeriodicalIF":0.5,"publicationDate":"2025-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1134/S106456242460221X.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143676271","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Prediction of Industrial Cyber Attacks Using Normalizing Flows

IF 0.5 4区数学

Doklady Mathematics Pub Date : 2025-03-22 DOI: 10.1134/S1064562424602269

V. P. Stepashkina, M. I. Hushchyn

引用次数: 0

Deep Learning-Driven Approach for Handwritten Chinese Character Classification

IF 0.5 4区数学

Doklady Mathematics Pub Date : 2025-03-22 DOI: 10.1134/S1064562424602245

B. Kriuk, F. Kriuk

{"title":"Deep Learning-Driven Approach for Handwritten Chinese Character Classification","authors":"B. Kriuk, F. Kriuk","doi":"10.1134/S1064562424602245","DOIUrl":"10.1134/S1064562424602245","url":null,"abstract":"Handwritten character recognition (HCR) is a challenging problem for machine learning researchers. Unlike printed text data, handwritten character datasets have more variation due to human-introduced bias. With numerous unique character classes present, some data, such as Logographic Scripts or Sino-Korean character sequences, bring new complications to the HCR problem. The classification task on such datasets requires the model to learn high-complexity details of the images that share similar features. With recent advances in computational resource availability and further computer vision theory development, some research teams have effectively addressed the arising challenges. Although known for achieving high accuracy while keeping the number of parameters small, many common approaches are still not generalizable and use dataset-specific solutions to achieve better results. Due to complex structure, existing methods frequently prevent the solutions from gaining popularity. This paper proposes a highly scalable approach for detailed character image classification by introducing the model architecture, data preprocessing steps, and testing design instructions. We also perform experiments to compare the performance of our method with that of existing ones to show the improvements achieved.","PeriodicalId":531,"journal":{"name":"Doklady Mathematics","volume":"110 1 supplement","pages":"S278 - S287"},"PeriodicalIF":0.5,"publicationDate":"2025-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143676206","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

MDS-ViTNet: Improving Saliency Prediction for Eye-Tracking with Vision Transformer

IF 0.5 4区数学

Doklady Mathematics Pub Date : 2025-03-22 DOI: 10.1134/S1064562424602117

I. Polezhaev, I. Goncharenko, N. Iurina

{"title":"MDS-ViTNet: Improving Saliency Prediction for Eye-Tracking with Vision Transformer","authors":"I. Polezhaev, I. Goncharenko, N. Iurina","doi":"10.1134/S1064562424602117","DOIUrl":"10.1134/S1064562424602117","url":null,"abstract":"In this paper, we present a novel methodology we call MDS-ViTNet (Multi Decoder Saliency by Vision Transformer Network) for enhancing visual saliency prediction or eye-tracking. This approach holds significant potential for diverse fields, including marketing, medicine, robotics, and retail. We propose a network architecture that leverages the Vision Transformer, moving beyond the conventional ImageNet backbone. The framework adopts an encoder-decoder structure, with the encoder utilizing a Swin transformer to efficiently embed most important features. This process involves a Transfer Learning method, wherein layers from the Vision Transformer are converted by the Encoder Transformer and seamlessly integrated into a CNN Decoder. This methodology ensures minimal information loss from the original input image. The decoder employs a multi-decoding technique, utilizing dual decoders to generate two distinct attention maps. These maps are subsequently combined into a singular output via an additional CNN model. Our trained model MDS-ViTNet achieves state-of-the-art results across several benchmarks. Committed to fostering further collaboration, we intend to make our code, models, and datasets accessible to the public.","PeriodicalId":531,"journal":{"name":"Doklady Mathematics","volume":"110 1 supplement","pages":"S230 - S235"},"PeriodicalIF":0.5,"publicationDate":"2025-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1134/S1064562424602117.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143676210","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Neural Network-Based Coronary Dominance Classification of RCA Angiograms

IF 0.5 4区数学

Doklady Mathematics Pub Date : 2025-03-22 DOI: 10.1134/S1064562424602026

I. Kruzhilov, E. Ikryannikov, A. Shadrin, R. Utegenov, G. Zubkova, I. Bessonov

{"title":"Neural Network-Based Coronary Dominance Classification of RCA Angiograms","authors":"I. Kruzhilov, E. Ikryannikov, A. Shadrin, R. Utegenov, G. Zubkova, I. Bessonov","doi":"10.1134/S1064562424602026","DOIUrl":"10.1134/S1064562424602026","url":null,"abstract":"Coronary arterial dominance classification is essential for SYNTAX score estimation, which is a tool used to determine the complexity of coronary artery disease and guide patient selection toward optimal revascularization strategy. We developed coronary dominance classification algorithm based on the analysis of right coronary artery (RCA) angiograms using neural network.We employed convolutional neural network ConvNext and Swin transformer for 2D image (frames) classification, along with a majority vote for cardio angiographic view classification. An auxiliary network was also used to detect irrelevant images which were then excluded from the data set.5-fold cross validation gave the following dominance classification metrics (p = 95%): macro recall = 93.1% ± 4.3%, accuracy = 93.5% ± 3.8%, macro F1 = 89.2% ± 5.6%. The most common case in which the model regularly failed was RCA occlusion, as it requires utilization of left coronary artery (LCA) information.The use of machine learning approaches to classify coronary dominance based on RCA alone has been shown to be successful with satisfactory accuracy. However, for higher accuracy, it is necessary to utilize LCA information in the case of an occluded RCA and detect cases where there is high uncertainty.","PeriodicalId":531,"journal":{"name":"Doklady Mathematics","volume":"110 1 supplement","pages":"S212 - S222"},"PeriodicalIF":0.5,"publicationDate":"2025-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1134/S1064562424602026.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143676208","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Application of Metaheuristic Algorithms for Optimization of Recloser Placement in a Power Supply System with Distributed Generation

IF 0.5 4区数学

Doklady Mathematics Pub Date : 2025-03-22 DOI: 10.1134/S1064562424602282

N. N. Sergeev, P. V. Matrenin

{"title":"Application of Metaheuristic Algorithms for Optimization of Recloser Placement in a Power Supply System with Distributed Generation","authors":"N. N. Sergeev, P. V. Matrenin","doi":"10.1134/S1064562424602282","DOIUrl":"10.1134/S1064562424602282","url":null,"abstract":"Efficiency and reliability optimization of distribution networks is an important task in the design of power supply systems, and its complexity increases with the development of new technologies such as distributed generation. One way to improve network reliability is through the installation and optimal placement of automatic circuit reclosers. The presence of distributed generation units and reclosers significantly increases the dimensionality of the optimization problem, thus necessitating the use of alternative approaches to solve it. The goal of the research is to analyze the effectiveness of metaheuristic algorithms in the recloser quantity and allocation optimization problem in a distribution network. The scientific novelty of the study lies in simultaneously considering the failure rate of network elements and changes in operating condition in case of contingencies. The practical significance of the work is demonstrated through the effectiveness of using metaheuristic methods when selecting the optimal equipment configuration in electrical networks. To solve the optimization problem of recloser placement in a 24-bus 10 kV network, the genetic algorithm, evolutionary strategy, and adaptive particle swarm optimization were considered. Computational experiments showed that the genetic algorithm is the most efficient in this case. The results can be further used in the development of methodological guidelines for designing distribution networks of various voltage classes.","PeriodicalId":531,"journal":{"name":"Doklady Mathematics","volume":"110 1 supplement","pages":"S87 - S94"},"PeriodicalIF":0.5,"publicationDate":"2025-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143676268","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

SciRus: Tiny and Powerful Multilingual Encoder for Scientific Texts

IF 0.5 4区数学

Doklady Mathematics Pub Date : 2025-03-22 DOI: 10.1134/S1064562424602178

N. Gerasimenko, A. Vatolin, A. Ianina, K. Vorontsov

{"title":"SciRus: Tiny and Powerful Multilingual Encoder for Scientific Texts","authors":"N. Gerasimenko, A. Vatolin, A. Ianina, K. Vorontsov","doi":"10.1134/S1064562424602178","DOIUrl":"10.1134/S1064562424602178","url":null,"abstract":"LLM-based representation learning is widely used to build effective information retrieval systems, including scientific domains. For making science more open and affordable, it is important that these systems support multilingual (and cross-lingual) search and do not require significant computational power. To address this we propose SciRus-tiny, light multilingual encoder trained from scratch on 44 M abstracts (15B tokens) of research papers and then tuned in a contrastive manner using citation data. SciRus-tiny outperforms SciNCL, English-only SOTA-model for scientific texts, on 13/24 tasks, achieving SOTA on 7, from SciRepEval benchmark. Furthermore, SciRus-tiny is much more effective than SciNCL: it is almost 5x smaller (23 M parameters vs. 110 M), having approximately 2x smaller embeddings (312 vs. 768) and 2x bigger context length (1024 vs. 512). In addition to the tiny model, we also propose the SciRus-small (61 M parameters and 768 embeddings size), which is more powerful and can be used for complicated downstream tasks. We further study different ways of contrastive pre-training and demonstrate that almost SOTA results can be achieved without citation information, operating with only title-abstract pairs.","PeriodicalId":531,"journal":{"name":"Doklady Mathematics","volume":"110 1 supplement","pages":"S193 - S202"},"PeriodicalIF":0.5,"publicationDate":"2025-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1134/S1064562424602178.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143676295","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Unraveling the Hessian: A Key to Smooth Convergence in Loss Function Landscapes

IF 0.5 4区数学

Doklady Mathematics Pub Date : 2025-03-22 DOI: 10.1134/S1064562424601987

N. S. Kiselev, A. V. Grabovoy

引用次数: 0

Universal Representations for Well-Logging Data via Ensembling of Self-Supervised Models

IF 0.5 4区数学

Doklady Mathematics Pub Date : 2025-03-22 DOI: 10.1134/S1064562424602257

V. A. Zholobov, E. D. Romanenkova, S. A. Egorov, N. A. Gevorgyan, A. A. Zaytsev

{"title":"Universal Representations for Well-Logging Data via Ensembling of Self-Supervised Models","authors":"V. A. Zholobov, E. D. Romanenkova, S. A. Egorov, N. A. Gevorgyan, A. A. Zaytsev","doi":"10.1134/S1064562424602257","DOIUrl":"10.1134/S1064562424602257","url":null,"abstract":"Time series representation learning is crucial in applications requiring sophisticated data analysis. In some areas, like the Oil and Gas industry, the problem is particularly challenging due to missing values and anomalous samples caused by sensor failures in highly complex manufacturing environments. Self-supervised learning is one of the most popular solutions for obtaining data representation. However, being either generative or contrastive, these methods suffer from the limited applicability of obtained embeddings, – so general usage is more often declared than achieved.This study introduces and examines various generative self-supervised architectures for complex industrial time series. Moreover, we propose a new way to ensemble several generative approaches, leveraging the best advantages of each method. The suggested procedure is designed to tackle a wide range of scenarios with missing and multiscale data.For numerical experiments, we use various-scale datasets of well logs from diverse oilfields. Evaluation includes change point detection, clustering, and transfer learning, with the last two problems being introduced for the first time. It shows that variational autoencoders excel in clustering, autoregressive models better detect change points, and the proposed ensemble succeeds in both tasks.","PeriodicalId":531,"journal":{"name":"Doklady Mathematics","volume":"110 1 supplement","pages":"S126 - S136"},"PeriodicalIF":0.5,"publicationDate":"2025-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1134/S1064562424602257.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143676297","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0