Science China Information Sciences最新文献

{"title":"Envisioning future deep learning theories: some basic concepts and characteristics","authors":"Weijie J. Su","doi":"10.1007/s11432-023-4129-1","DOIUrl":"https://doi.org/10.1007/s11432-023-4129-1","url":null,"abstract":"<p>To advance deep learning methodologies in the next decade, a theoretical framework for reasoning about modern neural networks is needed. While efforts are increasing toward demystifying why deep learning is so effective, a comprehensive picture remains lacking, suggesting that a better theory is possible. We argue that a future deep learning theory should inherit three characteristics: a hierarchically structured network architecture, parameters iteratively optimized using stochastic gradient-based methods, and information from the data that evolves compressively. As an instantiation, we integrate these characteristics into a graphical model called neurashed. This model effectively explains some common empirical patterns in deep learning. In particular, neurashed enables insights into implicit regularization, information bottleneck, and local elasticity. Finally, we discuss how neurashed can guide the development of deep learning theories.</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"28 1","pages":""},"PeriodicalIF":8.8,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142268456","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Weighted sum power maximization for STAR-RIS-aided SWIPT systems with nonlinear energy harvesting","authors":"Weiping Shi, Cunhua Pan, Feng Shu, Yongpeng Wu, Jiangzhou Wang, Yongqiang Bao, Jin Tian","doi":"10.1007/s11432-024-4102-3","DOIUrl":"https://doi.org/10.1007/s11432-024-4102-3","url":null,"abstract":"<p>The conventional reconfigurable intelligent surface (RIS) is limited to reflecting incident signals, thereby imposing constraints on the placement of the transmitter and receiver, which hinders achieving comprehensive signal coverage across an entire area. This paper investigates a simultaneously transmitting and reflecting (STAR)-RIS-aided simultaneous wireless information and power transfer (SWIPT) system with a nonlinear energy harvesting model under three different RIS transmission protocols: energy splitting (ES), time switching (TS), and mode switching (MS). The objective of this paper is to maximize the weighted sum power (WSP) of all energy harvesting receivers (EHRs) while ensuring fairness in the collected power among them. This is achieved by jointly optimizing the transmit beamforming at the base station (BS) and the transmission and reflection coefficients at the STAR-RIS, subject to rate constraints for information decoding receivers (IDRs), transmit power constraint at the BS, and coefficient constraints of each element at the STAR-RIS corresponding to the three protocols. Solving this optimization problem poses challenges because of the complicated objective function and numerous coupled optimization variables of the ES STAR-RIS. To address this complexity, an efficient alternating optimization (AO) approach is proposed as an iterative solution method that achieves suboptimal results. The AO algorithm is then extended to MS STAR-RIS and TS STAR-RIS. Specifically, for MS STRA-RIS, binary constraints in the STAR-RIS coefficient optimization subproblem are handled using the first-order approximation technique along with the penalty function method. For TS STAR-RIS, apart from optimizing BS transmit beamforming and STAR-RIS coefficients subproblems, the transmission and reflection time allocation of STAR-RIS also needs optimization. Numerical findings demonstrate that compared to conventional RIS-aided systems, utilizing three different protocols in a STAR-RIS-aided system can enhance power collection at EHRs while expanding the receiver placement range. Furthermore, TS STAR-RIS performs best when the IDRs do not require high achieved rates. Otherwise, ES is the best choice.</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"16 1","pages":""},"PeriodicalIF":8.8,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142252170","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Automatically identifying imperfections and attacks in practical quantum key distribution systems via machine learning","authors":"Jiaxin Xu, Xiao Ma, Jingyang Liu, Chunhui Zhang, Hongwei Li, Xingyu Zhou, Qin Wang","doi":"10.1007/s11432-023-3988-x","DOIUrl":"https://doi.org/10.1007/s11432-023-3988-x","url":null,"abstract":"<p>The realistic security of quantum key distribution (QKD) systems is currently a hot research topic in the field of quantum communications. There are always defects in practical devices, and eavesdroppers can make use of the security risk points of various devices to obtain key information. To date, current types of security analysis tend to analyze each security risk point individually, thereby posing great challenges for the overall security evaluation of QKD systems. In this paper, for the first time, we employ machine learning algorithms to identify the defects of different devices and certain attacks in real time, with an accuracy of 98%. It provides a novel solution for the practical security evaluation of QKD systems, thereby addressing the bottleneck problem of multiple risk points being difficult to address simultaneously in QKD systems, thus paving the way for the future large-scale application of quantum communication networks.</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"2010 1","pages":""},"PeriodicalIF":8.8,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142189438","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Ultra-low power IGZO optoelectronic synaptic transistors for neuromorphic computing","authors":"Li Zhu, Sixian Li, Junchen Lin, Yuanfeng Zhao, Xiang Wan, Huabin Sun, Shancheng Yan, Yong Xu, Zhihao Yu, Chee Leong Tan, Gang He","doi":"10.1007/s11432-023-3966-8","DOIUrl":"https://doi.org/10.1007/s11432-023-3966-8","url":null,"abstract":"<p>Inspired by biological visual systems, optoelectronic synapses with image perception, memory retention, and preprocessing capabilities offer a promising pathway for developing high-performance artificial perceptual vision computing systems. Among these, oxide-based optoelectronic synaptic transistors are well-known for their enduring photoconductive properties and ease of integration, which hold substantial potential in this regard. In this study, we utilized indium gallium zinc oxide as a semiconductor layer and high-k ZrAlO_<i>x</i> as a gate dielectric layer to engineer low-power high-performance synaptic transistors with photonic memory. Crucial biological synaptic functions, including excitatory postsynaptic currents, paired-pulse facilitation, and the transition from short-term to long-term plasticity, were replicated via optical pulse modulation. This simulation was sustained even at an operating voltage as low as 0.0001 V, exhibiting a conspicuous photonic synaptic response with energy consumption as low as 0.0845 fJ per synaptic event. Furthermore, an optoelectronic synaptic device was employed to model “learn-forget-relearn” behavior similar to that exhibited by the human brain, as well as Morse code encoding. Finally, a 3 × 3 device array was constructed to demonstrate its advantages in image recognition and storage. This study provides an effective strategy for developing readily integrable, ultralow-power optoelectronic synapses with substantial potential in the domains of morphological visual systems, biomimetic robotics, and artificial intelligence.</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"73 1","pages":""},"PeriodicalIF":8.8,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142189439","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"An efficient schedulability analysis based on worst-case interference time for real-time systems","authors":"Hongbiao Liu, Mengfei Yang, Lei Qiao, Xi Chen, Jian Gong","doi":"10.1007/s11432-022-3891-4","DOIUrl":"https://doi.org/10.1007/s11432-022-3891-4","url":null,"abstract":"<p>Real-time systems are widely implemented in the Internet of Things (IoT) and safety-critical systems, both of which have generated enormous social value. Aiming at the classic schedulability analysis problem in real-time systems, we proposed an exact Boolean analysis based on interference (EBAI) for schedulability analysis in real-time systems. EBAI is based on worst-case interference time (WCIT), which considers both the release jitter and blocking time of the task. We improved the efficiency of the three existing tests and provided a comprehensive summary of related research results in the field. Abundant experiments were conducted to compare EBAI with other related results. Our evaluation showed that in certain cases, the runtime gain achieved using our analysis method may exceed 73% compared to the state-of-the-art schedulability test. Furthermore, the benefits obtained from our tests grew with the number of tasks, reaching a level suitable for practical application. EBAI is oriented to the five-tuple real-time task model with stronger expression ability and possesses a low runtime overhead. These characteristics make it applicable in various real-time systems such as spacecraft, autonomous vehicles, industrial robots, and traffic command systems.</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"66 1","pages":""},"PeriodicalIF":8.8,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142189446","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Relative difficulty distillation for semantic segmentation","authors":"Dong Liang, Yue Sun, Yun Du, Songcan Chen, Sheng-Jun Huang","doi":"10.1007/s11432-023-4061-2","DOIUrl":"https://doi.org/10.1007/s11432-023-4061-2","url":null,"abstract":"<p>Current knowledge distillation (KD) methods primarily focus on transferring various structured knowledge and designing corresponding optimization goals to encourage the student network to imitate the output of the teacher network. However, introducing too many additional optimization objectives may lead to unstable training, such as gradient conflicts. Moreover, these methods ignored the guidelines of relative learning difficulty between the teacher and student networks. Inspired by human cognitive science, in this paper, we redefine knowledge from a new perspective — the student and teacher networks’ relative difficulty of samples, and propose a pixel-level KD paradigm for semantic segmentation named relative difficulty distillation (RDD). We propose a two-stage RDD framework: teacher-full evaluated RDD (TFE-RDD) and teacher-student evaluated RDD (TSE-RDD). RDD allows the teacher network to provide effective guidance on learning focus without additional optimization goals, thus avoiding adjusting learning weights for multiple losses. Extensive experimental evaluations using a general distillation loss function on popular datasets such as Cityscapes, CamVid, Pascal VOC, and ADE20k demonstrate the effectiveness of RDD against state-of-the-art KD methods. Additionally, our research showcases that RDD can integrate with existing KD methods to improve their upper performance bound. Codes are available at https://github.com/sunyueue/RDD.git.</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"51 1","pages":""},"PeriodicalIF":8.8,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142189442","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"OpBench: an operator-level GPU benchmark for deep learning","authors":"Qingwen Gu, Bo Fan, Zhengning Liu, Kaicheng Cao, Songhai Zhang, Shimin Hu","doi":"10.1007/s11432-023-3989-3","DOIUrl":"https://doi.org/10.1007/s11432-023-3989-3","url":null,"abstract":"<p>Operators (such as Conv and ReLU) play an important role in deep neural networks. Every neural network is composed of a series of differentiable operators. However, existing AI benchmarks mainly focus on accessing model training and inference performance of deep learning systems on specific models. To help GPU hardware find computing bottlenecks and intuitively evaluate GPU performance on specific deep learning tasks, this paper focuses on evaluating GPU performance at the operator level. We statistically analyze the information of operators on 12 representative deep learning models from six prominent AI tasks and provide an operator dataset to show the different importance of various types of operators in different networks. An operator-level benchmark, OpBench, is proposed on the basis of this dataset, allowing users to choose from a given range of models and set the input sizes according to their demands. This benchmark offers a detailed operator-level performance report for AI and hardware developers. We also evaluate four GPU models on OpBench and find that their performances differ on various types of operators and are not fully consistent with the performance metric FLOPS (floating point operations per second).</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"30 1","pages":""},"PeriodicalIF":8.8,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142189444","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Deep learning for code generation: a survey","authors":"Huangzhao Zhang, Kechi Zhang, Zhuo Li, Jia Li, Jia Li, Yongmin Li, Yunfei Zhao, Yuqi Zhu, Fang Liu, Ge Li, Zhi Jin","doi":"10.1007/s11432-023-3956-3","DOIUrl":"https://doi.org/10.1007/s11432-023-3956-3","url":null,"abstract":"<p>In the past decade, thanks to the powerfulness of deep-learning techniques, we have witnessed a whole new era of automated code generation. To sort out developments, we have conducted a comprehensive review of solutions to deep learning-based code generation. In this survey, we generally formalize the pipeline and procedure of code generation and categorize existing solutions according to taxonomy from perspectives of architecture, model-agnostic enhancing strategy, metrics, and tasks. In addition, we outline the challenges faced by current dominant large models and list several plausible directions for future research. We hope that this survey may provide handy guidance to understanding, utilizing, and developing deep learning-based code-generation techniques for researchers and practitioners.</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"14 1","pages":""},"PeriodicalIF":8.8,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142189445","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Quantum search with prior knowledge","authors":"Xiaoyu He, Xiaoming Sun, Jialing Zhang","doi":"10.1007/s11432-023-3972-y","DOIUrl":"https://doi.org/10.1007/s11432-023-3972-y","url":null,"abstract":"<p>The combination of contextual side information and search is a powerful paradigm in the scope of artificial intelligence. The prior knowledge enables the identification of possible solutions but may be imperfect. Contextual information can arise naturally, for example in game AI where prior knowledge is used to bias move decisions. In this work we investigate the problem of taking quantum advantage of contextual information, especially searching with prior knowledge. We propose a new generalization of Grover’s search algorithm that achieves the optimal expected success probability of finding the solution if the number of queries is fixed. Experiments on small-scale quantum circuits verify the advantage of our algorithm. Since contextual information exists widely, our method has wide applications. We take game tree search as an example.</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"10 1","pages":""},"PeriodicalIF":8.8,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142189448","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Physics-informed deep Koopman operator for Lagrangian dynamic systems","authors":"Xuefeng Wang, Yang Cao, Shaofeng Chen, Yu Kang","doi":"10.1007/s11432-022-4050-4","DOIUrl":"https://doi.org/10.1007/s11432-022-4050-4","url":null,"abstract":"<p>Accurate mechanical system models are crucial for safe and stable control. Unlike linear systems, Lagrangian systems are highly nonlinear and difficult to optimize because of their unknown system model. Recent research thus used deep neural networks to generate linear models of original systems by mapping nonlinear dynamic systems into a linear space with a Koopman observable function encoder. The controller then relies on the Koopman linear model. However, without physical information constraints, ensuring control consistency between the original nonlinear system and the Koopman system is tough, as the learning process of the Koopman observation function is unsupervised. This paper thus proposes a two-stage learning algorithm that uses structural subnetworks to build a physics-informed network topology to simultaneously learn the Koopman observable functions and the system energy representation. In the Koopman matrix learning session, a quadratic-constrained optimization problem is solved to ensure that the Koopman representation satisfies the energy difference matching hard constraint. The proposed energy-preserving deep Lagrangian Koopman (EPDLK) framework effectively represents the dynamics of the Lagrangian system while ensuring control consistency. The effectiveness of EPDLK is compared with those of various Koopman observable function construction methods in multistep prediction and trajectory tracking tasks. EPDLK achieves better control consistency by guaranteeing energy difference matching, which facilitates the application of the control law generated on the Koopman system directly to the original nonlinear Lagrangian system.</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"35 1","pages":""},"PeriodicalIF":8.8,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142189443","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}