{"title":"Multi-Agent Hierarchical Reinforcement Learning by Integrating Options into MAXQ","authors":"Jing Shen, Guochang Gu, Haibo Liu","doi":"10.1109/IMSCCS.2006.90","DOIUrl":"https://doi.org/10.1109/IMSCCS.2006.90","url":null,"abstract":"MAXQ is a new framework for multi-agent reinforcement learning. But the MAXQ framework cannot decompose all subtasks into more refined hierarchies and the hierarchies are difficult to be discovered automatically. In this paper, a multi-agent hierarchical reinforcement learning approach, named OptMAXQ, by integrating Options into MAXQ is presented. In the OptMAXQ framework, the MAXQ framework is used to introduce knowledge into reinforcement learning and the option framework is used to construct hierarchies automatically. The performance of OptMAXQ is demonstrated in two-robot trash collection task and compared with MAXQ. The simulation results show that the OptMAXQ is more practical than MAXQ in partial known environment","PeriodicalId":202629,"journal":{"name":"First International Multi-Symposiums on Computer and Computational Sciences (IMSCCS'06)","volume":"93 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128573342","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Zhe Zhang, Daxin Liu, Zhengxian Wei, Changsong Sun
{"title":"Research on Triple Modular Redundancy Dynamic Fault-Tolerant System Model","authors":"Zhe Zhang, Daxin Liu, Zhengxian Wei, Changsong Sun","doi":"10.1109/IMSCCS.2006.119","DOIUrl":"https://doi.org/10.1109/IMSCCS.2006.119","url":null,"abstract":"With the increasing amount of distributed computing systems applied in wide range of critical domains, the requirement of high reliability and high availability of distributed computing systems tend to more and more urgent, that the study of distributed fault-tolerant system become more significance. This paper provides a model of triple-modular redundancy dynamic fault-tolerant system, and the reliability of the model is analyzed through an algorithm","PeriodicalId":202629,"journal":{"name":"First International Multi-Symposiums on Computer and Computational Sciences (IMSCCS'06)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128664077","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Research of Tooling Life-Cycle Management System Supporting Networked Agile Manufacturing","authors":"Su'an Wang, Yadong Fang, Zhongbao Qin, L. Du","doi":"10.1109/IMSCCS.2006.114","DOIUrl":"https://doi.org/10.1109/IMSCCS.2006.114","url":null,"abstract":"To improve flexible and fast manufacturing capability of national defense industry, the paper brings forward tooling life-cycle management ideal supporting networked agile manufacturing by analyzing tooling management status of one national defense manufacturing enterprise, and then discusses how to design system by means of constructing tooling life-cycle management system (TLMS) static and dynamic modeling in UML (Unified Model Language). Lastly, the thesis introduces TMLS function structure tree and software infrastructure","PeriodicalId":202629,"journal":{"name":"First International Multi-Symposiums on Computer and Computational Sciences (IMSCCS'06)","volume":"64 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134436119","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Degenerated Primer Design to Amplify the Heavy Chain Variable Region from Immunoglobulin cDNA","authors":"Wang Ying, Chen Wei, Li Xu, Cheng Bing","doi":"10.1109/IMSCCS.2006.53","DOIUrl":"https://doi.org/10.1109/IMSCCS.2006.53","url":null,"abstract":"The amplification of variable regions (Fv) of immunoglobulins (Ig) becomes a major challenge in cloning antibody genes either from hybridoma cell lines or splenic B cells. When we amplified the heavy-chain variable regions (VH) gene from one hybridoma cell line which were not amplified successfully under conventional protocols, a novel method was developed to design the degenerated primer of immunoglobulin cDNA and rapid amplification of cDNA ends polymerase chain reaction (RACE PCR) protocols were performed to recognize the VH gene from the hybridoma cell line. The most highly conserved region in the middle of the VH regions of the Ig cDNA was identified and a degenerated 5'primer was designed using our algorithms. The VH gene was amplified by the 3'RACE and 5'RACE. The VH sequence of CSA cells was 399 bp. The new protocol rescued the amplifications of the VH gene that had failed under conventional protocols and notably increased amplification specificity. The algorithm improved the primer design efficiency and is useful in building VH and VL gene libraries or cloning of unknown genes in gene families","PeriodicalId":202629,"journal":{"name":"First International Multi-Symposiums on Computer and Computational Sciences (IMSCCS'06)","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134468280","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Towards Merging Pervasive Computing into Grid - Lightweight Portal, Dynamic Collaborating and Semantic Supporting","authors":"Wei Shi, Shanping Li, Xin Lin","doi":"10.1109/IMSCCS.2006.137","DOIUrl":"https://doi.org/10.1109/IMSCCS.2006.137","url":null,"abstract":"The emerging grid infrastructure is one of the key perspectives of pervasive computing. However, it is difficult to implement all the grid semantics on pervasive devices due to the inherent limitations of most pervasive devices. It is not yet clear that how to organize large quantities of heterogeneous pervasive devices collaborating with each other to solve user-centered tasks. And exploiting grid infrastructure to accomplish resource-intensive tasks is still a challenge for resource-weak pervasive devices. In this paper, we put forward the concept of user's task oriented nomadic autonomy domain (UTONAD) which models the pervasive devices collaborating patterns when they are included in a global resource-sharing platform. Based on UTONAD, we propose the architecture of our implementation that provides a uniform method to manage QoS guaranteed interactions between pervasive devices and grid in a dynamic (both temporal and spatial) and user-transparent fashion","PeriodicalId":202629,"journal":{"name":"First International Multi-Symposiums on Computer and Computational Sciences (IMSCCS'06)","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125058464","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Fast and Blind Restoration Scheme for the Initial States of LFSRs","authors":"Xiangyang Luo, Ping Wang, Peizhong Lu, Fenlin Liu","doi":"10.1109/IMSCCS.2006.62","DOIUrl":"https://doi.org/10.1109/IMSCCS.2006.62","url":null,"abstract":"A fast and blind restoration scheme for LFSR (linear feedback shift register) sequences with high error ratio is proposed. This scheme combines searching-offline and decoding-online to restore the initial state of a LFSR sequence. During the restoring process, two key steps are included: one is to quickly find as many parity-check equations as possible to construct the convolutional code with low rate and high correcting capability, and the other is to decode the convolutional codes based on the improved Viterbi algorithm","PeriodicalId":202629,"journal":{"name":"First International Multi-Symposiums on Computer and Computational Sciences (IMSCCS'06)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125709620","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Dynamic Generation Algorithm for Meta Process Using Markov Decision Processes","authors":"Qi Sui, Haiyang Wang","doi":"10.1109/IMSCCS.2006.5","DOIUrl":"https://doi.org/10.1109/IMSCCS.2006.5","url":null,"abstract":"This paper introduced concepts of meta services and meta process, which can largely simplify the complexity of travel services composition. And corresponding to these concepts, a dynamic generation algorithm for meta process is presented instead of predefining a rigorous workflow model, which used Markov decision processes (MDP) referring to transition probability of meta service and the tourists' preference represented by preference reward function. And differently with some AI planning method, this approach more adept to the stochastic and dynamic environment and deals with the various requirements of tourists more effectively and flexibly. Additional this approach also can resolve similar problems in other services computing domain","PeriodicalId":202629,"journal":{"name":"First International Multi-Symposiums on Computer and Computational Sciences (IMSCCS'06)","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129876944","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Toward Fault-Tolerant Atomic Data Access in Mutable Distributed Hash Tables","authors":"Yi Jiang, Guangtao Xue, Jinyuan You","doi":"10.1109/IMSCCS.2006.135","DOIUrl":"https://doi.org/10.1109/IMSCCS.2006.135","url":null,"abstract":"Though the peer-to-peer network based on distributed hash table have scalability and availability features, but most of the distributed hash tables lack the support for the atomic data accessing, this ability is important to build data structures on distributed system. This paper gives the design and implementation of an atomic accessible mutable distributed hash table - FamDHT. The FamDHT use the Paxos distributed consensus protocol to ensure that the simultaneous access to data is executed under a total order agreement. The FamDHT is build on top of the Bamboo DHT, a robust, open-source DHT implementation. At the end of the paper, an evaluation was given","PeriodicalId":202629,"journal":{"name":"First International Multi-Symposiums on Computer and Computational Sciences (IMSCCS'06)","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127879365","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Proactive Service Request Broker for PC-based Grid Computing","authors":"Zhi Wang, Qi Chen, Chuanshan Gao","doi":"10.1109/IMSCCS.2006.21","DOIUrl":"https://doi.org/10.1109/IMSCCS.2006.21","url":null,"abstract":"Grids are emerging as a global computation infrastructure that enables the sharing, exchange, discovery, selection and aggregation of geographically distributed, heterogeneous resources. In this paper, we studied and implemented a simple proactive service broking middleware (PSRB) for building PC-based grid that promises a cost-effective approach for high performance computing. It employs a robust mediation server group to centrally serve grid nodes and clients by exchanging their service requests and responses as a message broker. At the same time, it provides a few resource agents to proactively monitor the resources on grid nodes and forecast their future availability. Predictions are accepted as the primary fact for PSRB to select the best grid node for job assignment. Based on all these, PSRB provides a fairly reliable infrastructure to build PC-based grid that allows the heterogeneity and autonomy of grid nodes and ensures impartial resource utilizations","PeriodicalId":202629,"journal":{"name":"First International Multi-Symposiums on Computer and Computational Sciences (IMSCCS'06)","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120972839","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Novel Web Service Catalog System Supporting Distributed Service Publication and Discovery","authors":"Shoujian Yu, Qin Zhu, Xiaoling Xia, Jiajin Le","doi":"10.1109/IMSCCS.2006.19","DOIUrl":"https://doi.org/10.1109/IMSCCS.2006.19","url":null,"abstract":"Current proposals for Web services infrastructures focus on centralized approaches such as UDDI. However, with the growing number of Web services available on the Web, such a centralized approach quickly becomes impractical. In this paper, we present a novel distributed Web services catalog system, to support decentralized Web service publication and discovery. By utilizing the distributed hash table (DHT) functionality of peer-to-peer (P2P) computing, Web service catalog information is routed directly to each node in this system, and each node shoulders part of the catalog. But in P2P network, every peer is equally treated, and it does not allow peers with common business interest to federate partnerships. In fact, most applications only focus on publication and discovery of services with interest to their respective business domains. In our work, we have made improvements for this limitation, and proposed the hierarchical Web services catalog system, which enables efficient Web service publication and discovery within a specific business domain. Our initial experiments have shown that the distributed catalog system has good scalability","PeriodicalId":202629,"journal":{"name":"First International Multi-Symposiums on Computer and Computational Sciences (IMSCCS'06)","volume":"595 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123189511","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}