ACM Transactions on Autonomous and Adaptive Systems最新文献

{"title":"Reinforcement Learning of Informed Initial Policies for Decentralized Planning","authors":"Landon Kraemer, Bikramjit Banerjee","doi":"10.1145/2668130","DOIUrl":"https://doi.org/10.1145/2668130","url":null,"abstract":"Decentralized partially observable Markov decision processes (Dec-POMDPs) offer a formal model for planning in cooperative multiagent systems where agents operate with noisy sensors and actuators, as well as local information. Prevalent solution techniques are centralized and model based—limitations that we address by distributed reinforcement learning (RL). We particularly favor alternate learning, where agents alternately learn best responses to each other, which appears to outperform concurrent RL. However, alternate learning requires an initial policy. We propose two principled approaches to generating informed initial policies: a naive approach that lays the foundation for a more sophisticated approach. We empirically demonstrate that the refined approach produces near-optimal solutions in many challenging benchmark settings, staking a claim to being an efficient (and realistic) approximate solver in its own right. Furthermore, alternate best response learning seeded with such policies quickly learns high-quality policies as well.","PeriodicalId":50919,"journal":{"name":"ACM Transactions on Autonomous and Adaptive Systems","volume":"25 1","pages":"18:1-18:32"},"PeriodicalIF":2.7,"publicationDate":"2015-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"90207190","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Distributed Data-Centric Adaptive Sampling for Cyber-Physical Systems","authors":"Eun Kyung Lee, H. Viswanathan, D. Pompili","doi":"10.1145/2644820","DOIUrl":"https://doi.org/10.1145/2644820","url":null,"abstract":"A data-centric joint adaptive sampling and sleep scheduling solution, SILENCE, for autonomic sensor-based systems that monitor and reconstruct physical or environmental phenomena is proposed. Adaptive sampling and sleep scheduling can help realize the much needed resource efficiency by minimizing the communication and processing overhead in densely deployed autonomic sensor-based systems. The proposed solution exploits the spatiotemporal correlation in sensed data and eliminates redundancy in transmitted data through selective representation without compromising on accuracy of reconstruction of the monitored phenomenon at a remote monitor node. Differently from existing adaptive sampling solutions, SILENCE employs temporal causality analysis to not only track the variation in the underlying phenomenon but also its cause and direction of propagation in the field. The causality analysis and the same correlations are then leveraged for adaptive sleep scheduling aimed at saving energy in wireless sensor networks (WSNs). SILENCE outperforms traditional adaptive sampling solutions as well as the recently proposed compressive sampling techniques. Real experiments were performed on a WSN testbed monitoring temperature and humidity distribution in a rack of servers, and the simulations were performed on TOSSIM, the TinyOS simulator.","PeriodicalId":50919,"journal":{"name":"ACM Transactions on Autonomous and Adaptive Systems","volume":"30 1","pages":"21:1-21:27"},"PeriodicalIF":2.7,"publicationDate":"2015-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82799988","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Property-Driven Design for Robot Swarms: A Design Method Based on Prescriptive Modeling and Model Checking","authors":"Manuele Brambilla, A. Brutschy, M. Dorigo, M. Birattari","doi":"10.1145/2700318","DOIUrl":"https://doi.org/10.1145/2700318","url":null,"abstract":"In this article, we present property-driven design, a novel top-down design method for robot swarms based on prescriptive modeling and model checking. Traditionally, robot swarms have been developed using a code-and-fix approach: in a bottom-up iterative process, the developer tests and improves the individual behaviors of the robots until the desired collective behavior is obtained. The code-and-fix approach is unstructured, and the quality of the obtained swarm depends completely on the expertise and ingenuity of the developer who has little scientific or technical support in his activity. Property-driven design aims at providing such scientific and technical support, with many advantages compared to the traditional unstructured approach. Property-driven design is composed of four phases: first, the developer formally specifies the requirements of the robot swarm by stating its desired properties; second, the developer creates a prescriptive model of the swarm and uses model checking to verify that this prescriptive model satisfies the desired properties; third, using the prescriptive model as a blueprint, the developer implements a simulated version of the desired robot swarm and validates the prescriptive model developed in the previous step; fourth, the developer implements the desired robot swarm and validates the previous steps. We demonstrate property-driven design using two case studies: aggregation and foraging.","PeriodicalId":50919,"journal":{"name":"ACM Transactions on Autonomous and Adaptive Systems","volume":"114 1","pages":"17:1-17:28"},"PeriodicalIF":2.7,"publicationDate":"2015-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88253096","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Multiagent Reinforcement Social Learning toward Coordination in Cooperative Multiagent Systems","authors":"Jianye Hao, Ho-fung Leung, Zhong Ming","doi":"10.1145/2644819","DOIUrl":"https://doi.org/10.1145/2644819","url":null,"abstract":"Most previous works on coordination in cooperative multiagent systems study the problem of how two (or more) players can coordinate on Pareto-optimal Nash equilibrium(s) through fixed and repeated interactions in the context of cooperative games. However, in practical complex environments, the interactions between agents can be sparse, and each agent's interacting partners may change frequently and randomly. To this end, we investigate the multiagent coordination problems in cooperative environments under a social learning framework. We consider a large population of agents where each agent interacts with another agent randomly chosen from the population in each round. Each agent learns its policy through repeated interactions with the rest of the agents via social learning. It is not clear a priori if all agents can learn a consistent optimal coordination policy in such a situation. We distinguish two different types of learners depending on the amount of information each agent can perceive: individual action learner and joint action learner. The learning performance of both types of learners is evaluated under a number of challenging deterministic and stochastic cooperative games, and the influence of the information sharing degree on the learning performance also is investigated—a key difference from the learning framework involving repeated interactions among fixed agents.","PeriodicalId":50919,"journal":{"name":"ACM Transactions on Autonomous and Adaptive Systems","volume":"37 1","pages":"20:1-20:20"},"PeriodicalIF":2.7,"publicationDate":"2015-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86153088","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Multi-Cloud Provisioning and Load Distribution for Three-Tier Applications","authors":"N. Grozev, R. Buyya","doi":"10.1145/2662112","DOIUrl":"https://doi.org/10.1145/2662112","url":null,"abstract":"Cloud data centers are becoming the preferred deployment environment for a wide range of business applications because they provide many benefits compared to private in-house infrastructure. However, the traditional approach of using a single cloud has several limitations in terms of availability, avoiding vendor lock-in, and providing legislation-compliant services with suitable Quality of Experience (QoE) to users worldwide. One way for cloud clients to mitigate these issues is to use multiple clouds (i.e., a Multi-Cloud). In this article, we introduce an approach for deploying three-tier applications across multiple clouds in order to satisfy their key nonfunctional requirements. We propose adaptive, dynamic, and reactive resource provisioning and load distribution algorithms that heuristically optimize overall cost and response delays without violating essential legislative and regulatory requirements. Our simulation with realistic workload, network, and cloud characteristics shows that our method improves the state of the art in terms of availability, regulatory compliance, and QoE with acceptable sacrifice in cost and latency.","PeriodicalId":50919,"journal":{"name":"ACM Transactions on Autonomous and Adaptive Systems","volume":"32 1","pages":"13:1-13:21"},"PeriodicalIF":2.7,"publicationDate":"2014-10-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82342934","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Distributive Justice for Self-Organised Common-Pool Resource Management","authors":"J. Pitt, D. Busquets, S. Macbeth","doi":"10.1145/2629567","DOIUrl":"https://doi.org/10.1145/2629567","url":null,"abstract":"In this article, we complement Elinor Ostrom’s institutional design principles for enduring common-pool resource management with Nicholas Rescher’s theory of distributive justice based on the canon of legitimate claims. Two of Ostrom’s principles are that the resource allocation method should be congruent with the local environment, and that those affected by the allocation method (the appropriators) should participate in its selection. However, these principles do not say anything explicitly about the fairness of the allocation method or the outcomes it produces: for this, we need a mechanism for distributive justice. Rescher identified a number of different mechanisms, each of which had both its merits and demerits, and instead maintained that distributive justice consisted in identifying the legitimate claims in context, accommodating multiple claims in case of plurality, and reconciling them in case of conflict. Accordingly, we specify a logical axiomatisation of the principles with the canon of legitimate claims, whereby a set of claims is each represented as a voting function, which collectively determine the rank order in which resources are allocated. The appropriators vote on the weight attached to the scoring functions, and so self-organise the allocation method, taking into account both the plurality of and conflict between the claims. Therefore, the appropriators exercise collective choice over the method, and the method itself is congruent with the local environment, taking into account both the resources available and the relative claims of the appropriators. Experiments with a variant of the linear public good game show that this pluralistic self-organising approach produces a better balance of utility and fairness (for agents that comply with the rules of the game) compared to monistic or fixed approaches, provide “fairness over time” (a series of ostensibly unfair individual allocations is revealed to be cumulatively fair), and offer an intuition of how to resolve the free-rider phenomenon in provision and appropriation of common-pool resources.","PeriodicalId":50919,"journal":{"name":"ACM Transactions on Autonomous and Adaptive Systems","volume":"64 1","pages":"14:1-14:39"},"PeriodicalIF":2.7,"publicationDate":"2014-10-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"85644636","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"An Intelligent Agent for Bilateral Negotiation with Unknown Opponents in Continuous-Time Domains","authors":"Siqi Chen, Gerhard Weiss","doi":"10.1145/2629577","DOIUrl":"https://doi.org/10.1145/2629577","url":null,"abstract":"Automated negotiation among self-interested autonomous agents has gained tremendous attention due to the diversity of its broad range of potential real-world applications. This article deals with a prominent type of such negotiations, namely, multiissue negotiation that runs under continuous-time constraints and in which the negotiating agents have no prior knowledge about their opponents’ preferences and strategies. A negotiation strategy called Dragon is described that employs sparse pseudoinput Gaussian processes. Specifically, Dragon enables an agent (1) to precisely model the behavior of its opponents with comparably low computational load and (2) to make decisions effectively and adaptively in very complex negotiation settings. Extensive experimental results, based on a number of negotiation scenarios and state-of-the-art negotiating agents from Automated Negotiating Agents Competitions, are provided. Moreover, the robustness of our strategy is evaluated through both empirical game-theoretic and spatial evolutionary game-theoretic analysis.","PeriodicalId":50919,"journal":{"name":"ACM Transactions on Autonomous and Adaptive Systems","volume":"1 1","pages":"16:1-16:24"},"PeriodicalIF":2.7,"publicationDate":"2014-10-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88198742","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Utility-Based Mechanism for Structural Self-Organization in Service-Oriented MAS","authors":"E. Noguera, M. Rebollo, Matteo Vasirani, Alberto Fernández","doi":"10.1145/2651423","DOIUrl":"https://doi.org/10.1145/2651423","url":null,"abstract":"Structural relations established among agents influence the performance of decentralized service discovery process in multiagent systems. Moreover, distributed systems should be able to adapt their structural relations to changes in environmental conditions. In this article, we present a service-oriented multiagent systems, where agents initially self-organize their structural relations based on the similarity of their services. During the service discovery process, agents integrate a mechanism that facilitates the self-organization of their structural relations to adapt the structure of the system to the service demand. This mechanism facilitates the task of decentralized service discovery and improves its performance. Each agent has local knowledge about its direct neighbors and the queries received during discovery processes. With this information, an agent is able to analyze its structural relations and decide when it is more appropriate to modify its direct neighbors and select the most suitable acquaintances to replace them. The experimental evaluation shows how this self-organization mechanism improves the overall performance of the service discovery process in the system when the service demand changes.","PeriodicalId":50919,"journal":{"name":"ACM Transactions on Autonomous and Adaptive Systems","volume":"8 1","pages":"12:1-12:24"},"PeriodicalIF":2.7,"publicationDate":"2014-10-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77769068","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The Complexity of Adding Multitolerance","authors":"Jingshu Chen, Ali Ebnenasir, S. Kulkarni","doi":"10.1145/2629664","DOIUrl":"https://doi.org/10.1145/2629664","url":null,"abstract":"We focus on the problem of adding multitolerance to an existing fault-intolerant program. A multitolerant program tolerates multiple classes of faults and provides a potentially different level of fault tolerance to each of them. We consider three levels of fault tolerance, namely failsafe (i.e., satisfy safety in the presence of faults), nonmasking (i.e., recover to legitimate states after the occurrence of faults), and masking (both). For the case where the program is subject to two classes of faults, we consider six categories of multitolerant programs—FF, FN, FM, MM, MN, and NN, where F, N, and M represent failsafe, nonmasking, and masking levels of tolerance provided to each class of fault. We show that the problem of adding FF, NN, and MN multitolerance can be solved in polynomial time (in the state space of the program). However, the problem is NP-complete for adding FN, MM, and FM multitolerance. We note that the hardness of adding MM and FM multitolerance is especially atypical given that MM and FM multitolerance can be added efficiently under more restricted scenarios where multiple faults occur simultaneously in the same computation. We also present heuristics for managing the complexity of MM multitolerance. Finally, we present real-world multitolerant programs and discuss the trade-off involved in design decisions while developing such programs.","PeriodicalId":50919,"journal":{"name":"ACM Transactions on Autonomous and Adaptive Systems","volume":"26 1","pages":"15:1-15:33"},"PeriodicalIF":2.7,"publicationDate":"2014-10-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82635031","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Improving the Effectiveness of Testing Pervasive Software via Context Diversity","authors":"Huai Wang, W. Chan, T. Tse","doi":"10.1145/2620000","DOIUrl":"https://doi.org/10.1145/2620000","url":null,"abstract":"Context-aware pervasive software is responsive to various contexts and their changes. A faulty implementation of the context-aware features may lead to unpredictable behavior with adverse effects. In software testing, one of the most important research issues is to determine the sufficiency of a test suite to verify the software under test. Existing adequacy criteria for testing traditional software, however, have not explored the dimension of serial test inputs and have not considered context changes when constructing test suites. In this article, we define the concept of context diversity to capture the extent of context changes in serial inputs and propose three strategies to study how context diversity may improve the effectiveness of the data-flow testing criteria. Our case study shows that the strategy that uses test cases with higher context diversity can significantly improve the effectiveness of existing data-flow testing criteria for context-aware pervasive software. In addition, test suites with higher context diversity are found to execute significantly longer paths, which may provide a clue that reveals why context diversity can contribute to the improvement of effectiveness of test suites.","PeriodicalId":50919,"journal":{"name":"ACM Transactions on Autonomous and Adaptive Systems","volume":"13 1","pages":"9:1-9:28"},"PeriodicalIF":2.7,"publicationDate":"2014-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81713282","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}