Jeffrey Pang, Phillip B. Gibbons, M. Kaminsky, S. Seshan, Haifeng Yu
{"title":"Defragmenting DHT-based Distributed File Systems","authors":"Jeffrey Pang, Phillip B. Gibbons, M. Kaminsky, S. Seshan, Haifeng Yu","doi":"10.1109/ICDCS.2007.97","DOIUrl":"https://doi.org/10.1109/ICDCS.2007.97","url":null,"abstract":"Existing DHT-based file systems use consistent hashing to assign file blocks to random machines. As a result, a user task accessing an entire file or multiple files needs to retrieve blocks from many different machines. This paper demonstrates that significant availability and performance gains can be achieved if instead, users are able to retrieve all the data needed for a given task from only a few DHT nodes. We explore the design and implications of such a \"defragmented\" DHT-based distributed file system, called D2, that also maintains important DHT properties like storage load balance. We show using real-world file system traces that a simple key encoding scheme is sufficient to maintain good defragmentation for most user tasks. Using both simulation and an actual 1,000 node deployment, we show that D2 increases availability by over an order of magnitude and improves user-perceived latency by 30- 100% compared to a traditional design.","PeriodicalId":170317,"journal":{"name":"27th International Conference on Distributed Computing Systems (ICDCS '07)","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114375599","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Embedded Gossip: Lightweight Online Measurement for Large-Scale Applications","authors":"Wenbin Zhu, P. Bridges, A. Maccabe","doi":"10.1109/ICDCS.2007.107","DOIUrl":"https://doi.org/10.1109/ICDCS.2007.107","url":null,"abstract":"For large-scale parallel applications, lightweight online monitoring can enable a wide range of online adaptations, including load balancing, power management, and progress monitoring. The processing and monitoring overhead of centralized global tracing techniques make them unsuitable for such tasks. Purely local tools, on the other hand, fail to provide the global information necessary for many desirable online adaptations of large-scale applications. In this paper, we describe a novel distributed online measurement method for large-scale applications called Embedded Gossip (EG). EG works by piggybacking performance information about application behavior on existing application messages and merging received information with previously known data in a fashion customized to the needs of a particular monitoring task. EG thus provides each process with both local and global views of application behavior with low overhead. To illustrate the capabilities of Embedded Gossip, we also show that it disseminates global information in a timely fashion for a wide range of monitoring tasks, including critical path profiling, workload imbalance monitoring, and progress monitoring. This global information has a wide range of potential uses, including imbalance detection for load balancing and energy management tools, progress monitoring for batch schedulers, and a wide range of other performance debugging and optimization techniques.","PeriodicalId":170317,"journal":{"name":"27th International Conference on Distributed Computing Systems (ICDCS '07)","volume":"108 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114955656","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Non-Threshold based Event Detection for 3D Environment Monitoring in Sensor Networks","authors":"Mo Li, Yunhao Liu, Lei Chen","doi":"10.1109/ICDCS.2007.123","DOIUrl":"https://doi.org/10.1109/ICDCS.2007.123","url":null,"abstract":"Event detection is a crucial task for wireless sensor network applications, especially environment monitoring. Existing approaches for event detection are mainly based on some predefined threshold values, and thus are often inaccurate and incapable of capturing complex events. For example, in coal mine monitoring scenarios, gas leakage or water osmosis can hardly be described by the overrun of specified attribute thresholds, but some complex pattern in the full-scale view of the environmental data. To address this issue, we propose a non-threshold based approach for the real 3D sensor monitoring environment. We employ energy-efficient methods to collect a time series of data maps from the sensor network and detect complex events through matching the gathered data to spatio-temporal data patterns. Finally, we conduct trace driven simulations to prove the efficacy and efficiency of this approach on detecting events of complex phenomena from real-life records.","PeriodicalId":170317,"journal":{"name":"27th International Conference on Distributed Computing Systems (ICDCS '07)","volume":"67 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128598102","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
O. Benjelloun, H. Garcia-Molina, Heng Gong, H. Kawai, T. E. Larson, David Menestrina, Sutthipong Thavisomboon
{"title":"D-Swoosh: A Family of Algorithms for Generic, Distributed Entity Resolution","authors":"O. Benjelloun, H. Garcia-Molina, Heng Gong, H. Kawai, T. E. Larson, David Menestrina, Sutthipong Thavisomboon","doi":"10.1109/ICDCS.2007.96","DOIUrl":"https://doi.org/10.1109/ICDCS.2007.96","url":null,"abstract":"Entity resolution (ER) matches and merges records that refer to the same real-world entities, and is typically a compute-intensive process due to complex matching functions and high data volumes. We present a family of algorithms, D-Swoosh, for distributing the ER workload across multiple processors. The algorithms use generic match and merge functions, and ensure that new merged records are distributed to processors that may have matching records. We perform a detailed performance evaluation on a testbed of 15 processors. Our experiments use actual comparison shopping data provided by Yahoo!.","PeriodicalId":170317,"journal":{"name":"27th International Conference on Distributed Computing Systems (ICDCS '07)","volume":"70 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134379960","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"SCAP: Smart Caching inWireless Access Points to Improve P2P Streaming","authors":"Enhua Tan, Lei Guo, Songqing Chen, Xiaodong Zhang","doi":"10.1109/ICDCS.2007.134","DOIUrl":"https://doi.org/10.1109/ICDCS.2007.134","url":null,"abstract":"The increasing number of wireless users in Internet P2P applications causes two new performance problems due to the requirement of uploading the downloaded traffic for other peers, limited bandwidth of wireless communications, and resource competition between the access point and wireless stations. First, an active P2P wireless user can significantly reduce the downloading throughput of other wireless users in the WLAN. Second, the slowdown of a P2P wireless user communication can also delay its relay and data sharing service for other dependent wired/wireless peers. In order to address these problems, in this paper, we propose an efficient caching mechanism called SCAP (Smart Caching in Access Points). Conducting intensive Internet measurements on representative P2P streaming applications, we observe a high percentage of duplicated data packets in successive downloading and uploading data streams. Through duplication detection and caching at the access point, these duplicated packets can be compressed so that the uploading traffic in the WLAN is significantly reduced. Our prototype-based experimental evaluation demonstrates that by effectively reducing the redundant P2P traffic in the WLAN, SCAP improves the throughput of the WLAN by up to 88% and reduces the response delay to other Internet users meanwhile.","PeriodicalId":170317,"journal":{"name":"27th International Conference on Distributed Computing Systems (ICDCS '07)","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133002477","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Protocol Design for Dynamic Delaunay Triangulation","authors":"Dong-Young Lee, S. Lam","doi":"10.1109/ICDCS.2007.130","DOIUrl":"https://doi.org/10.1109/ICDCS.2007.130","url":null,"abstract":"Delaunay triangulation (DT) is a useful geometric structure for networking applications. In this paper we investigate the design of join, leave, and maintenance protocols to construct and maintain a distributed DT dynamically. We define a distributed DT and present a necessary and sufficient condition for a distributed DT to be correct. This condition is used as a guide for protocol design. We present join and leave protocols as well as correctness proofs for serial joins and leaves. In addition, to handle concurrent joins and leaves as well as node failures, we present a maintenance protocol. An accuracy metric is defined for a distributed DT. Experimental results show that our join, leave and maintenance protocols are scalable, and they achieve high accuracy for systems under churn and with node failures. We also present application protocols for greedy routing, clustering, broadcast, and multicast within a radius, and discuss and prove their correctness.","PeriodicalId":170317,"journal":{"name":"27th International Conference on Distributed Computing Systems (ICDCS '07)","volume":"232 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123254898","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Location Verification Algorithms forWireless Sensor Networks","authors":"Yawen Wei, Zhen Yu, Y. Guan","doi":"10.1109/ICDCS.2007.117","DOIUrl":"https://doi.org/10.1109/ICDCS.2007.117","url":null,"abstract":"Because the knowledge of sensors' locations is very important to many location-based applications proposed for wireless sensor networks, many secure localization and location verification schemes have been proposed to provide robust location estimations for sensors. We propose two lightweight location verification algorithms, namely, greedy filtering by matrix (GFM) and trustability indicator (TI). Unlike other schemes, our algorithms do not require any specialized hardware or deployment knowledge. In GFM algorithm, the verification center(VC) calculates several matrices based on sensors' estimated locations and their neighborhood observations, and uses these matrixes to identify and revoke inconsistent locations. In TI algorithm, VC calculates trustability indicators for the sensors and reject those whose indicators are below a threshold. Simulation results demonstrated the effectiveness of both algorithms in detecting abnormal location and their robustness under attacks.","PeriodicalId":170317,"journal":{"name":"27th International Conference on Distributed Computing Systems (ICDCS '07)","volume":"41 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123002058","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Testing Security Properties of Protocol Implementations - a Machine Learning Based Approach","authors":"Guoqiang Shu, David Lee","doi":"10.1109/ICDCS.2007.147","DOIUrl":"https://doi.org/10.1109/ICDCS.2007.147","url":null,"abstract":"Security and reliability of network protocol implementations are essential for communication services. Most of the approaches for verifying security and reliability, such as formal validation and black-box testing, are limited to checking the specification or conformance of implementation. However, in practice, a protocol implementation may contain engineering details, which are not included in the system specification but may result in security flaws. We propose a new learning-based approach to systematically and automatically test protocol implementation security properties. Protocols are specified using symbolic parameterized extended finite state machine (SP-EFSM) model, and an important security property - message confidentiality under the general Dolev-Yao attacker model - is investigated. The new testing approach applies black-box checking theory and a supervised learning algorithm to explore the structure of an implementation under test while simulating the teacher with a conformance test generation scheme. We present the testing procedure, analyze its complexity, and report experimental results.","PeriodicalId":170317,"journal":{"name":"27th International Conference on Distributed Computing Systems (ICDCS '07)","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123325181","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Selfishness, Not Always A Nightmare: Modeling Selfish MAC Behaviors in Wireless Mobile Ad Hoc Networks","authors":"Lin Chen, J. Leneutre","doi":"10.1109/ICDCS.2007.138","DOIUrl":"https://doi.org/10.1109/ICDCS.2007.138","url":null,"abstract":"In wireless mobile ad hoc networks where nodes are selfish and non-cooperative, a natural and crucial question is how well or how bad the MAC layer protocol IEEE 802.11 DCF performs. In this paper, we study this question by modeling the selfish MAC protocol as a non- cooperative repeated game where players follow the TIT- FOR-TAT (TFT) strategy which is regarded as the best strategy in such environments. We show for single-hop ad hoc networks the game admits a number of Nash Equilibria (NE). We then perform NE refinement to eliminate the inefficient NE and show that there exists one efficient NE maximizing both local and global payoff. We also propose an algorithm to approach the efficient NE. We then extend our efforts to multi-hop case by showing that the game converges to a NE which may not be globally optimal but quasi- optimal in the sense that the global payoff is only slightly less than the optimal case. As conclusion, we answer the posed question by showing that selfishness does not always lead to network collapse. On the contrary, it can help the network operate at a NE globally which is optimal or quasi-optimal under the condition that players are long-sighted and follow the TFT strategy.","PeriodicalId":170317,"journal":{"name":"27th International Conference on Distributed Computing Systems (ICDCS '07)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128981802","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Fast Algorithms for Heavy Distinct Hitters using Associative Memories","authors":"Nagender Bandi, D. Agrawal, A. E. Abbadi","doi":"10.1109/ICDCS.2007.110","DOIUrl":"https://doi.org/10.1109/ICDCS.2007.110","url":null,"abstract":"Real-time detection of worm attacks, port scans and distributed denial of service (DDoS) attacks, as network packets belonging to these security attacks flow through a network router, is of paramount importance. In a typical worm attack, a worm infected host tries to spread the worm by scanning a number of other hosts thus resulting in significant number of network connections at an intermediate router. Detecting such attacks amounts to finding all hosts that are associated with unusually high number of other hosts, which is equivalent to solving the classic heavy distinct hitter problem over data streams. While several heavy distinct hitter solutions have been proposed and evaluated in a standard CPU setting, most of the above applications typically execute on special networking architectures called network processing units (NPUs). These NPUs interface with special associative memories known as the ternary content addressable memories (TCAMs) to provide gigabit rate forwarding at network routers. In this paper, we describe how the integrated architecture of NPU and TCAMs can be exploited to develop high-speed solutions for heavy distinct hitters.","PeriodicalId":170317,"journal":{"name":"27th International Conference on Distributed Computing Systems (ICDCS '07)","volume":"220 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116365580","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}