AD '11Pub Date : 2011-03-25DOI: 10.1145/1966895.1966899
P. Baumann
{"title":"Accelerating computationally intensive queries on massive earth science data: (system demonstration)","authors":"P. Baumann","doi":"10.1145/1966895.1966899","DOIUrl":"https://doi.org/10.1145/1966895.1966899","url":null,"abstract":"We present a technique to speed up array query processing, just-in-time compilation, which is novel on arrays. Suitable query fragments are detected, compiled, and linked into the server dynamically. This technique has been integrated in an operational array DBMS where its impact on efficiency can be studied in a holistic environment.\u0000 We will demonstrate this on 1D to 4D real-life spatio-temporal raster data sets taken from the Earth Sciences. Performance results on computationally intensive queries are encouraging and can be examined interactively in a variety of demonstration examples.","PeriodicalId":424540,"journal":{"name":"AD '11","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-03-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133696760","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AD '11Pub Date : 2011-03-25DOI: 10.1145/1966895.1966898
Emad Soroush, M. Balazinska
{"title":"Hybrid merge/overlap execution technique for parallel array processing","authors":"Emad Soroush, M. Balazinska","doi":"10.1145/1966895.1966898","DOIUrl":"https://doi.org/10.1145/1966895.1966898","url":null,"abstract":"Whether in business or science, multi-dimensional arrays are a common abstraction in data analytics and many systems exist for efficiently processing arrays. As dataset grow in size, it is becoming increasingly important to process these arrays in parallel. In this paper, we discuss different types of array operations and review how they can be processed in parallel using two different existing techniques. The first technique, which we call merge, consists in partitioning an array, processing the partitions in parallel, then merging the results to reconcile computations that span partition boundaries. The second technique, which we call overlap, consists in partitioning an array into subarrays that overlap by a given number of cells along each dimension. Thanks to this overlap, the array partitions can be processed in parallel without any merge phase. We discuss when each technique can be applied to an array operation. We show that even for a single array operation, a different approach may yield the best performance for different regions of an array. Following this observation, we introduce a new parallel array processing technique that combines the merge and overlap approaches. Our technique enables a parallel array processing system to mix-and-match the merge and overlap techniques within a single operation on an array. Through experiments on real, scientific data, we show that this hybrid approach outperforms the other two techniques.","PeriodicalId":424540,"journal":{"name":"AD '11","volume":"131 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-03-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121343962","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AD '11Pub Date : 2011-03-25DOI: 10.1145/1966895.1966897
L. Dobos, A. Szalay, J. Blakeley, T. Budavári, I. Csabai, Dragan Tomic, M. Milovanović, M. Tintor, Andrija Jovanovic
{"title":"Array requirements for scientific applications and an implementation for microsoft SQL server","authors":"L. Dobos, A. Szalay, J. Blakeley, T. Budavári, I. Csabai, Dragan Tomic, M. Milovanović, M. Tintor, Andrija Jovanovic","doi":"10.1145/1966895.1966897","DOIUrl":"https://doi.org/10.1145/1966895.1966897","url":null,"abstract":"This paper outlines certain scenarios from the fields of astrophysics and fluid dynamics simulations which require high performance data warehouses that support array data type. A common feature of all these use cases is that subsetting and preprocessing the data on the server side (as far as possible inside the database server process) is necessary to avoid the client-server overhead and to minimize IO utilization. Analyzing and summarizing the requirements of the various fields help software engineers to come up with a comprehensive design of an array extension to relational database systems that covers a wide range of scientific applications.\u0000 We also present a working implementation of an array data type for Microsoft SQL Server 2008 to support large-scale scientific applications. We introduce the design of the array type, results from a performance evaluation, and discuss the lessons learned from this implementation. The library can be downloaded from our website at http://voservices.net/sqlarray/.","PeriodicalId":424540,"journal":{"name":"AD '11","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-03-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114732832","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AD '11Pub Date : 2011-03-25DOI: 10.1145/1966895.1966900
M. Folk, G. Heber, Q. Koziol, Elena Pourmal, Dana Robinson
{"title":"An overview of the HDF5 technology suite and its applications","authors":"M. Folk, G. Heber, Q. Koziol, Elena Pourmal, Dana Robinson","doi":"10.1145/1966895.1966900","DOIUrl":"https://doi.org/10.1145/1966895.1966900","url":null,"abstract":"In this paper, we give an overview of the HDF5 technology suite and some of its applications. We discuss the HDF5 data model, the HDF5 software architecture and some of its performance enhancing capabilities.","PeriodicalId":424540,"journal":{"name":"AD '11","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-03-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123596093","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AD '11Pub Date : 2010-12-01DOI: 10.1145/1966895.1966896
M. Kersten, Y. Zhang, M. Ivanova, N. Nes
{"title":"SciQL, a query language for science applications","authors":"M. Kersten, Y. Zhang, M. Ivanova, N. Nes","doi":"10.1145/1966895.1966896","DOIUrl":"https://doi.org/10.1145/1966895.1966896","url":null,"abstract":"Scientific applications are still poorly served by contemporary relational database systems. At best, the system provides a bridge towards an external library using user-defined functions, explicit import/export facilities or linked-in Java/C# interpreters. Time has come to rectify this with SciQL1, a SQL query language for scientific applications with arrays as first class citizens. It provides a seamless symbiosis of array-, set-, and sequence- interpretation using a clear separation of the mathematical object from its underlying implementation. A key innovation is to extend value-based grouping in SQL:2003 with structural grouping, i.e., fixed-sized and unbounded groups based on explicit relationships between their dimension attributes. It leads to a generalization of window-based query processing with wide applicability in science domains. This paper is focused on the language features, extensively illustrated with examples of its intended use.","PeriodicalId":424540,"journal":{"name":"AD '11","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2010-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115509791","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}