{"title":"Streaming Classical Multidimensional Scaling","authors":"Xi Zhang, Hao Huang, K. Mueller, Shinjae Yoo","doi":"10.1109/NYSDS.2018.8538942","DOIUrl":"https://doi.org/10.1109/NYSDS.2018.8538942","url":null,"abstract":"Multidimensional scaling (MDS) is a widely used embedding method that preserves the similarity among data items in the embedded low dimensional space. While MDS has shown great versatility, it is computationally expensive and non-scalable as it requires operation on the whole data distance matrix. This can be problematic when the dataset is not fixed but is constantly growing over time, such as in streaming applications. To address this problem, we introduce a streaming, one-pass, limited memory version of the classical MDS algorithm, named scMDS, specifically designed for streaming high-dimensional data. In scMDS, m-dimensional samples are arriving sequentially and are embedded in a k-dimensional subspace (k ≪ m) with high approximation quality to batch classical MDS (cMDS). It also reduces the time complexity from O(mn2) to O($mn_{t^{2}}$), where n is the size of the data stream and $n_{t}$ is the size of a data batch at time-streaming data processing we pair our algorithm with a t (nt≪ n). To overcome the concept-drift during novel realignment method.","PeriodicalId":366010,"journal":{"name":"2018 New York Scientific Data Summit (NYSDS)","volume":"72 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122715206","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Data Analysis on Multivariate Image Set","authors":"Shruti K. Nair, S. Ha, Wei Xu","doi":"10.1109/NYSDS.2018.8538941","DOIUrl":"https://doi.org/10.1109/NYSDS.2018.8538941","url":null,"abstract":"An image set can include not just the images themselves but also the extracted features, metadata and so on. For example, x-ray images obtained from synchrotron beamlines are large-scale highdynamic-range data depicting a variety of material properties after incorporating scientific analysis results. Previously, we presented a framework MultiSciView as an image set visualization and exploration system for x-ray scattering data. This tool is general enough to deal with any multivariate images. In this work, we aim to complement it with a set of data analysis modules. First, we present feature analysis by proposing a new correlation metric to reduce the data redundancy. Then we encode each image as a high dimensional vector and analyze the patterns hidden in the image set. Finally, we add an auxiliary visualization to plot the average and entropy images of the interested subset. We conducted one case study to show that our system can effectively analyze the image set, identify preferred image patterns, anomalous images and erroneous experimental settings. Eventually a better comprehension of the material nanostructure properties can be achieved.","PeriodicalId":366010,"journal":{"name":"2018 New York Scientific Data Summit (NYSDS)","volume":"109 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130742173","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nicole Soder, C. Deluca, David Biersach, M. Dephillips
{"title":"Assessing the Cryptographic Strength of RSA Moduli Using Algorithmic Entropy Reduction in Bivariate Polynomials","authors":"Nicole Soder, C. Deluca, David Biersach, M. Dephillips","doi":"10.1109/NYSDS.2018.8538957","DOIUrl":"https://doi.org/10.1109/NYSDS.2018.8538957","url":null,"abstract":"We develop an innovative approach to factoring semiprimes, numbers that are the product of two large primes. These composite numbers form the foundation of the widely used RSA encryption method which protects vast amounts of digitally transmitted data. With the current rate of technological growth, experts believe that the best-known factoring algorithms would take thousands of years to break a single RSA composite. But with a more efficient method, government agencies could quickly decrypt big data sets to better confront national security concerns. Our technique, Algorithmic Entropy Reduction in Bivariate Polynomials, takes a different approach from traditional methods in that it addresses the binary expansion of the primes without relying on number theory. The method evaluates if a valid solution can exist for a given bivariate polynomial (BVP) representation of each prime at every bit position. In doing so, the method ascertains each binary digit from least significant bit on the right-hand side to most significant bit on the left-hand side (entropy reduction). We execute the algorithm by generating code in C++ and Python and reduce its computational work by a factor of two using multiprocessing.","PeriodicalId":366010,"journal":{"name":"2018 New York Scientific Data Summit (NYSDS)","volume":"222 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115534108","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jin Xu, Shilpi Bhattacharyya, D. Katramatos, Shinjae Yoo, Meng Yue
{"title":"Forecasting Smart Grid Load on the Wire","authors":"Jin Xu, Shilpi Bhattacharyya, D. Katramatos, Shinjae Yoo, Meng Yue","doi":"10.1109/NYSDS.2018.8538948","DOIUrl":"https://doi.org/10.1109/NYSDS.2018.8538948","url":null,"abstract":"Here we present a use case of analysis on the wire framework applied to smart grid load forecasting application. In our preliminary study, we were able to demonstrate electrical load forecasting on each household level in streaming fashion.","PeriodicalId":366010,"journal":{"name":"2018 New York Scientific Data Summit (NYSDS)","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124649100","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Visualization and quantum computation of Moiré superconductivity in bilayer graphene, carbon nanocones and nanostrips","authors":"S. Elghazoly, M. McGuigan","doi":"10.1109/NYSDS.2018.8538949","DOIUrl":"https://doi.org/10.1109/NYSDS.2018.8538949","url":null,"abstract":"The purpose of this project is to visualize and study the electronic properties of carbon nanostructures. The principal structures studied are AA and AB (Bernal) stacked graphene bilayers, an 80C nanocone with pentagonal defects at its apex, and a 16C nanoring. Graphene and other based nanostructures have attracted much attention due to their unique electronic properties. For example, bilayer graphene shows superconductivity at certain magic angles of rotation. Open source molecular modeling and visualization programs like SAMSON, Avogadro, and VMD are utilized to generate the structures above. Using bond information in order to find nearest neighbors, an adjacency matrix can be generated from which the energy eigenvalues are solvable and are used to visualize the dispersion relation, free energy, persistent current, Green’s function, and entanglement entropy. The goal of this project is to study the effect of rotation on the electronic properties of bilayer graphene, particularly as it pertains to the magic angle where superconductivity occurs, as well as the electronic properties of the bilayer Moiré patterns which emerge due to the superposition of the identical lattices at different angles. An important goal is to incorporate phonon interactions into the tight-binding model for carbon structures, and to utilize quantum based computing as a tool to find the ground state properties of these structures, particularly in the terminating case where boundary conditions have an effect.","PeriodicalId":366010,"journal":{"name":"2018 New York Scientific Data Summit (NYSDS)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126442612","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Line C. Pouchard, K. Huck, Gyorgy Matyasfalvi, Dingwen Tao, L. Tang, H. V. Dam, Shinaje Yoo
{"title":"Prescriptive provenance for streaming analysis of workflows at scale","authors":"Line C. Pouchard, K. Huck, Gyorgy Matyasfalvi, Dingwen Tao, L. Tang, H. V. Dam, Shinaje Yoo","doi":"10.1109/NYSDS.2018.8538951","DOIUrl":"https://doi.org/10.1109/NYSDS.2018.8538951","url":null,"abstract":"We extend our approach capturing and relating the provenance and performance metrics of computational workflows as a diagnostic tool for runtime optimization and placement. One important challenge is the volume of extracted data, both for performance metrics and provenance, even when specifying filters and focusing on quantities of interest in a simulation. We reduce this data by performing anomaly detection on streaming data and store provenance for the detected anomalies, an approach we call prescriptive provenance. This paper discusses the Chimbuko architecture enabling the approach. We present the use of a protein structure propagation workflow based on NWChemEx. We are testing algorithms for anomaly detection and present preliminary results here obtained with Local Outlier Factor. While scaling remains a challenge, these results show that our robust Chimbuko architecture for streaming analysis with prescriptive provenance is a promising approach.","PeriodicalId":366010,"journal":{"name":"2018 New York Scientific Data Summit (NYSDS)","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122442946","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Performance Evaluation for Tape Storage Data Recall with T10KD Drive","authors":"Guangwei Che, D. Yu","doi":"10.1109/NYSDS.2018.8538947","DOIUrl":"https://doi.org/10.1109/NYSDS.2018.8538947","url":null,"abstract":"Tape library storage is the best solution for archiving big data due to high capacity, low cost and long retention. However, as the nature of sequential media, its main drawback versus disk storage is the slow performance for data recall in random access order. With emerging of the modern technology from the enterprise tape drives, Recommended Access Order (RAO) should improve the data recall performance considerably.The Scientific Data and Computing Center (SDCC) in Brookhaven National Lab (BNL) is operating a tape library storage system which contains more than 100 PB data. It is managed by High Performance Storage System (HPSS) software. Starting from HPSS 7.5.1 release, a new feature named Tape Ordered Recall (TOR) has offered a great advantage for large amount of data recall. It is designed to reduce the file access time from tape media. In our test with Oracle T10K enterprise drive, TOR’s tape recall rates for small files (1GB) is almost 4 times of the unordered recall. TOR supports both RAO and non-RAO drives. It will use the RAO feature if available. Otherwise, it makes a linear offset ordering to recall the data. Additionally, Quaid is a smart recall tool introduced in HPSS7.5.1 as well. It uses new batch stage API to stage files in the background, improves the tape data recall performance significantly. Efficient Retrieval and Access to Data Archived on Tape, aka ERADAT, is a tape storage data recall tool developed mainly in SDCC. It sorts the staging requests based on file’s logical position order before submitting jobs to HPSS. Over a decade of utilization and improvement, ERADAT has been proven as a great performer serving BNL’s worldwide scientific community. In this paper we will present the test results for each individual tool under certain test setting and configuration, analyze and evaluate their performance advantage and disadvantage, find the best solution for BNL’s tape storage production system for different data characteristics.","PeriodicalId":366010,"journal":{"name":"2018 New York Scientific Data Summit (NYSDS)","volume":"73 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122648545","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A. Malik, Micheal Lu, N. Wang, Yeiwei Lin, Shinjae Yoo
{"title":"Detailed Performance Analysis of Distributed Tensorflow on a GPU Cluster using Deep Learning Algorithms","authors":"A. Malik, Micheal Lu, N. Wang, Yeiwei Lin, Shinjae Yoo","doi":"10.1109/NYSDS.2018.8538946","DOIUrl":"https://doi.org/10.1109/NYSDS.2018.8538946","url":null,"abstract":"Long training times for building a high accuracy deep neural networks (DNNs) is impeding research for new DNN architectures. For example, time for training GoogleNet with the ImageNet dataset on a single Nvidia K20 GPU almost takes 25 days. Therefore, there is a great need in the AI community to speed up the training phase, especially when using a large dataset. For this, we need Distributed Deep Neural Networks (DDNNs) that can scale well with more computation resources. However, this involves two challenges.First, the deep learning framework or training library must support inter-node communication. Second, the user must modify the code to take advantage of the inter-node communication. The changes to the code can be minimal to significant depending upon the user expertize in the distributed systems. Current DNN frameworks support distributed learning using MPI. However, these frameworks come with poorly understood overheads associated with communication and data management. Tensorflow provides APIs for distributed learning using MPI programming model and gRPC. These APIs are not easy to use for a domain expert for designing an efficient distributed learning model. Recently, Uber Inc. provides the Horovod Framework which gives a fast and easy way to support distributed learning using Tensorflow, Pytorach, and Keras. In this paper we provide a detailed performance analysis of distributed Tensorflow using Horovod. We implemented distributed learning for AlexNet, GoogleNet, and ResNet50 using Horovod. We used Nvidia K 40,K80, and P100 GPUs for our experimentation. We used synthetic image data with different runtime variables (batch size and number of GPUs). Our results shows that the Horovod framework gives almost linear throughput (images/sec) scalability up to 256 GPUs.","PeriodicalId":366010,"journal":{"name":"2018 New York Scientific Data Summit (NYSDS)","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127665073","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Renuka Diwan, Ayesha Rizvi, Shilpi Bhattacharyya, D. Katramatos, K. Yager
{"title":"Application of Analysis on the Wire to Streaming NSLS-II Beamline Data","authors":"Renuka Diwan, Ayesha Rizvi, Shilpi Bhattacharyya, D. Katramatos, K. Yager","doi":"10.1109/NYSDS.2018.8538955","DOIUrl":"https://doi.org/10.1109/NYSDS.2018.8538955","url":null,"abstract":"The National Synchrotron Light Source II (NSLSII) is a scientific facility that generates vast amounts of many different types of data, including 2D images produced by X-ray scattering experiments. These images are captured and sent from the detectors at rates that usually range from several tens to several hundreds of frames per second. Such high incoming data rates can be difficult to keep up with if real/near-real time image analysis is required. We propose to use the Analysis on Wire framework to introduce intermediate processing while these images are in transit from the detector to their destination. As the 2D images come out of the detector, we capture the corresponding raw data packets while they travel through the network. We use TCP stream reconstruction to correctly reassemble the image data and subsequently execute the analysis code in parallel to analyze multiple images simultaneously. Our measurements indicate that the approach is feasible and effective.","PeriodicalId":366010,"journal":{"name":"2018 New York Scientific Data Summit (NYSDS)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125058502","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Visualizing effective potentials and using the IBM-Q to study quantum field theory models in 0+1 dimensions","authors":"Christopher Kane, M. McGuigan","doi":"10.1109/NYSDS.2018.8538962","DOIUrl":"https://doi.org/10.1109/NYSDS.2018.8538962","url":null,"abstract":"This work explores the utility of two novel techniques applied to studying quantum field theory, namely quantum computing and visualizing effective potentials. We studied the ground state properties of the anharmonic oscillator and the Wess-Zumino model using the variational quantum eigensolver algorithm. We performed calculations for three and four qubit systems using IBM’s open source quantum computing simulation software. Using effective potentials, we study a simple dark matter system with a Higgs and dark Higgs like particle interacting through a porthole fermion.","PeriodicalId":366010,"journal":{"name":"2018 New York Scientific Data Summit (NYSDS)","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121389488","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}