Proceedings of International Symposium on Grids & Clouds 2019 — PoS(ISGC2019)最新文献

{"title":"A Resource-saving Job Monitoring System of High Performance Computing using Parent and Child Process","authors":"Kajornsak Piyoungkorn, Phithak Thaenkaew, C. Vorakulpipat","doi":"10.22323/1.351.0034","DOIUrl":"https://doi.org/10.22323/1.351.0034","url":null,"abstract":"High performance computing has been more important in the past decade. In the present day, data used for processing becomes enormous. Where a high performance computing resource is needed to help process the data. Some scientific experiments involving big data. Which requires high speed data processing cannot be done by an ordinary computer system. Also, there is a need for support of parallel processing. The solution starts by dividing the job into a number of sections to be processed into parts and the processing unit each processing unit of data at the same time. Then, the system sends the calculated result back to the compiled. This mechanism will speed up the processing time to complete the task and generate more output at the same time. Therefore, a solution in this study is to maximize efficiency when using the resources of the computer which involves the processing power of the processor (CPU Cores).When the HPC system has a large number of concurrent users and requests processing resources that do not match the actual usage. Therefore requires a system to detect job requests that use inefficient computing resources to help users and system administrators to work effectively.","PeriodicalId":106243,"journal":{"name":"Proceedings of International Symposium on Grids & Clouds 2019 — PoS(ISGC2019)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125848819","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Toward Single Sign-on Establishment for Inter-Cloud Environment","authors":"E. Sakane, Takeshi Nishimura, K. Aida, Motonori Nakamura","doi":"10.22323/1.351.0028","DOIUrl":"https://doi.org/10.22323/1.351.0028","url":null,"abstract":"This paper investigates a mechanism that establishes single sign-on for inter-cloud computing environment built as the optimized result of the needs of users. Arranging requirements and issues for the mechanism, a single sign-on system for an inter-cloud computing environment is presented. As concrete service in the inter-cloud environment, we deal with Amazon Web Service with SAML version 2.0 and implement a prototype system. We also evaluate the prototype implementation and consider applicability to the other services.","PeriodicalId":106243,"journal":{"name":"Proceedings of International Symposium on Grids & Clouds 2019 — PoS(ISGC2019)","volume":"190 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132346927","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Improving efficiency of analysis jobs in CMS","authors":"T. Ivanov, S. Belforte, M. Wolf, M. Mascheroni, A. P. Yzquierdo, J. Letts, J. Hernández, L. Cristella, D. Ciangottini, J. Balcas, A. Woodard, K. H. Anampa, B. Bockelman, D. Foyo","doi":"10.1051/epjconf/201921403006","DOIUrl":"https://doi.org/10.1051/epjconf/201921403006","url":null,"abstract":"Hundreds of physicists analyze data collected by the Compact Muon Solenoid (CMS) experiment at the Large Hadron Collider using the CMS Remote Analysis Builder and the CMS global pool to exploit the resources of the Worldwide LHC Computing Grid. Efficient use of such an extensive and expensive resource is crucial. At the same time, the CMS collaboration is committed to minimizing time to insight for every scientist, by pushing for fewer possible access restrictions to the full data sample and supports the free choice of applications to run on the computing resources. Supporting such variety of workflows while preserving efficient resource usage poses special challenges. In this paper we report on three complementary approaches adopted in CMS to improve the scheduling efficiency of user analysis jobs: automatic job splitting, automated run time estimates and automated site selection for jobs.","PeriodicalId":106243,"journal":{"name":"Proceedings of International Symposium on Grids & Clouds 2019 — PoS(ISGC2019)","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126371743","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Simulation of the cache hit rate for data readout at the Tokyo Tier-2 center","authors":"T. Kishimoto, J. Tanaka, T. Mashimo, M. Kaneda, N. Matsui","doi":"10.22323/1.351.0030","DOIUrl":"https://doi.org/10.22323/1.351.0030","url":null,"abstract":"The Tokyo Tier-2 center, which is located in the International Center for Elementary Particle Physics at the University of Tokyo, provides computing resources to the ATLAS experiment in the Worldwide LHC Computing Grid. In order to improve the I/O performance and scalability of file servers in the future system, a possibility of introducing a cache system using fast devices such as SSD is under discussion. Therefore, a simulation has been performed to understand the cache behavior using past data access logs at the center. This paper reports a method of the simulation and gives a discussion about its results.","PeriodicalId":106243,"journal":{"name":"Proceedings of International Symposium on Grids & Clouds 2019 — PoS(ISGC2019)","volume":"51 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126286999","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Building a minimum viable Security Operations Centre for the modern grid environment","authors":"D. Crooks, L. Valsan","doi":"10.22323/1.351.0010","DOIUrl":"https://doi.org/10.22323/1.351.0010","url":null,"abstract":"The modern security landscape affecting grid and cloud sites is constantly evolving, with threats being seen from a range of avenues, including social engineering as well as more direct approaches. It is vital to build up operational security capabilities across the Worldwide LHC Computing Grid (WLCG) in order to improve the defence of the community as a whole. As reported at ISGC 2017 and 2018, the WLCG Security Operations Centres (SOC) Working Group (WG) has been working with sites across the WLCG to develop a model for a Security Operations Centre reference design. We present the current status of a minimum viable SOC design applicable to a range of different WLCG sites, centred around a few key components. \u0000 \u0000The design uses the Zeek Network Intrusion Detection System for monitoring what is happening at the network level in strategic locations: for example at border between the local cluster and external networks, the border between different local network domains or at core infrastructure nodes. The MISP Open Source Threat Intelligence Platform is used to share information regarding relevant security events and the associated Indicators of Compromise (IoCs). By feeding IoCs from MISP into Zeek we have a platform that allows the community to share threat intelligence that is immediately actionable across the entire grid. \u0000 \u0000The logs produced by Zeek are processed using the Elasticsearch, Logstash, Kibana (Elastic) stack for real time indexing and visualisation. This provides sites with a powerful tool for incident response and network forensics. The alerts raised by Zeek are further aggregated, correlated and enriched by an advanced notification processing engine. This ensures that most false positives are automatically whitelisted while at the same time reducing the total number of raised alerts that need to be managed by the computer security team of each site. By enriching these alerts and adding context of what happened around the moment the malicious activity was detected, the time needed to handle these alerts is greatly reduced. \u0000 \u0000We present possible deployment strategies for all these components in a grid context as well as the integration between them. We also report on the current status of work on integrating other sources of data, in particular using netflow / sflow, into this model. \u0000 \u0000Lastly we discuss how making use of these SOC capabilities distributed across the participating sites can lead to increasing the operational security across the entire grid.","PeriodicalId":106243,"journal":{"name":"Proceedings of International Symposium on Grids & Clouds 2019 — PoS(ISGC2019)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131714912","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Blueprint of Log Based Monitoring and Diagnosing Framework in Large Distributed Environments","authors":"Yining Zhao, Xiaodong Wang, Haili Xiao, Xue-bin Chi","doi":"10.22323/1.351.0033","DOIUrl":"https://doi.org/10.22323/1.351.0033","url":null,"abstract":"Distributed systems have kept scaling upward since this concept appears, and they soon evolve to environments that contain heterogeneous components playing different roles, making it difficult to understand how the large environment works or if any undesired matters happened from security point of view. Logs, produced by devices, sub-systems and running processes, are a very important source to help system maintainers to get relative security knowledge. But there are too many logs and too many kinds of logs to deal with, which makes manual checking impossible. In this work we will share some of our experiences in log processing and analyzing. We have summarized some common major steps that appear in most of the existing log analysis approaches, including log selection, log classification, information analyses and result feedback. We also represent a general framework that monitors events, analyzes hidden information and diagnoses the healthy state for large distributed computing environments bases on logs. Although the framework we initially designed was for the maintenance for CNGrid, its process is adaptable to other distributed computing environments.","PeriodicalId":106243,"journal":{"name":"Proceedings of International Symposium on Grids & Clouds 2019 — PoS(ISGC2019)","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127133870","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}