基于分布特征分析的Fastq文件压缩算法

Shengyu Lu, Hanping Chen, Lifa Peng, Beizhan Wang, Hongji Wang, Xiuze Zhou
{"title":"基于分布特征分析的Fastq文件压缩算法","authors":"Shengyu Lu, Hanping Chen, Lifa Peng, Beizhan Wang, Hongji Wang, Xiuze Zhou","doi":"10.1109/ICCSE.2018.8468742","DOIUrl":null,"url":null,"abstract":"With the continuous development of sequencing technology scientists in the cost of DNA sequencing in reduce gradually, it also makes the number of DNA sequencing data to increase substantially. While the genome data is need to store, the traditional computer room has not enough to store such large data. Therefore, more and more genome data need to be uploaded to the cloud. Due to the speed of growth of communication have been much faster than the growth of the genomic data, so it is particularly important for genome data compression to reduce the cost of scientific research institutions and it is of great significance to speed up the sharing of genomic data. Fastq file is an important format of genomic data, and now the compression algorithm for fastq files is mainly include of DSRC, FQC, etc. These algorithms are also compressed based on the characteristics of fastq files. In order to improve the rate of compression, we propose an algorithm of DDSRC and establish the statistical models for the distribution characteristics of strings in fastq files to perform more efficient compression algorithms. This paper will explain the algorithm based on the distribution characteristics analysis and compare the results with other compression algorithms.","PeriodicalId":228760,"journal":{"name":"2018 13th International Conference on Computer Science & Education (ICCSE)","volume":"25 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2018-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":"{\"title\":\"A Compression Algorithm of Fastq File Based on Distribution Characteristics Analysis\",\"authors\":\"Shengyu Lu, Hanping Chen, Lifa Peng, Beizhan Wang, Hongji Wang, Xiuze Zhou\",\"doi\":\"10.1109/ICCSE.2018.8468742\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"With the continuous development of sequencing technology scientists in the cost of DNA sequencing in reduce gradually, it also makes the number of DNA sequencing data to increase substantially. While the genome data is need to store, the traditional computer room has not enough to store such large data. Therefore, more and more genome data need to be uploaded to the cloud. Due to the speed of growth of communication have been much faster than the growth of the genomic data, so it is particularly important for genome data compression to reduce the cost of scientific research institutions and it is of great significance to speed up the sharing of genomic data. Fastq file is an important format of genomic data, and now the compression algorithm for fastq files is mainly include of DSRC, FQC, etc. These algorithms are also compressed based on the characteristics of fastq files. In order to improve the rate of compression, we propose an algorithm of DDSRC and establish the statistical models for the distribution characteristics of strings in fastq files to perform more efficient compression algorithms. This paper will explain the algorithm based on the distribution characteristics analysis and compare the results with other compression algorithms.\",\"PeriodicalId\":228760,\"journal\":{\"name\":\"2018 13th International Conference on Computer Science & Education (ICCSE)\",\"volume\":\"25 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2018-08-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"1\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2018 13th International Conference on Computer Science & Education (ICCSE)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/ICCSE.2018.8468742\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2018 13th International Conference on Computer Science & Education (ICCSE)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ICCSE.2018.8468742","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 1

摘要

随着测序技术的不断发展,科学家们在DNA测序的成本在逐渐降低的同时,也使得DNA测序数据的数量大幅增加。而基因组数据是需要存储的,传统的机房不足以存储如此大的数据。因此,越来越多的基因组数据需要上传到云端。由于通信的增长速度已经远远快于基因组数据的增长速度,因此基因组数据压缩对于降低科研机构的成本尤为重要,加快基因组数据的共享具有重要意义。Fastq文件是基因组数据的重要格式,目前Fastq文件的压缩算法主要包括DSRC、FQC等。这些算法还根据fastq文件的特点进行了压缩。为了提高压缩速率,我们提出了一种DDSRC算法,并建立了fastq文件中字符串分布特征的统计模型,以实现更高效的压缩算法。本文将在分析分布特性的基础上对算法进行说明,并与其他压缩算法的结果进行比较。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
A Compression Algorithm of Fastq File Based on Distribution Characteristics Analysis
With the continuous development of sequencing technology scientists in the cost of DNA sequencing in reduce gradually, it also makes the number of DNA sequencing data to increase substantially. While the genome data is need to store, the traditional computer room has not enough to store such large data. Therefore, more and more genome data need to be uploaded to the cloud. Due to the speed of growth of communication have been much faster than the growth of the genomic data, so it is particularly important for genome data compression to reduce the cost of scientific research institutions and it is of great significance to speed up the sharing of genomic data. Fastq file is an important format of genomic data, and now the compression algorithm for fastq files is mainly include of DSRC, FQC, etc. These algorithms are also compressed based on the characteristics of fastq files. In order to improve the rate of compression, we propose an algorithm of DDSRC and establish the statistical models for the distribution characteristics of strings in fastq files to perform more efficient compression algorithms. This paper will explain the algorithm based on the distribution characteristics analysis and compare the results with other compression algorithms.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信