用于低地轨道超大型星座网络机载分布式处理的大数据应用模拟平台设计

Zhikai Zhang, Shushi Gu, Zhang Qinyu, Jiayin Xue
{"title":"用于低地轨道超大型星座网络机载分布式处理的大数据应用模拟平台设计","authors":"Zhikai Zhang, Shushi Gu, Zhang Qinyu, Jiayin Xue","doi":"10.23919/JCC.ja.2022-0617","DOIUrl":null,"url":null,"abstract":"Due to the restricted satellite payloads in LEO mega-constellation networks (LMCNs), remote sensing image analysis, online learning and other big data services desirably need onboard distributed processing (OBDP). In existing technologies, the efficiency of big data applications (BDAs) in distributed systems hinges on the stable-state and low-latency links between worker nodes. However, LMCNs with high-dynamic nodes and long-distance links can not provide the above conditions, which makes the performance of OBDP hard to be intuitively measured. To bridge this gap, a multidimensional simulation platform is indispensable that can simulate the network environment of LMCNs and put BDAs in it for performance testing. Using STK's APIs and parallel computing framework, we achieve real-time simulation for thousands of satellite nodes, which are mapped as application nodes through software defined network (SDN) and container technologies. We elaborate the architecture and mechanism of the simulation platform, and take the Starlink and Hadoop as realistic examples for simulations. The results indicate that LMCNs have dynamic end-to-end latency which fluctuates periodically with the constellation movement. Compared to ground data center networks (GDCNs), LMCNs deteriorate the computing and storage job throughput, which can be alleviated by the utilization of erasure codes and data flow scheduling of worker nodes.","PeriodicalId":504777,"journal":{"name":"China Communications","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Big data application simulation platform design for onboard distributed processing of LEO mega-constellation networks\",\"authors\":\"Zhikai Zhang, Shushi Gu, Zhang Qinyu, Jiayin Xue\",\"doi\":\"10.23919/JCC.ja.2022-0617\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Due to the restricted satellite payloads in LEO mega-constellation networks (LMCNs), remote sensing image analysis, online learning and other big data services desirably need onboard distributed processing (OBDP). In existing technologies, the efficiency of big data applications (BDAs) in distributed systems hinges on the stable-state and low-latency links between worker nodes. However, LMCNs with high-dynamic nodes and long-distance links can not provide the above conditions, which makes the performance of OBDP hard to be intuitively measured. To bridge this gap, a multidimensional simulation platform is indispensable that can simulate the network environment of LMCNs and put BDAs in it for performance testing. Using STK's APIs and parallel computing framework, we achieve real-time simulation for thousands of satellite nodes, which are mapped as application nodes through software defined network (SDN) and container technologies. We elaborate the architecture and mechanism of the simulation platform, and take the Starlink and Hadoop as realistic examples for simulations. The results indicate that LMCNs have dynamic end-to-end latency which fluctuates periodically with the constellation movement. Compared to ground data center networks (GDCNs), LMCNs deteriorate the computing and storage job throughput, which can be alleviated by the utilization of erasure codes and data flow scheduling of worker nodes.\",\"PeriodicalId\":504777,\"journal\":{\"name\":\"China Communications\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-07-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"China Communications\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.23919/JCC.ja.2022-0617\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"China Communications","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.23919/JCC.ja.2022-0617","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

由于低地轨道超大型星座网络(LMCN)的卫星有效载荷有限,遥感图像分析、在线学习和其他大数据服务都需要理想的星载分布式处理(OBDP)。在现有技术中,分布式系统中大数据应用(BDA)的效率取决于工作节点之间的稳定状态和低延迟链路。然而,具有高动态节点和长距离链路的 LMCN 无法提供上述条件,这使得 OBDP 的性能难以直观衡量。要弥补这一缺陷,一个能模拟 LMCN 网络环境并将 BDA 放入其中进行性能测试的多维仿真平台必不可少。利用 STK 的应用程序接口和并行计算框架,我们实现了数千个卫星节点的实时仿真,这些节点通过软件定义网络(SDN)和容器技术映射为应用节点。我们详细阐述了仿真平台的架构和机制,并以 Starlink 和 Hadoop 为仿真实例。结果表明,LMCN 具有动态的端到端延迟,会随着星座移动而周期性波动。与地面数据中心网络(GDCNs)相比,LMCNs会降低计算和存储任务的吞吐量,这可以通过使用擦除码和工作节点的数据流调度来缓解。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Big data application simulation platform design for onboard distributed processing of LEO mega-constellation networks
Due to the restricted satellite payloads in LEO mega-constellation networks (LMCNs), remote sensing image analysis, online learning and other big data services desirably need onboard distributed processing (OBDP). In existing technologies, the efficiency of big data applications (BDAs) in distributed systems hinges on the stable-state and low-latency links between worker nodes. However, LMCNs with high-dynamic nodes and long-distance links can not provide the above conditions, which makes the performance of OBDP hard to be intuitively measured. To bridge this gap, a multidimensional simulation platform is indispensable that can simulate the network environment of LMCNs and put BDAs in it for performance testing. Using STK's APIs and parallel computing framework, we achieve real-time simulation for thousands of satellite nodes, which are mapped as application nodes through software defined network (SDN) and container technologies. We elaborate the architecture and mechanism of the simulation platform, and take the Starlink and Hadoop as realistic examples for simulations. The results indicate that LMCNs have dynamic end-to-end latency which fluctuates periodically with the constellation movement. Compared to ground data center networks (GDCNs), LMCNs deteriorate the computing and storage job throughput, which can be alleviated by the utilization of erasure codes and data flow scheduling of worker nodes.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信