Enhanced Plastic Recycling using RGB+Depth Fusion with MassFaster and MassMask R-CNN

Dillam Jossue Diaz Romero, Isiah Zaplana, Simon Van den Eynde, Wouter Sterkens, T. Goedemé, Jef Peeters
{"title":"Enhanced Plastic Recycling using RGB+Depth Fusion with MassFaster and MassMask R-CNN","authors":"Dillam Jossue Diaz Romero, Isiah Zaplana, Simon Van den Eynde, Wouter Sterkens, T. Goedemé, Jef Peeters","doi":"10.1109/TransAI54797.2022.00010","DOIUrl":null,"url":null,"abstract":"The rapid increase in waste generation from electrical and electronic equipment (WEEE) has created the need for more advanced sensor-based systems to sort this complex type of waste. Therefore, this study proposes a method for object detection, instance segmentation, and mass estimation of plastics and contaminants using the fusion of RGB and depth (D) images. The methodology is based on the Faster and Mask R-CNN with an extra head for the mass estimation. In addition, a pre-processing method to enhance the depth image (ED) is proposed. To evaluate the data fusion and pre-processing method, two data sets of plastics and impurities were created containing images with and without overlapping samples. The first data set contains 174 RGB images and depth (D) maps of 3146 samples, excluding their mass value, while the second data set contains 42 RGB and D images of 766 pieces together with their mass. The first and second data sets were used to evaluate the performance of Mask and Faster R-CNN. Further, the second data set was used to evaluate the network’s performance with the additional head for mass estimation.The proposed method achieved 0.75 R2, 1.39 RMSE, and 0.81 MAE with an IoU greater than 50% using the network Resnet50_FPN_RGBED. Hence, it can be concluded that the presented method can distinguish plastics from other materials with reasonable accuracy. Furthermore, the mass of each detected particle can be estimated individually, which is of great relevance for the recycling sector. Knowing the mass distribution and the percentage of contaminants in a waste stream of mixed plastics can be valuable for adjusting the parameters of upstream and downstream sorting processes.","PeriodicalId":445732,"journal":{"name":"2022 Fourth International Conference on Transdisciplinary AI (TransAI)","volume":"387 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2022-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2022 Fourth International Conference on Transdisciplinary AI (TransAI)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/TransAI54797.2022.00010","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 1

Abstract

The rapid increase in waste generation from electrical and electronic equipment (WEEE) has created the need for more advanced sensor-based systems to sort this complex type of waste. Therefore, this study proposes a method for object detection, instance segmentation, and mass estimation of plastics and contaminants using the fusion of RGB and depth (D) images. The methodology is based on the Faster and Mask R-CNN with an extra head for the mass estimation. In addition, a pre-processing method to enhance the depth image (ED) is proposed. To evaluate the data fusion and pre-processing method, two data sets of plastics and impurities were created containing images with and without overlapping samples. The first data set contains 174 RGB images and depth (D) maps of 3146 samples, excluding their mass value, while the second data set contains 42 RGB and D images of 766 pieces together with their mass. The first and second data sets were used to evaluate the performance of Mask and Faster R-CNN. Further, the second data set was used to evaluate the network’s performance with the additional head for mass estimation.The proposed method achieved 0.75 R2, 1.39 RMSE, and 0.81 MAE with an IoU greater than 50% using the network Resnet50_FPN_RGBED. Hence, it can be concluded that the presented method can distinguish plastics from other materials with reasonable accuracy. Furthermore, the mass of each detected particle can be estimated individually, which is of great relevance for the recycling sector. Knowing the mass distribution and the percentage of contaminants in a waste stream of mixed plastics can be valuable for adjusting the parameters of upstream and downstream sorting processes.
使用RGB+深度融合与MassFaster和MassMask R-CNN增强塑料回收
电气和电子设备(WEEE)产生的废物迅速增加,需要更先进的基于传感器的系统来分类这种复杂类型的废物。因此,本研究提出了一种使用RGB和深度(D)图像融合的塑料和污染物的目标检测、实例分割和质量估计方法。该方法基于Faster和Mask R-CNN,并为质量估计增加了一个额外的头部。此外,还提出了一种增强深度图像的预处理方法。为了评估数据融合和预处理方法,创建了两个包含有和没有重叠样本的图像的塑料和杂质数据集。第一个数据集包含174个RGB图像和3146个样本的深度(D)图,不包括它们的质量值;第二个数据集包含42个RGB图像和766个样本的深度(D)图及其质量。第一组和第二组数据集用于评估Mask和Faster R-CNN的性能。此外,第二个数据集用于评估网络的性能,并使用额外的头部进行质量估计。该方法使用Resnet50_FPN_RGBED网络实现了0.75 R2, 1.39 RMSE和0.81 MAE, IoU大于50%。因此,可以得出结论,该方法可以以合理的精度区分塑料和其他材料。此外,每个检测到的颗粒的质量可以单独估计,这对回收部门有很大的相关性。了解混合塑料废物流中污染物的质量分布和百分比对于调整上游和下游分拣过程的参数是有价值的。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信
小红书