Rafael E.P. Ferreira , Maria Angels de Luis Balaguer , Tiago Bresolin , Ranveer Chandra , Guilherme J.M. Rosa , Heather M. White , João R.R. Dórea
{"title":"Multi-modal machine learning for the early detection of metabolic disorder in dairy cows using a cloud computing framework","authors":"Rafael E.P. Ferreira , Maria Angels de Luis Balaguer , Tiago Bresolin , Ranveer Chandra , Guilherme J.M. Rosa , Heather M. White , João R.R. Dórea","doi":"10.1016/j.compag.2024.109563","DOIUrl":null,"url":null,"abstract":"<div><div>In precision livestock farming (PLF), wearable sensors, computer vision, and genomic tests generate large amounts of data, which can be challenging to integrate and analyze jointly due to their diverse nature. However, incorporating both genomic and phenotypic data together can be beneficial for developing predictive models in animal biology. The development of automated and modular data pipelines using scalable solutions such as cloud computing can be an effective strategy to integrate and analyze animal-level information in real-time. The objectives of this study were (1) to propose a cloud computing-based framework to automate the processing and integration of phenotypic and genotypic data, and (2) to assess different data fusion strategies (early and late fusion, and cooperative learning) for the early detection of subclinical ketosis (SCK) in dairy cows, integrating wearable sensors, imaging systems, and genotypic data in livestock farms. We developed a modular pipeline for image analysis, which includes body segmentation, frame quality assessment, animal identification, and body condition score (BCS), which were crucial for producing the features used in SCK detection. The body segmentation module achieved a Dice similarity coefficient of 0.990, the frame quality assessment module reached 99.1 % accuracy, the animal identification module attained 93.2 % accuracy, and the BCS module achieved accuracies of 81.1 % and 96.2 % when allowing up to 0.25 and 0.50 prediction error, respectively. For SCK detection, early fusion and cooperative learning achieved the lowest mean absolute errors in predicting plasma beta-hydroxybutyrate as a continuous variable (as low as 0.242). Late fusion, combined with an ordinary least squares regression, achieved the highest F<sub>1</sub> scores for binary SCK prediction (up to 0.750). These results suggest that data fusion techniques can be effectively used to integrate genotypic and phenotypic data from multiple sensors. Additionally, SCK detection can be performed on dairy farms using the proposed cloud computing-based framework, which was implemented with modular, independent services that can be customized and reused for a variety of tasks.</div></div>","PeriodicalId":50627,"journal":{"name":"Computers and Electronics in Agriculture","volume":"227 ","pages":"Article 109563"},"PeriodicalIF":7.7000,"publicationDate":"2024-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computers and Electronics in Agriculture","FirstCategoryId":"97","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0168169924009542","RegionNum":1,"RegionCategory":"农林科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AGRICULTURE, MULTIDISCIPLINARY","Score":null,"Total":0}
引用次数: 0
Abstract
In precision livestock farming (PLF), wearable sensors, computer vision, and genomic tests generate large amounts of data, which can be challenging to integrate and analyze jointly due to their diverse nature. However, incorporating both genomic and phenotypic data together can be beneficial for developing predictive models in animal biology. The development of automated and modular data pipelines using scalable solutions such as cloud computing can be an effective strategy to integrate and analyze animal-level information in real-time. The objectives of this study were (1) to propose a cloud computing-based framework to automate the processing and integration of phenotypic and genotypic data, and (2) to assess different data fusion strategies (early and late fusion, and cooperative learning) for the early detection of subclinical ketosis (SCK) in dairy cows, integrating wearable sensors, imaging systems, and genotypic data in livestock farms. We developed a modular pipeline for image analysis, which includes body segmentation, frame quality assessment, animal identification, and body condition score (BCS), which were crucial for producing the features used in SCK detection. The body segmentation module achieved a Dice similarity coefficient of 0.990, the frame quality assessment module reached 99.1 % accuracy, the animal identification module attained 93.2 % accuracy, and the BCS module achieved accuracies of 81.1 % and 96.2 % when allowing up to 0.25 and 0.50 prediction error, respectively. For SCK detection, early fusion and cooperative learning achieved the lowest mean absolute errors in predicting plasma beta-hydroxybutyrate as a continuous variable (as low as 0.242). Late fusion, combined with an ordinary least squares regression, achieved the highest F1 scores for binary SCK prediction (up to 0.750). These results suggest that data fusion techniques can be effectively used to integrate genotypic and phenotypic data from multiple sensors. Additionally, SCK detection can be performed on dairy farms using the proposed cloud computing-based framework, which was implemented with modular, independent services that can be customized and reused for a variety of tasks.
期刊介绍:
Computers and Electronics in Agriculture provides international coverage of advancements in computer hardware, software, electronic instrumentation, and control systems applied to agricultural challenges. Encompassing agronomy, horticulture, forestry, aquaculture, and animal farming, the journal publishes original papers, reviews, and applications notes. It explores the use of computers and electronics in plant or animal agricultural production, covering topics like agricultural soils, water, pests, controlled environments, and waste. The scope extends to on-farm post-harvest operations and relevant technologies, including artificial intelligence, sensors, machine vision, robotics, networking, and simulation modeling. Its companion journal, Smart Agricultural Technology, continues the focus on smart applications in production agriculture.