Exploring pose estimation as a tool for the assessment of brush use patterns in dairy cows

Abstract

Access to mechanical brushes enables grooming behaviour in dairy cows and has shown benefits for cow welfare, including improved cleanliness, comfort, stress reduction. Brush-use may also promote a positive emotional state. Reduced brush use has been associated with health issues, suggesting its potential for automated health monitoring. This study aimed at evaluating whether data generated by pose estimation could be used to assess brush use patterns in loose-housed dairy cows. It presents an approach for automatically identifying the body segment being brushed as an application of pose estimation. Data collection was carried out at the Swedish Livestock Research Centre in a loose housing system equipped with an automatic milking system and two mechanical rotating brushes. Recordings spanned 25:30 h and used three cameras, at different positions, monitoring a single mechanical brush placed in a passageway between cubicle rows. One human observer with access to recordings from all three synchronized cameras annotated the data-set on a second-by-second basis. The observer recorded: (1) the number of cows using the brush; (2) the anatomical segment being brushed; and (3) whether brushing resumed after a pause. The same video recordings were processed with object detection and pose estimation, which predicted the location of bounding boxes for cows and for the brush as well as corresponding keypoints. Using the brush and cow keypoint locations, we attempted to detect brushing by anatomical region. In a first stage, machine-learning models were trained to predict brushing state (independent of location) using keypoint distance to the brush, achieving an accuracy of 86.3 %. To mitigate the risk of error propagation, we relied on human annotations to segment the video to confirmed brushing bouts for analysis in the second stage. To identify the anatomical location of brushing, two methods were evaluated: (1) simply assigning the brushing location to the closest keypoint, achieving 73 % average accuracy across classes, and (2) projecting brush and anatomical keypoints onto a spline modelling the cow’s backline, resulting in 87 % accuracy. Misclassifications were predominantly limited to adjacent body segments. Given that intra-observer reliability was 90 %, the spline-based method was deemed sufficiently reliable for research applications to accurately monitor the specific body segments being brushed.