Quantitative estimation of the strength of agreements in goal-oriented meetings

Abstract

Ineffective meetings occur frequently and participants leave with different understandings of what has been decided upon. For meetings that require quick responses (e.g., disaster-response planning), everyone must leave the meeting on the same page to ensure the successful execution of the mission. Detecting patterns of weak agreements in planning meetings is the first step towards designing an intelligent agent that encourages team members to revisit decisions that may adversely affect the team's performance, and to spur dialog that results in higher quality plans. This paper presents a statistical approach to learning patterns of strong and weak agreements without using domain-specific content or keywords, meaning the algorithm takes as input information about how the team plans but does not require potentially sensitive data on what is being planned. Our approach applies statistical machine learning to dialog features, which prior studies in cognitive psychology have shown qualitatively capture the level of joint commitment to plan choices. We analyze a real-world conversation dataset, the AMI corpus, to quantitatively verify that dialog features improve the estimation of strength of agreements over prior approaches. We show these results are consistent across a number of different supervised and unsupervised learning algorithms, and that can achieve up to 94% average accuracy in estimating the strength of agreements.