Towards a Multi-Granulated Statistical Framework for Human–Machine Collaboration in Image Classification

In the past decade, despite significant advancements in Artificial Intelligence (AI) and deep learning technologies, they still fall short of fully replicating the complex functions of the human brain. This highlights the importance of researching human-machine collaborative systems. This study introduces a statistical framework capable of finely modeling integrated performance, breaking it down into the individual performance term and the diversity term, thereby enhancing interpretability and estimation accuracy. Extensive multi-granularity experiments were conducted using this framework on various image classification datasets, revealing the differences between humans and machines in classification tasks from macro to micro levels. This difference is key to improving human-machine collaborative performance, as it allows for complementary strengths. The study found that Human-Machine collaboration (HM) often outperforms individual human (H) or machine (M) performances, but not always. The superiority of performance depends on the interplay between the individual performance term and the diversity term. To further enhance the performance of human-machine collaboration, a novel Human-Adapter-Machine (HAM) model is introduced. Specifically, HAM can adaptively adjust decision weights to enhance the complementarity among individuals. Theoretical analysis and experimental results both demonstrate that HAM outperforms the traditional HM strategy and the individual agent (H or M).