Training-Free Affordance Labeling and Exploration Using Subspace Projection and Manifold Curvature Over Pre-Trained Deep Networks

Abstract

The advancement in computing power has significantly reduced the training times for deep learning, enabling the rapid development of networks designed for object recognition. However, the exploration of object utility, the object’s affordance, as opposed to object recognition, has received comparatively less attention. Existing object affordance models exhibit shortcomings, including limited robustness across diverse architectures and insufficient performance in complex environments. This work focuses on using pre-trained networks trained on object classification datasets to explore object affordances. While these networks have proven instrumental in transfer learning for classification tasks, the presented approach in this study diverges from conventional object classification methods by labeling affordances without modifying the final layers. Instead, pre-trained networks are employed to learn affordance labels without requiring specialized classification layers. Two approaches are tested: the Subspace Projection Method and the Manifold Curvature Method, which facilitate the determination of affordance labels without such modifications. Both the Subspace Projection Method and the Manifold Curvature Method were evaluated using nine distinct pre-trained networks across two different affordance datasets. The Subspace Projection Method achieved a True Positive Rate of up to 94% and 96% for the best-performing networks on each dataset, while the Manifold Curvature Method attained True Positive Rates exceeding 98% and 99% with its top-performing networks. Furthermore, both methods identify affordance labels that are not marked in the ground truth but are present in various cases. The robustness of the Manifold Curvature Method and the exploration capability of both methods highlight the effectiveness of proposed techniques for affordance labeling.