Learning and reasoning in logic tensor networks: theory and application to semantic image interpretation

This paper presents a revision of Real Logic and its implementation with Logic Tensor Networks and its application to Semantic Image Interpretation. Real Logic is a framework where learning from numerical data and logical reasoning are integrated using first order logic syntax. The symbols of the signature of Real Logic are interpreted in the data-space, i.e, on the domain of real numbers. The integration of learning and reasoning obtained in Real Logic allows us to formalize learning as approximate satisfiability in the presence of logical constraints, and to perform inference on symbolic and numerical data. After introducing a refined version of the formalism, we describe its implementation into Logic Tensor Networks which uses deep learning within Google's TensorFlow™. We evaluate LTN on the task of classifying objects and their parts in images, where we combine state-of-the-art-object detectors with a part-of ontology. LTN outperforms the state-of-the-art on object classification, and improves the performances on part-of relation detection with respect to a rule-based baseline.