Analyzing Hierarchical Relationships and Quality of Embedding in Latent Space

Abstract

Existing learning models partition the generated representations using hyperplanes which form well defined groups of similar embeddings that is uniquely mapped to a particular class. However, in practical applications, the embedding space does not form distinct boundaries to segregate the class representations. There exists interaction among similar classes which cannot be visually determined in high-dimensional space. Moreover, the structure of the latent space remains obscure. As learned representations are frequently reused to reduce the inference time, it is important to analyse how semantically related classes interact among themselves in the latent space. Therefore, we propose a boundary estimation algorithm that minimises the inclusion of other classes in the embedding space to form groups of similar representations and compare the quality of these class embeddings for various models in an already encoded space. These groups are overlapping to denote ambiguous embeddings that cannot be mapped to a particular class with high confidence. The algorithm determines which representations to be included or discarded to form well defined regions, separating discriminating, ambiguous and rejected embeddings to depict a particular class. Later, we construct relation trees to evaluate the hierarchical relationships formed among the classes, and compare it with the WordNet ontology using phylogenetic tree comparison methods.