Towards Learning Hierarchical Structures with SyncMap

Abstract

Objects or events perceived by human are often organized in a sequence that forms into chunks which exhibit hierarchical structure, e.g., words or videos. Such a sequence can be represented as a group of temporally correlated variables at multiple levels referred as chunk. In this work, an unsupervised method known as SyncMap is used to perform chunking on sequences of input data with hierarchical structure. We design a fixed and probabilistic chunk experiment to test our model capability, measured by the mutual information between the predicted chunk with the ground truth. Surprisingly, without too much modification on the original algorithm, the result has shown that SyncMap can perform chunking with hierarchical structure, although with limitation. Possible future works are proposed to overcome the limitation. Observation on the dynamic of weight map also indicates that SyncMap adapts to the low-level hierarchical representation of chunks faster than the one on the higher level.