Structured Prediction Networks through Latent Cost Learning

Abstract

Structured prediction provides a flexible modeling framework to deal with several relevant problems. Sequences, Trees, Disjoint Intervals and Matching are some useful examples of the type of structures we would like to predict. An elegant learning scheme for this prediction setting is the Structured Perceptron algorithm, which is sure to converge under some linear separability conditions. The framework integrates a very simple Structured layer on top of a latent costs network. Our key contribution is a novel loss function that incorporates structural information and simplifies learning. The effectiveness of this framework is illustrated with sequence prediction problems. We explore LSTM neural network architectures to model the latent costs layer, since our experiments concern NLP tasks. We perform basic experiments with Chunking in English. The SPN predictor outperforms its CRF equivalent. Our initial findings strongly indicate that SPN is a versatile framework with a powerful learning strategy.