Efficient Multi-Dimensional Flow Correlation

Abstract

Flow correlation algorithms compare flows to determine similarity, and are especially useful and well studied for detecting flow chains through "stepping stone" hosts. Most correlation algorithms use only one characteristic and require all values in the correlation matrix (the correlation value of all flows to all other flows) to be updated on every event. We have developed an algorithm that tracks multiple (n) characteristics per flow, and requires updating only the flow's n values upon an event, not all the values for all the flows. The n correlation values are used as coordinates for a point in n-space; two flows are considered correlated if there is a very small Euclidean distance between them. Our results show that this algorithm is efficient in space and compute time, is resilient against anomalies in the flow, and has uses outside of stepping stone detection.