LENA: Llama-Based Embeddings of Neutralized Assembly for Cross-Compiler/Optimization Binary Code Similarity Detection

Abstract

Binary code similarity detection (BCSD) is fundamental to software security, yet remains challenging under cross-compiler and cross-optimization settings. Existing methods often rely on contrastive objectives that are sensitive to false negatives, while recent alternatives may learn redundant embeddings that limit fine-grained discrimination. In this paper, we present LENA, a BCSD framework that adapts a Llama backbone through an optimization-translation task and trains a lightweight projector using a hybrid objective that combines contrastive learning with variance and covariance regularization. Experiments on x86-64 benchmarks show that LENA outperforms state-of-the-art baselines, improving average Recall@1 by 10.45% on GCC, 5.36% on Clang, and 8.55% in cross-compiler settings, while also improving average R-Precision in vulnerability detection. We further provide a supporting tie-aware evaluation analysis and introduce Tie-sensitive Reciprocal Rank (TsRR), which provides deterministic reciprocal-rank-based reporting when tied similarity scores occur and reduces ambiguity in BCSD evaluation.