$\boldsymbol{Steiner}$-Hardness: A Query Hardness Measure for Graph-Based ANN Indexes

Zeyu Wang, Qitong Wang, Xiaoxing Cheng, Peng Wang, Themis Palpanas, Wei Wang
{"title":"$\\boldsymbol{Steiner}$-Hardness: A Query Hardness Measure for Graph-Based ANN Indexes","authors":"Zeyu Wang, Qitong Wang, Xiaoxing Cheng, Peng Wang, Themis Palpanas, Wei Wang","doi":"arxiv-2408.13899","DOIUrl":null,"url":null,"abstract":"Graph-based indexes have been widely employed to accelerate approximate\nsimilarity search of high-dimensional vectors. However, the performance of\ngraph indexes to answer different queries varies vastly, leading to an unstable\nquality of service for downstream applications. This necessitates an effective\nmeasure to test query hardness on graph indexes. Nonetheless, popular\ndistance-based hardness measures like LID lose their effects due to the\nignorance of the graph structure. In this paper, we propose $Steiner$-hardness,\na novel connection-based graph-native query hardness measure. Specifically, we\nfirst propose a theoretical framework to analyze the minimum query effort on\ngraph indexes and then define $Steiner$-hardness as the minimum effort on a\nrepresentative graph. Moreover, we prove that our $Steiner$-hardness is highly\nrelevant to the classical Directed $Steiner$ Tree (DST) problems. In this case,\nwe design a novel algorithm to reduce our problem to DST problems and then\nleverage their solvers to help calculate $Steiner$-hardness efficiently.\nCompared with LID and other similar measures, $Steiner$-hardness shows a\nsignificantly better correlation with the actual query effort on various\ndatasets. Additionally, an unbiased evaluation designed based on\n$Steiner$-hardness reveals new ranking results, indicating a meaningful\ndirection for enhancing the robustness of graph indexes. This paper is accepted\nby PVLDB 2025.","PeriodicalId":501123,"journal":{"name":"arXiv - CS - Databases","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-08-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - CS - Databases","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2408.13899","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

Abstract

Graph-based indexes have been widely employed to accelerate approximate similarity search of high-dimensional vectors. However, the performance of graph indexes to answer different queries varies vastly, leading to an unstable quality of service for downstream applications. This necessitates an effective measure to test query hardness on graph indexes. Nonetheless, popular distance-based hardness measures like LID lose their effects due to the ignorance of the graph structure. In this paper, we propose $Steiner$-hardness, a novel connection-based graph-native query hardness measure. Specifically, we first propose a theoretical framework to analyze the minimum query effort on graph indexes and then define $Steiner$-hardness as the minimum effort on a representative graph. Moreover, we prove that our $Steiner$-hardness is highly relevant to the classical Directed $Steiner$ Tree (DST) problems. In this case, we design a novel algorithm to reduce our problem to DST problems and then leverage their solvers to help calculate $Steiner$-hardness efficiently. Compared with LID and other similar measures, $Steiner$-hardness shows a significantly better correlation with the actual query effort on various datasets. Additionally, an unbiased evaluation designed based on $Steiner$-hardness reveals new ranking results, indicating a meaningful direction for enhancing the robustness of graph indexes. This paper is accepted by PVLDB 2025.
$\boldsymbol{Steiner}$-Hardness:基于图的 ANN 索引的查询硬度度量
基于图的索引已被广泛用于加速高维向量的近似相似性搜索。然而,图索引在回答不同查询时的性能差异很大,导致下游应用的服务质量不稳定。这就需要一种有效的方法来测试图索引的查询硬度。然而,由于图结构的不确定性,基于大众距离的硬度测量(如 LID)失去了效果。在本文中,我们提出了$Steiner$-hardness,一种基于连接的新型图本地查询硬度度量。具体来说,我们首先提出了一个理论框架来分析图索引上的最小查询工作量,然后将$Steiner$-硬度定义为呈现图上的最小工作量。此外,我们还证明了$Steiner$-hardness与经典的有向$Steiner$树(DST)问题高度相关。在这种情况下,我们设计了一种新颖的算法,将我们的问题简化为DST问题,然后利用它们的求解器来帮助高效计算$Steiner$-hardness。与LID和其他类似度量相比,$Steiner$-hardness与各种数据集上的实际查询工作量的相关性显著提高。此外,基于$Steiner$-hardness设计的无偏评估揭示了新的排名结果,为增强图索引的鲁棒性指明了有意义的方向。本文已被 PVLDB 2025 接收。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信