On the effectiveness of IP reputation for spam filtering

H. Esquivel, Aditya Akella, Tatsuya Mori
{"title":"On the effectiveness of IP reputation for spam filtering","authors":"H. Esquivel, Aditya Akella, Tatsuya Mori","doi":"10.1109/COMSNETS.2010.5431981","DOIUrl":null,"url":null,"abstract":"Modern SMTP servers apply a variety of mechanisms to stem the volume of spam delivered to users. These techniques can be broadly classified into two categories: pre-acceptance approaches, which apply prior to a message being accepted (e.g. IP reputation), and post-acceptance techniques which apply after a message has been accepted (e.g. content based signatures). We argue that the effectiveness of these measures varies based on the SMTP sender type. This paper focuses on the most light-weight pre-acceptance filtering mechanism — IP reputation. We first classify SMTP senders into three main categories: legitimate servers, end-hosts, and spam gangs, and empirically study the limits of effectiveness regarding IP reputation filtering for each category. Next, we develop new techniques that build custom IP reputation lists, which significantly improve the performance of existing IP reputation lists. In compiling these lists, we leverage a somewhat surprising fact that both legitimate domains and spam domains often use the DNS Sender Policy Framework (SPF) in an attempt to pass simple authentication checks. That is, good/bad IP addresses can be systematically compiled by collecting good/bad domains and looking up their SPF resource records. We also evaluate the effectiveness of these lists over time. Finally, we aim to understand the characteristics of the three categories of email senders in depth. Overall, we find that it is possible to construct IP reputation lists that can identify roughly 90% of all spam and legitimate mail, but some of the lists, i.e. the lists for spam gangs, must be updated on a constant basis to maintain this high level of accuracy.","PeriodicalId":369006,"journal":{"name":"2010 Second International Conference on COMmunication Systems and NETworks (COMSNETS 2010)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2010-01-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"32","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2010 Second International Conference on COMmunication Systems and NETworks (COMSNETS 2010)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/COMSNETS.2010.5431981","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 32

Abstract

Modern SMTP servers apply a variety of mechanisms to stem the volume of spam delivered to users. These techniques can be broadly classified into two categories: pre-acceptance approaches, which apply prior to a message being accepted (e.g. IP reputation), and post-acceptance techniques which apply after a message has been accepted (e.g. content based signatures). We argue that the effectiveness of these measures varies based on the SMTP sender type. This paper focuses on the most light-weight pre-acceptance filtering mechanism — IP reputation. We first classify SMTP senders into three main categories: legitimate servers, end-hosts, and spam gangs, and empirically study the limits of effectiveness regarding IP reputation filtering for each category. Next, we develop new techniques that build custom IP reputation lists, which significantly improve the performance of existing IP reputation lists. In compiling these lists, we leverage a somewhat surprising fact that both legitimate domains and spam domains often use the DNS Sender Policy Framework (SPF) in an attempt to pass simple authentication checks. That is, good/bad IP addresses can be systematically compiled by collecting good/bad domains and looking up their SPF resource records. We also evaluate the effectiveness of these lists over time. Finally, we aim to understand the characteristics of the three categories of email senders in depth. Overall, we find that it is possible to construct IP reputation lists that can identify roughly 90% of all spam and legitimate mail, but some of the lists, i.e. the lists for spam gangs, must be updated on a constant basis to maintain this high level of accuracy.
论IP信誉对垃圾邮件过滤的有效性
现代SMTP服务器应用各种机制来阻止发送给用户的垃圾邮件数量。这些技术可以大致分为两类:预接受方法,在消息被接受之前应用(例如IP信誉),以及后接受技术,在消息被接受之后应用(例如基于内容的签名)。我们认为,这些措施的有效性取决于SMTP发件人类型。本文主要研究最轻量级的预接受过滤机制——IP信誉。我们首先将SMTP发件人分为三大类:合法服务器、终端主机和垃圾邮件组,并对每一类IP信誉过滤的有效性限制进行实证研究。接下来,我们开发了构建自定义IP信誉列表的新技术,这大大提高了现有IP信誉列表的性能。在编制这些列表时,我们利用了一个有点令人惊讶的事实,即合法域和垃圾域都经常使用DNS发送方策略框架(SPF)来尝试通过简单的身份验证检查。即通过收集好/坏域并查找其SPF资源记录,可以系统地编译出好/坏IP地址。随着时间的推移,我们也会评估这些列表的有效性。最后,我们旨在深入了解三类电子邮件发送者的特征。总的来说,我们发现构建IP信誉列表可以识别大约90%的垃圾邮件和合法邮件,但是有些列表,例如垃圾邮件团伙的列表,必须不断更新以保持这种高水平的准确性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信