验证用于量化当地疫情的差异调查工具（DiD IT）

IF 3.9 3区医学 Q1 PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH

Public Health Pub Date : 2025-05-08 DOI:10.1016/j.puhe.2025.105735

Roger Morbey , Andre Charlett , Dan Todkill , Alex J. Elliot

{"title":"验证用于量化当地疫情的差异调查工具（DiD IT）","authors":"Roger Morbey , Andre Charlett , Dan Todkill , Alex J. Elliot","doi":"10.1016/j.puhe.2025.105735","DOIUrl":null,"url":null,"abstract":"<div><h3>Objectives</h3><div>The Difference-in-Differences Investigation Tool (‘DiD IT’) is a new tool used to estimate the impact of local threats to public health in England. ‘DiD IT’ is part of a daily all hazards syndromic surveillance service. We present a validation of the ‘DiD IT’ tool, using synthetic injects to assess how well it can estimate small, localised increases in the number of people presenting to health care. Furthermore, we assess how control settings within ‘DiD IT’ affect it's performance.</div></div><div><h3>Study design</h3><div>Validation Study</div></div><div><h3>Methods</h3><div>‘DiD IT’ was validated across ten different syndromic indicators, chosen to cover a range of data volumes and potential public health threats. Injects were added across different times of year and days of week, including public holidays. Also, different size of injects were created, including some with an impact spread to neighbouring locations or spread over several days. The control settings within ‘DiD IT’ were tested by varying the control location and periods, using, for example a ‘washout period’ or excluding nearest neighbours. Performance was measured by comparing the estimates for excess counts produced by ‘DiD IT’ with the actual synthetic injects added.</div></div><div><h3>Results</h3><div>‘DiD IT’ was able to provide a positive estimate in 99.8 % of trials, with a mean absolute error of 1.50. However, confidence intervals for the central estimate could not be produced in 42.5 % of trials. Furthermore, the 95 % confidence intervals for the central estimates only included the actual inject count within 62.8 % of the intervals. Unsurprising, mean errors were slightly higher when synthetic injects were not concentrated in one location on one day but were spread across neighbouring areas or days. Selecting longer control periods and using more locations as controls tended to lower the errors slightly. Including a washout period or excluding neighbouring locations from the controls did not improve performance.</div></div><div><h3>Conclusions</h3><div>We have shown that ‘DiD IT’ is accurate for assessing the impact of local incidents but that further work is needed to improve the how the uncertainty of these estimates are communicated to users.</div></div>","PeriodicalId":49651,"journal":{"name":"Public Health","volume":"243 ","pages":"Article 105735"},"PeriodicalIF":3.9000,"publicationDate":"2025-05-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Validation of a Difference-in-Differences Investigation Tool (DiD IT) for quantifying local outbreaks\",\"authors\":\"Roger Morbey , Andre Charlett , Dan Todkill , Alex J. Elliot\",\"doi\":\"10.1016/j.puhe.2025.105735\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><h3>Objectives</h3><div>The Difference-in-Differences Investigation Tool (‘DiD IT’) is a new tool used to estimate the impact of local threats to public health in England. ‘DiD IT’ is part of a daily all hazards syndromic surveillance service. We present a validation of the ‘DiD IT’ tool, using synthetic injects to assess how well it can estimate small, localised increases in the number of people presenting to health care. Furthermore, we assess how control settings within ‘DiD IT’ affect it's performance.</div></div><div><h3>Study design</h3><div>Validation Study</div></div><div><h3>Methods</h3><div>‘DiD IT’ was validated across ten different syndromic indicators, chosen to cover a range of data volumes and potential public health threats. Injects were added across different times of year and days of week, including public holidays. Also, different size of injects were created, including some with an impact spread to neighbouring locations or spread over several days. The control settings within ‘DiD IT’ were tested by varying the control location and periods, using, for example a ‘washout period’ or excluding nearest neighbours. Performance was measured by comparing the estimates for excess counts produced by ‘DiD IT’ with the actual synthetic injects added.</div></div><div><h3>Results</h3><div>‘DiD IT’ was able to provide a positive estimate in 99.8 % of trials, with a mean absolute error of 1.50. However, confidence intervals for the central estimate could not be produced in 42.5 % of trials. Furthermore, the 95 % confidence intervals for the central estimates only included the actual inject count within 62.8 % of the intervals. Unsurprising, mean errors were slightly higher when synthetic injects were not concentrated in one location on one day but were spread across neighbouring areas or days. Selecting longer control periods and using more locations as controls tended to lower the errors slightly. Including a washout period or excluding neighbouring locations from the controls did not improve performance.</div></div><div><h3>Conclusions</h3><div>We have shown that ‘DiD IT’ is accurate for assessing the impact of local incidents but that further work is needed to improve the how the uncertainty of these estimates are communicated to users.</div></div>\",\"PeriodicalId\":49651,\"journal\":{\"name\":\"Public Health\",\"volume\":\"243 \",\"pages\":\"Article 105735\"},\"PeriodicalIF\":3.9000,\"publicationDate\":\"2025-05-08\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Public Health\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0033350625001817\",\"RegionNum\":3,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Public Health","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0033350625001817","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PUBLIC, ENVIRONMENTAL & OCCUPATIONAL HEALTH","Score":null,"Total":0}

引用次数: 0

摘要

目的差异中差异调查工具（“DiD IT”）是一种用于估计英格兰当地威胁对公共卫生影响的新工具。“DiD IT”是每日所有危害综合征监测服务的一部分。我们提出了对“DiD IT”工具的验证，使用合成注射剂来评估它在估计就医人数的局部小幅增加方面的效果。此外，我们评估了“DiD IT”中的控制设置如何影响其性能。研究设计验证研究方法通过10种不同的综合征指标对“did IT”进行验证，选择涵盖一系列数据量和潜在的公共卫生威胁。注射是在一年中的不同时间和一周中的不同日子进行的，包括公共假期。此外，还产生了不同规模的注入，其中一些影响会蔓延到邻近地区，或者持续数天。“DiD IT”中的控制设置通过改变控制位置和周期进行测试，例如使用“冲洗期”或排除最近的邻居。通过比较“DiD IT”产生的过量计数估计值与实际添加的合成注入量来衡量性能。结果“did IT”能够在99.8%的试验中提供正估计，平均绝对误差为1.50。然而，在42.5%的试验中无法产生中心估计的置信区间。此外，中心估计的95%置信区间仅包括62.8%区间内的实际注入数量。不出所料，当合成注射不是在一天内集中在一个地点，而是分散在邻近地区或几天时，平均误差略高。选择较长的控制周期和使用更多的位置作为控制往往会稍微降低误差。包括冲洗期或从控制中排除邻近地点并没有提高性能。结论我们已经表明，“DiD IT”对于评估当地事件的影响是准确的，但需要进一步的工作来改进如何将这些估计的不确定性传达给用户。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Validation of a Difference-in-Differences Investigation Tool (DiD IT) for quantifying local outbreaks

Objectives

The Difference-in-Differences Investigation Tool (‘DiD IT’) is a new tool used to estimate the impact of local threats to public health in England. ‘DiD IT’ is part of a daily all hazards syndromic surveillance service. We present a validation of the ‘DiD IT’ tool, using synthetic injects to assess how well it can estimate small, localised increases in the number of people presenting to health care. Furthermore, we assess how control settings within ‘DiD IT’ affect it's performance.

Study design

Validation Study

Methods

‘DiD IT’ was validated across ten different syndromic indicators, chosen to cover a range of data volumes and potential public health threats. Injects were added across different times of year and days of week, including public holidays. Also, different size of injects were created, including some with an impact spread to neighbouring locations or spread over several days. The control settings within ‘DiD IT’ were tested by varying the control location and periods, using, for example a ‘washout period’ or excluding nearest neighbours. Performance was measured by comparing the estimates for excess counts produced by ‘DiD IT’ with the actual synthetic injects added.

Results

‘DiD IT’ was able to provide a positive estimate in 99.8 % of trials, with a mean absolute error of 1.50. However, confidence intervals for the central estimate could not be produced in 42.5 % of trials. Furthermore, the 95 % confidence intervals for the central estimates only included the actual inject count within 62.8 % of the intervals. Unsurprising, mean errors were slightly higher when synthetic injects were not concentrated in one location on one day but were spread across neighbouring areas or days. Selecting longer control periods and using more locations as controls tended to lower the errors slightly. Including a washout period or excluding neighbouring locations from the controls did not improve performance.

Conclusions

We have shown that ‘DiD IT’ is accurate for assessing the impact of local incidents but that further work is needed to improve the how the uncertainty of these estimates are communicated to users.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Public Health 医学-公共卫生、环境卫生与职业卫生

CiteScore

7.60

自引率

0.00%

发文量

280

审稿时长

37 days

期刊介绍： Public Health is an international, multidisciplinary peer-reviewed journal. It publishes original papers, reviews and short reports on all aspects of the science, philosophy, and practice of public health.