Pilot Study of an AI-Assisted Open-Source Intelligence Pipeline for Detecting Chemical and Radiological Threat Signals

Abstract

Background: Timely detection of chemical and radiological threats is critical for global public health. These events arise suddenly, cross borders, and can overwhelm response capacity, yet no global event-based surveillance system exists to detect chemical and radiological threats. Many are first detected or only detected through news and social media, collectively known as open-source intelligence (OSINT). OSINT provides a near-real-time signal source that is scalable for the systematic extraction and structuring of limited data. This study evaluates the feasibility of an artificial intelligence (AI)-assisted OSINT pipeline for detecting and characterizing chemical and radiological signals. Methods: A lexicon of 45-lay accessible search terms queried the Bing News Application Programming Interface from Aug 2024 to Jan 2025. Retrieved articles were scraped and summarized using GPT-4o, with non-English articles translated using Google Translate. A human-in-the-loop workflow was used for the verification and structured data extraction. Epidemiological variables were manually extracted into Microsoft Excel v.2410, analyzed in STATA/BE 18.0, and mapped using ArcGIS Pro v.3.1. The extraction pipeline applied here was validated against a human-annotated gold standard in a companion study. Results: A total of 802 unique incidents were identified (797 chemical-related and 5 radiological). The United States (60.0%), India (10.8%), and the United Kingdom (9.2%) accounted for the majority of reported events, reflecting media and language biases rather than the true incidence. Common settings included city streets (19.8%), private residences (15.0%), and highways (7.2%). Most incidents were accidental (90.1%), frequently involved unknown gases (43.6%), and resulted in at least 1968 affected individuals (647 nonhospitalized injuries, 1030 hospitalizations, 291 fatalities). Deliberate events most frequently involved acid attacks (predominantly by male perpetrators) and pepper-spray attacks (predominantly by female perpetrators). Conclusion: This pilot study demonstrates the feasibility of integrating large language models into an OSINT-based event detection pipeline for chemical and radiological threats. While not suitable for estimating true global incidence, this approach enables scalable, near-real-time detection of public health signals and may support early warning systems and situational awareness.