Prompt injection of OpenAI custom GPTs leaks informatics secrets.

Abstract

Low-code platforms like OpenAI custom GPTs (cGPTs) promise easy development of specialized AI assistants for complex bioinformatics and clinical tasks, allowing researchers to integrate proprietary data into intuitive chatbot interfaces. However, these commercial frameworks operate as opaque "black boxes," fundamentally clashing with open-science values and principles of reproducibility. To audit their hidden configurations, we performed "jailbreak" (prompt injection) attacks. We found that all tested cGPTs were critically vulnerable, leading to the leakage of private system instructions, full knowledge base files, and proprietary API details. This systemic failure poses severe security and privacy risks, particularly when handling sensitive patient data, clinical notes, or proprietary basic science assets. While low-code tools lower the barrier to AI adoption, their commercial nature and security flaws warrant extreme caution, forcing biomedical researchers to weigh convenience against the non-negotiable standards of data integrity and security.