Generative artificial intelligence use in automated urban ecological assessments requires substantial human oversight

Abstract

Automated data processing pipelines and generative artificial intelligence (AI) present new opportunities for scaling ecological assessments across urban areas, yet the practical utility and limitations remain untested. This study provides a workflow for automated urban ecological reporting, which integrates 25 public datasets and performs statistical and spatial data analyses to quantify indicators of biodiversity and ecosystem services. The workflow incorporates large language models to aid synthesis and writing. Reports were generated for diverse cities worldwide and reviewed by domain experts to assess quality, trust, and potential to inform urban planning. Respondents found that while the structure and data integration had potential to be helpful, the draft reports required substantial human revision. Factual sections relying on high-quality datasets needed the fewest changes, whereas content based heavily on AI inference, such as descriptions of climate change adaptation options, were inaccurate, generic, or culturally inappropriate. Despite these limitations, participants generally viewed the reports as potentially helpful. Of the total labour required to create reports, respondents estimated that around 10% could be substituted by automation. Our findings suggest that AI-assisted automated report generation may be scaled up to support urban sustainability efforts, but only with strong human oversight and transparent disclosure of AI use. Trust in automated assessments depends on transparency, and the inclusion of local voices in legitimising final outputs. Even with automation, substantial investment in human labour will be required to make ecological assessments available for towns and cities around the world.