IEEE-ISTAS 2025

Study Finds Iterative AI Code Generation Accumulates Security Vulnerabilities Over Multiple Iterations

A 2025 IEEE-ISTAS 2025 research paper titled Security Degradation in Iterative AI Code Generation: A Systematic Analysis of the Paradox examined how repeated prompting of large language models leads to worsening code security. Researchers started with 10 secure code samples in C and Java, then applied four prompting strategies across 10 iterations each, generating 400 code samples that were analyzed with both manual review and automated scanners. The study found the strongest correlation between rising code complexity and vulnerability count, with 158 vulnerabilities emerging from feature-addition prompts and only 38 from explicit security-improvement requests. Even when asked to fix issues, GPT-4o frequently introduced new, subtler flaws such as timing side-channels, SQL injection risks, and use-after-free errors while addressing obvious problems. The authors recommend mandatory human review after every few iterations and greater use of SAST tools, noting that the illusion of progress can mask accumulating weaknesses. Limitations include testing only GPT-4o and the absence of human corrections during the iterative process.