AI Code Isn’t Always Clean: How to Spot and Fix Errors

AI code can contain bugs, security vulnerabilities, or simply misunderstand what you’re trying to build. That’s why developers still need to review and test AI code rigorously before deploying it into production.

AI coding assistants like GitHub Copilot, Amazon CodeWhisperer, and Hostinger’s Horizons are revolutionising the way developers write code. These tools can generate AI code on the fly, suggest optimisations, and even auto-complete functions. But as impressive as they are, they aren’t infallible.

In this article, we’ll explore:

• Why AI code makes mistakes

• How frequently those errors occur

• And what developers can do to catch and correct them

AI tools may boost speed and productivity, but human oversight remains critical. The key is treating AI as a coding assistant—not a replacement.

Why do code errors happen in the first place

At the core of most AI code assistants is a large language model (LLM). These models are trained on billions of lines of code and learn to predict the code that comes next based on patterns. 

“The problem is that these models don’t truly “understand” code. Instead, they estimate likely code completions, guess based on past examples, and sometimes just make things up – this is called hallucination. Plus, they don’t always see the full context of a project. These issues lead to common coding errors like missing or incorrect code blocks, wrong logic, or non-existent functions or libraries,” says Dainius Kavoliūnas, Head of Hostinger Horizons.

A recent study1 found that different models make different mistakes even in the same task. This suggests it’s not just a bug in one model – it’s how they all work. 

AI code accuracy is improving, but it’s still far from perfect

The good news: LLMs are getting better. Models like GPT-4 (2023) now write correct code on the first try about 85.7% of the time, according to a Vanderbilt University study2. That’s a huge improvement compared to older models that often failed entirely3.

It doesn’t stop there. Industry is working on improvements by training on more secure and accurate code, adding automated testing during code generation, and building smarter filters.

Still, improving AI is a process, not a one-time task, and it won’t become 100% accurate in a week or a month.

Tips to avoid AI code errors

Despite these limitations, AI coding tools remain powerful aids, especially when paired with the right features and strategies. 

“If an AI coding tool makes a mistake, a good first step is to just ask it to fix the problem. That’s why we built automatic, free error fixing into Horizons. When an error is detected, Horizons attempts to fix it – resolving about 80% of issues automatically, without any user action,” says Dainius Kavoliūnas.

He also shares a few practical tips to handle AI errors more effectively:

• Try a new approach. Revert to an earlier, error-free version of the code and update the prompt. 
• Be specific. Write clear, detailed instructions. Break tasks into smaller steps, give examples, and add restrictions when prompting.
• Test often. After each change, make sure everything works as intended. A quick test will catch many bugs and logic errors before they sneak into production.
• Ask for a rewrite. If AI can’t fix the error, ask it to explain or simplify the code. Rewriting code is great for cleanup and may reduce bugs.
• Review the code. If the AI-generated code is visible and understandable, take a moment to look it over, especially when it involves data, logic, or anything related to security.

AI will keep getting smarter, and one day it may be capable of writing bug-free code. Until then, it’s important to use AI assistance carefully to avoid letting its mistakes cause problems.