BLOG AI in Coding: GitClear Report Shows Downward Pressure on Code Quality
David Schmid
  • Author: David Schmid
  • Date: 30.01.2024 Last Update: 05.03.2024, 10:14
  • Categories: AI, Software Development

A whitepaper by GitClear titled Coding on Copilot - 2023 Data Shows Downward Pressure on Code Quality suggests that the use of AI coding assistants like GitHub Copilot and ChatGPT leads to a decrease in code quality and an increase in technical debt. The study analyzed 153 million lines of code from commercial and open-source projects on GitHub between January 2020 and December 2023. It found that with the growing use of AI assistants in 2023, there was a significant increase in redundant code that violated the DRY (Don't Repeat Yourself) principle.

Main Image Blog Article AI in Coding

Table of Contents

GitClear's Analysis

In 2023 the software development landscape underwent a shift, with AI-powered tools promising to revolutionize how code is written. They offer speed and efficiency gains, and with these promises, management is often tempted to push for rapid deployment and quicker turnaround times.

This eagerness to incorporate AI for faster coding and increased productivity (in the short run), however, raises critical questions about the quality and sustainability of the code produced.

The "Coding on Copilot - 2023 Data Shows Downward Pressure on Code Quality" study by William Harding and Matthew Kloster finds worrying trends in code churn (code that is reverted or updated soon after being written), indicating a trend towards less stable code being pushed to repositories. They predict this "mistake code" will double in 2024 compared to 2021.

Image sourced by gitclear: Code Churn by Year
Image: Code Churn by Year - A graphical representation from GitClear’s study, showing the increase in code churn over the years. Source

The study also finds an increase in added and copy/pasted code and a decrease in updated, deleted, and moved code, posing a major concern for long-term code maintainability. This trend suggests a lack of time for developers to evaluate and potentially reuse existing implementations, therefore leading to a decline in code quality.

The report assumes that less experienced developers may be particularly susceptible to accepting implicit copy/paste suggestions from AI tools, which can contribute to a decline in code quality and an increase in production incidents. It emphasizes the responsibility of engineering leaders to monitor the impact of AI tools on code quality and maintainability.

The authors conclude that the resulting technical debt negates the 55 percent efficiency gain touted by GitHub itself. Either the producing team must revise the code themselves, or subsequent reviewers will have more difficulties with it. The latter is likely to be the case, as the study also shows how high the productivity pressure is on developer teams.

The problem will intensify with the further increasing use of AI assistants. To address this, the authors recommend developing specialized cleaning tools to minimize these technical debts.

Personal Insights

The push for faster delivery and quicker time-to-market has always been a challenge in software development. This race might now be accelerated with AI tools, which could then lead to more technical debt. It is crucial to consider the long-term implications of the code we produce today. Are we, in our quest for immediate results, compromising the future maintainability and stability of our systems?

Already before the rise of AI tools, platforms like GitHub and Stack Overflow had begun to reshape the developer's role. The vast repositories of available code reduced the need to develop solutions from scratch, thus encouraging a culture of "copy and paste" programming. This trend is just augmented, as I see it: AI tools start replacing traditional code platforms like Stack Overflow. They offer more than just examples - they offer context-aware suggestions.

But, as pointed out in the study, AI code, if not properly managed, can lead to bloated and convoluted codebases that are difficult to maintain.

As AI takes on more routine and repetitive coding tasks, the value of deep engineering knowledge and understanding actually becomes more pronounced. Developers will need to focus more on system design, architecture, and understanding the broader implications of the code, beyond just the immediate task at hand.

🌟 Support My Quest

If the content within these pages has enriched your journey, consider showing your support by sharing a potion of coffee with me. Such a gesture, though small, is a mighty boon to my spirit and craft. It allows me to continue sharing the lore you hold dear.

☕ Buy Me a Coffee

Let it be known that the posts I pen are born from my own personal opinions and musings, presented before you in earnest, free of shadowed veils or hidden alliances. If you find truth and heart within my words, consider supporting me with a coffee. And believe me, as a father of two young spirits, this potion is indeed the elixir of my vigilance and creativity.

Beyond sharing my journey and insights, I craft customized solutions in the realm of tech to empower and fortify your own domains.

🔍 Discover my services


No comment on this post yet... Initiate the dialogue - be the first to illuminate this page with your thoughts!

Leave a Comment

Please preserve the rules of respect and avoid any shadow that might fall upon the realm. Keep your discourse pure and use simple characters. Your scroll shall contain no more than a thousand characters.

Only the worthy may share their wisdom beneath the sacred tree of insight. To prove yourself a true hero and not a shadowy automation, solve this puzzle: