Autonomous Debugging: The Next Step in Generative AI for Developers

Disclaimer: This blog post was written as part of a collaboration with Lightrun.

The Rise of Generative AI and the Fall of Stack Overflow

Generative AI, once viewed as a fantasy, a science-fiction tale only manifesting in Hollywood productions, has quickly integrated itself into every aspect of our daily lives, radically transforming how we interact with technology.

From personalized recommendations on streaming platforms when you open your Netflix app to advanced chatbots that handle complex queries when you invite ChatGPT or Gemini to solve your problems, AI is indeed part of our daily life.

These AI systems power many of the automated processes we now take for granted. They create human-like language, generate realistic graphics, design complex 3D models, develop customized marketing content, enhance photo editing, compose music, optimize supply chain logistics, assist in diagnosing medical conditions, translate languages in real-time, recommend financial investments, detect fraud, automate customer service responses, curate news articles, and even predict weather patterns with greater accuracy - all done with a simplicity that hides a complexity powering the underlying technology.

But how are developers and software engineers using this potential?

In a blog post called “The Fall of Stack Overflow”, Ayhan Ç explored a concerning patterns that have surfaced in the last several months. The post visually described the noticeable drop in both traffic and activity on the first and most popular question-and-answer site for developers, Stack Overflow.

Both the number of visits and user interactions are declining, as seen by the shared visuals, which indicate a 50% decrease in votes, questions, and answers. This drop suggests that the once-mainstay of the developer community, Stack Overflow, may be struggling to maintain its user base and relevancy. It also signifies a more significant change in the way tech enthusiasts and developers engage with internet resources.

"ChatGPT seems to be better than Stack Overflow, both in speed and accuracy", one user posted on Stack Overflow. "The speed and the accuracy of OpenAI responses are what gives it an edge over Stack Overflow and then users won't get banned or question limit imposed on users based on time intervals," added the user.

Generative AI for Developers: The Good, the Bad and the Ugly

With its advantages and limitations, generative AI is playing an important role today in the software development process. It is helping developers write code faster, debug more efficiently, and improve the overall quality of their code. This is exactly what a McKinsey study showed: developers can complete coding tasks up to twice as fast with generative AI.

According to Forbes, tools like Github Copilot, Amazon Q Developer, IBM watsonx Code Assistant, and CodeWP are the most used GenAI tools by developers.

However, these tools are just the tip of the iceberg. Even if they can help with debugging, their main purpose is to help developers write code faster. There are two problems here:

The first one is obvious - software engineering is not just about writing code. It's about understanding the problem, designing a solution, implementing it, and most importantly, maintaining and debugging it.

Writing code is just a small part of the process. According to recent research, developers spend half of their time debugging. That's the biggest time sink in software engineering. Software bugs cost the industry $316 billion a year. So, the real problem is not only writing code but also debugging it and being as efficient as possible in the process of identifying and fixing bugs. This involves understanding the problem, observing the behavior of the code, and monitoring its execution.

"Debugging is twice as hard as writing the code in the first place", Brian Kernighan once said. Every software engineer with a few years of experience can confirm this.

The second problem is that Generative AI, while being used extensively today by all types of developers, can produce code that is not always efficient, secure, and bug-free. As a result, code today is produced faster than ever before, but it also has more chances of being buggy.

In 2022-2023, the rise of AI Assistants are strongly correlated with "mistake code" being pushed to the repo. This is what "Coding on Copilot" report from GitClear found after analyzing 153 million lines of code

Generative AI for Developers: Moving Beyond the Status Quo

The Stack Overflow Developer Survey 2024 reveals two intriguing insights about developers' use of Generative AI tools. Firstly, 82% of developers utilizing this technology are applying it to code generation. However, fewer developers are harnessing it for other purposes:

• Debug: 56.7%
• Test code: 27.2%
• Deploy and monitor: 4.5%

The second fact shows that developers are more interested in using Generative AI tools to monitor, debug, and test code than to generate it.

Combining these two facts, we can infer that developers are not using Generative AI tools the way they would like to. They are using it to generate code, but they want to use it to debug, test, and monitor code.

A question arises: what's stopping developers from using Generative AI tools to debug and fix code? The answer is that most tools are not designed for this goal. If you take a look at the most popular Generative AI tools, you will see that they are focused on generating code. Even the testing features are limited to generating test cases, which is code generation too, and not effective for debugging, functional testing, or monitoring and observability.

Introducing Lightrun's Autonomous Debugging: The Next Step in GenAI for Developers

As we have seen, while debugging is the most time-consuming task in the software development process, developers are still not equipped with the right Generative AI tools to help them in this matter. At Lightrun, we noticed that the current debugging tools are not enough to help developers in their daily tasks. That’s why we built a complete developer-centric dynamic observability platform that helps developers to debug their applications in real-time without the need to redeploy or restart their applications. With the democratization of AI, we believe that its true potential lies in tackling the broader challenges developers face daily - particularly by augmenting their debugging capabilities, not just generating code and documentation.

That’s why we’re thrilled to introduce Lightrun’s Autonomous Debugging, the industry's first Runtime Autonomous Debugger powered by Generative AI, now available directly in your IDE.

Our new feature automates the entire runtime debugging process for live distributed applications, including mission-critical production workloads. This means that developers and Site Reliability Engineers (SREs) can break free from the endless cycle of troubleshooting and reduce the time from the initial ticket to pinpointing the exact line of code causing the issue (MTTR).

How Lightrun's Autonomous Debugging Works

Let’s dive into a real-world example to show how it works. In the following video, we demonstrate how Lightrun's AI assistant tackles a user authentication bug reported in a Jira ticket.

The process begins with our AI assistant analyzing a ticket directly within the IDE. From there, it suggests a potential root cause and recommends dynamic logs or snapshots—virtual breakpoints tailored to the specific runtime issue. Four simple steps between the problem and the solution:

1. Problem Analysis: We start by copying the title of the Jira ticket related to a user authentication bug and paste it into Lightrun's autonomous debugger input box. We can also add a context like a stack trace, a log message, or any additional information about the issue. With just one click, our AI assistant creates a hypothesis, suggesting what might be the root cause of the problem.
2. AI-Powered Suggestions: Lightrun’s autonomous debugger provides insights into what it thinks is happening and suggests inserting a snapshot at the relevant location. This snapshot is pre-configured by the AI. It captures the necessary runtime data to validate the hypothesis.
3. Runtime Verification: As the snapshot is captured from one of the live instances of the application, you can quickly verify the root cause. For example, in the case described in the video, Lightrun’s AI assistant helped the user with its suggestion to identify an issue with authorities variable and to confirm that this field is indeed problematic.
4. Resolution: With the root cause identified, you can now fix the issue and redeploy the solution with more confidence. To double-check, you can easily drill down into the variable tree in the snapshot pane and inspect variables.

Lightrun simulates and streamlines the development process for resolving runtime problems. This iterative technique involves identifying a possible root cause using ITOps and observability signals. Then, its dynamic observability SDK is used to add dynamic snapshots and logs on the fly to specific lines of code, enabling line-by-line runtime debugging. This cycle is repeated until the underlying cause is found. Potential root causes are suggested by Lightrun's GenAI runtime debugging models and verified by real-time production data collected by the SDK.

Lightrun’s Autonomous Debugging not only addresses the challenges of integrating GenAI into the software development lifecycle but also provides a safety net that ensures code quality remains intact. As generative AI solutions continue to enhance their code generation capabilities, the need for robust debugging tools has never been more critical.

You too can experience the power of Lightrun’s Autonomous Debugging by signing up for our Early Access Private Beta. Join the waitlist today and be among the first to experience the future of generative AI for developers.