Skip to main content
Published / updated

Debugging and Refactoring with AI

Before you start​

You need: small verified changes (Article 03) and debugging (Track 17).

Time: about 45 minutes, at the keyboard.

Learning objective​

Use AI effectively on a bug you have already investigated, and refactor without changing behaviour.

Topics​

  • What AI needs to help with a bug
  • Prompts that work for debugging
  • Prompts that waste your time
  • Verifying a suggested cause
  • Refactoring safely
  • Generating tests
  • Explaining unfamiliar code
  • Where AI is wrong about bugs

What AI needs to help with a bug​

AI is not a debugger. It is a second opinion on evidence you have gathered.

Useless:  My fee page is broken. What's wrong?

There is no evidence. Any answer is a guess, and a plausible guess costs you an hour chasing it.

Useful:

The fee summary returns paidAmount = 0 for a student who has paid ₹12,000.

Evidence:
- Network: GET /api/students/12/fees returns 200
  { "studentId": 12, "totalFees": 15000, "paidAmount": 0, "balance": 15000 }
- API log shows this SQL running:
  SELECT ISNULL(SUM(p.Amount), 0) FROM FeePayment p
  WHERE p.FeeAccountId = @accountId AND p.IsCancelled = 0
  with @accountId = 7
- In SSMS:
  SELECT * FROM FeeAccount WHERE StudentId = 12
  returns two rows: Id 7 (2023-24, paid 0) and Id 9 (2024-25, paid 12000)

The service code:
FeeAccount account = await _context.FeeAccounts
    .FirstOrDefaultAsync(a => a.StudentId == studentId);

I think it's picking the wrong account. Am I right, and what's the
correct fix given we have one account per academic year?

That prompt gets a correct answer because it contains the answer's ingredients. You did the debugging; AI confirms the reasoning and suggests the fix.

The work of gathering evidence is not something AI can do for you — it requires access to your running system, your logs and your data.

Prompts that work​

Here's the exception and the stack trace. Here's the method at the top
frame. What are the three most likely causes, in order?

[trace]
[code]
This query returns 780 rows; the students table has 800 for this school.
What could drop 20 rows?

[SQL]
I've ruled out: token expiry (exp is future), CORS (same origin),
and the SchoolId filter (present in the query).
What else should I check?
This works in development and fails in production with a timeout.
Same code, same query. What differences should I check first?
Explain why this LINQ generates 801 queries.

[code]

"What are the three most likely causes, in order?" is better than "what's wrong?" — it gives you a checklist to test rather than one answer to trust.

Telling it what you have ruled out is the highest-value line in a debugging prompt. Without it, the first three suggestions are the things you already checked.

Prompts that waste your time​

Bad:  It doesn't work.
Bad:  Fix my code. [pastes 500 lines]
Bad:  Why is this null?           (no context on what it is or where)
Bad:  [pastes the whole file] There's a bug somewhere.

These produce plausible answers that are usually about a different bug than yours. You then spend an hour on the suggestion, which is worse than having asked nothing.

Never ask before reproducing. If you cannot state the steps, expected and actual, you are not ready to ask — of an AI or a colleague.

Verifying a suggested cause​

Every suggestion is a hypothesis. Test it.

Suggestion: FirstOrDefault has no ordering, so the account returned is
            arbitrary.
Test:       SELECT * FROM FeeAccount WHERE StudentId = 12
Result:     Two rows. The service used the older one.
Verdict:    Confirmed.
Suggestion: The connection pool is exhausted.
Test:       SELECT COUNT(*) FROM sys.dm_exec_connections
Result:     4 connections.
Verdict:    Rejected.

A rejected suggestion is still progress — it eliminates a possibility.

AI suggestions are ranked by how common the cause is in general, not by how likely it is in your system. Common causes are a good place to start and a bad place to stop. When the suggestion is confidently specific about your code — "line 47 is the problem" — treat that as the least reliable part of the answer, because it is the part requiring knowledge of your system.

Refactoring safely​

The rule: refactoring changes structure, never behaviour.

1. Ensure tests exist and pass
2. Refactor
3. Tests still pass
4. Commit — refactoring only, no behaviour change

Without tests, do not refactor. You have no way to know whether behaviour changed, and generated refactors change behaviour more often than they appear to.

This method is 120 lines. Extract the fee calculation into a private method.
Do not change any behaviour. Keep the same edge-case handling, including
the null check on DiscountAmount.

[paste method]
Convert this to async. Every database call becomes async, methods return
Task, and every call site awaits. Do not change the logic.
This has the same query in four places. Extract it into one method.
Show me the diff for each call site so I can check them individually.

Refactor and behaviour change in separate commits, always. Mixed together, a reviewer cannot tell which of 200 changed lines alters what the program does — and neither can you in six months.

Common ways a generated refactor changes behaviour quietly:

ChangeEffect
First() → FirstOrDefault()Silent null instead of an exception
== → .Equals()Different null handling
Reordering conditionsDifferent short-circuit behaviour
int → long, float → decimalDifferent arithmetic, sometimes better
Adding a null checkHides a bug that should surface
Removing a seemingly redundant checkThe check was load-bearing
IEnumerable → ListChanges when the query executes

The last one is a classic in EF Core: materialising early turns one query into many, or moves filtering from SQL to memory.

Read the diff, not the summary. "I refactored this into three methods" tells you nothing about whether the third condition still short-circuits.

Generating tests​

Tests are one of the better uses of AI, with one condition.

Here's the grading method. Write xUnit tests covering:
- A passing student
- A failing student
- A student exactly on the passing mark
- An absent student with null marks
- A subject with passing marks of 0

Use Arrange/Act/Assert with explicit sections and descriptive names.

[paste method]

Specify the cases yourself. Ask for "tests for this method" and you get tests for the happy path plus whatever the model guesses — and if the code has a bug, the tests assert the bug.

// Generated from buggy code — asserts the bug
[Fact]
public void GetResult_AbsentStudent_ReturnsFail()
{
    // this is wrong, and it will pass
}

Write the case list from the requirement, not the implementation. That is the condition. It is the same discipline as writing your own test cases before reading the code.

Verify each test fails without the fix. Comment out the implementation, run the test, watch it fail. A test that passes either way tests nothing, and generated tests include those more often than hand-written ones.

Good uses:

  • Boilerplate arrange sections and test data builders
  • Parameterised [Theory] cases from a list you supply
  • Tests for existing, known-correct behaviour before a refactor

Explaining unfamiliar code​

The lowest-risk, highest-value use.

Explain this stored procedure step by step. I'm unsure about the CTE.

[SQL]
This is a VB.NET module from a legacy system. What does it do,
and what looks like a bug?

[code]
Trace what happens when POST /api/students is called in this controller.
Which middleware runs, in what order?
This regex validates roll numbers. Explain each part, and tell me
what it rejects that it probably shouldn't.

Verify the explanation against the code. It is right in front of you — a wrong explanation is catchable in a way that a wrong claim about a framework is not.

"What looks like a bug?" on legacy code is genuinely productive. It will list five things; two will be real, and finding those two by reading would have taken an afternoon.

Where AI is wrong about bugs​

SituationWhy it struggles
Your dataIt cannot see your database
Your configurationIt cannot see your environment
Race conditionsNeeds timing evidence you have to gather
Environment differencesIt does not know what differs
Recent framework versionsTraining cutoff
Business rulesIt does not know what correct means
"It worked yesterday"It does not know what changed — git log does

"It worked yesterday" is a git log question, not an AI question. So is "which commit broke this" — that is git bisect.

A confident answer about your specific code is the least reliable kind, because it requires exactly the knowledge the model does not have. Confidence in an AI answer correlates with fluency, not accuracy.

Errors you will hit​

The mistakeConsequence
Asking before reproducingA confident answer about a different bug
Pasting code with no evidenceSuggestions ranked by what is common, not what is true here
Not saying what you ruled outThe first three suggestions are what you already checked
Accepting a cause without testing itAn hour spent on the wrong thing
Refactoring without testsNo way to know behaviour changed
Reading the summary, not the diffA behaviour change described as a refactor

AI is a second opinion on evidence you gathered, not a substitute for gathering it.

Common mistakes​

  • Asking before reproducing
  • Pasting code with no evidence
  • Not saying what you have ruled out
  • Accepting the first suggested cause without testing it
  • Refactoring without tests
  • Mixing refactor and behaviour change in one commit
  • Reading the summary instead of the diff
  • Asking for tests without specifying the cases
  • Not checking that a test fails without the fix
  • Asking AI what git log would answer

Practice​

The course exercise is debug with AI, then verify.

  1. Ask about a bug with no evidence. Note how plausible and how wrong the answer is.
  2. Ask about the same bug with the full evidence set. Compare.
  3. Add "here's what I've ruled out" to a prompt and compare the suggestions.
  4. Ask for "the three most likely causes, in order" and test each.
  5. Test a suggestion and reject it. Record what it eliminated.
  6. Take a suggestion that names a specific line and check whether it is right.
  7. Refactor a method with no tests, then diff carefully and list what could have changed.
  8. Write tests first, then refactor, and confirm they still pass.
  9. Ask for a refactor and check the diff for each item in the behaviour-change table.
  10. Refactor IEnumerable to List in an EF Core query and count the SQL statements before and after.
  11. Ask for "tests for this method" on code with the absent-check bug. Check whether the tests assert the bug.
  12. Supply the case list from the requirement instead, and confirm the absent test fails.
  13. Comment out a fix and confirm the corresponding test fails.
  14. Paste an unfamiliar stored procedure and verify the explanation line by line.
  15. Ask "what looks like a bug?" on real legacy code and check each claim.
  16. Ask an AI which commit broke something, then answer it with git bisect.

Exercises 11 and 12 are the pair that matters. A test suite that asserts your bugs is worse than no test suite, because it makes them look verified.

You can now​

  • Give an AI the evidence a bug needs
  • Say what you have already ruled out
  • Test a suggested cause rather than trusting it
  • Refactor without changing behaviour
  • Prove a generated test fails without its fix

Review questions​

  1. What must you do before asking an AI about a bug?
  2. Why does listing what you have ruled out improve the answer?
  3. Why must refactoring and behaviour change be separate commits?
  4. Why can generated tests be worse than no tests?

Next: Reviewing AI-generated code

Published on by Sahasra Admin