Skip to main content
Published / updated

Debugging the Full Flow

Before you start

You need: Articles 01–05. This one puts them together across the whole stack.

Time: about 50 minutes, at the keyboard.

Learning objective

Trace one request end to end across four layers and identify the exact boundary where the data stops being correct.

Topics

  • The layers and their boundaries
  • Tracing a request through
  • The boundary method
  • Environment differences
  • Timing and race conditions
  • Caching
  • Intermittent failures
  • Writing what you found

The layers

Browser
│ user action, JavaScript, fetch

Network
│ HTTP request, headers, CORS, TLS

API
│ routing, binding, middleware, controller

Service
│ business logic

Repository
│ SQL generation, parameters

Database
query execution, data

A bug lives at exactly one boundary. The data is correct entering that boundary and wrong leaving it. Finding that boundary is the whole task; fixing it is usually easy.

The mistake is to start reading code at the layer you know best. Start at a boundary and halve the system.

Tracing a request through

A concrete failure: the fee summary page shows ₹0 for Ravi Kumar, who has paid ₹12,000.

1. Browser

DevTools → Network. Filter to Fetch/XHR.

GET /api/students/12/fees 200 142 ms

Response:

{ "studentId": 12, "totalFees": 15000, "paidAmount": 0, "balance": 15000 }

The API returned 0. The frontend is displaying what it was given, correctly. The frontend is now excluded — no need to read a line of JavaScript.

Had the response contained 12000 and the page still shown ₹0, the opposite conclusion would hold and the investigation would stop here.

2. API log

info: Executed DbCommand (8ms) [Parameters=[@__accountId_0='7']]
SELECT ISNULL(SUM([p].[Amount]), 0)
FROM [FeePayment] AS [p]
WHERE [p].[FeeAccountId] = @__accountId_0 AND [p].[IsCancelled] = 0

The API is running a query, with account id 7. The next question is whether that query is right, or whether 7 is the right account.

3. Database

SELECT * FROM FeeAccount WHERE StudentId = 12;
Id StudentId AcademicYear TotalFees PaidAmount
7 12 2023-24 15000 0
9 12 2024-25 15000 12000

Two accounts. The API picked the wrong one.

SELECT SUM(Amount) FROM FeePayment WHERE FeeAccountId = 9 AND IsCancelled = 0; -- 12000
SELECT SUM(Amount) FROM FeePayment WHERE FeeAccountId = 7 AND IsCancelled = 0; -- 0

4. The code

FeeAccount account = await _context.FeeAccounts
.FirstOrDefaultAsync(a => a.StudentId == studentId); // no academic year, no ordering

FirstOrDefault with no OrderBy returns whatever the database returns first, which is not guaranteed and changed when a row was inserted.

FeeAccount account = await _context.FeeAccounts
.Where(a => a.StudentId == studentId
&& a.SchoolId == schoolId
&& a.AcademicYear == currentAcademicYear)
.FirstOrDefaultAsync();

Four checks located a bug that could have taken a day of reading code. Each one halved the remaining system, and none required a hypothesis about where the fault was.

The boundary method

At each boundary, ask: is the data correct here?

BoundaryCheckIf correctIf wrong
Browser renderDoes the page match the response?Look at the responseFrontend rendering
Network responseIs the JSON correct?FrontendLook at the API
Controller inputAre the parameters right?Look at the serviceBinding or the client
Service outputIs the calculation right?Look at the repositoryBusiness logic
SQL textIs the query right?Look at the dataQuery construction
Query result in SSMSDoes the data support it?MappingThe data itself

Work from the outside in. Start at whichever end has cheaper evidence — usually the browser, because DevTools requires no rebuild.

Log the same identifier at every layer while you investigate:

_logger.LogInformation("Controller: studentId={StudentId} schoolId={SchoolId}", studentId, schoolId);
_logger.LogInformation("Service: found account {AccountId} year {Year}", account?.Id, account?.AcademicYear);
_logger.LogInformation("Repository: summing payments for account {AccountId}", accountId);

Three log lines often replace an hour of stepping, and they work in environments where a debugger cannot attach.

Environment differences

"It works in development" means the difference between the environments is the cause. Check them in this order:

DifferenceHow to check
ConfigurationLog the resolved values at startup — never the secrets
DataProduction has 800 students and edge cases; development has 20 clean ones
VersionWhich commit is actually deployed? Add a /version endpoint
PermissionsThe application's SQL login may lack a grant that yours has
Environment nameASPNETCORE_ENVIRONMENT decides which appsettings file wins
TimezoneThe server may be UTC and the developer machine IST
CultureDate and decimal parsing differ by locale
app.MapGet("/version", () => new
{
version = Assembly.GetExecutingAssembly().GetName().Version?.ToString(),
commit = Environment.GetEnvironmentVariable("GIT_COMMIT"),
environment = app.Environment.EnvironmentName
});

A /version endpoint ends the "did my change deploy?" question permanently.

Timezone and culture are the two that produce believable wrong answers. An attendance report off by one day, or 12,000 parsed as 12.000, both look like logic bugs and are not.

Production data is the most common difference. A student with two fee accounts, a name with an apostrophe, an exam with no results — development data rarely has these, which is why bugs wait until production to appear.

Timing and race conditions

Symptoms: works when stepping through, fails when running; fails once in twenty; different results on refresh.

// The list is fetched but rendered before it arrives
loadStudents();
renderStudents(students); // students is still empty

// Correct
const students = await loadStudents();
renderStudents(students);
// Two concurrent requests, both read PaidAmount = 0, both write 5000
account.PaidAmount = account.PaidAmount + payment.Amount;
await _context.SaveChangesAsync();

// Let the database do the arithmetic
await _context.Database.ExecuteSqlInterpolatedAsync(
$"UPDATE FeeAccount SET PaidAmount = PaidAmount + {payment.Amount} WHERE Id = {accountId}");

A read-modify-write across a network round trip is a lost update waiting to happen. Two receipts entered at the same moment and one disappears — with no error, no log, and a balance that does not reconcile.

A breakpoint changes timing, so a race condition often stops reproducing under a debugger. Use logging with timestamps instead.

A missing await is the most common race in .NET code:

_auditRepository.LogAsync(entry); // fire and forget
await _auditRepository.LogAsync(entry); // correct

Treat CS4014 as an error.

Caching

Caching turns "wrong data" into "wrong data that is also inconsistent". Layers to check, in order of how often they are the culprit:

LayerCheck
Browser cacheDevTools → Disable cache, hard reload
Service workerApplication → Service Workers → Unregister
CDNA different query string, or purge
Application memory cacheRestart the process and see if the value changes
Distributed cache (Redis)Read the key directly
EF Core change trackingAsNoTracking(), or a fresh context
SQL Server plan cacheOPTION (RECOMPILE)

If the value is right after a restart and wrong before, it is a cache.

// A cache key without SchoolId serves School A's data to School B
string key = $"students:{className}"; // wrong
string key = $"students:{schoolId}:{className}"; // correct

A missing tenant segment in a cache key is a data leak that appears intermittently — only when two schools request the same class within the cache window. It is very hard to reproduce and easy to prevent.

Intermittent failures

"It happens sometimes" is a category, not a mystery. The causes are a short list:

CauseSignature
Race conditionDepends on load or timing; stops under a debugger
CacheCorrect after a restart
Specific dataAlways the same records — it only looks random
Connection poolUnder load only
TimeoutUnder load, or a slow query
Multiple instancesOnly one server is broken — check which
Time-dependentMonth end, midnight, a leap day, DST

"Random" almost always means "a condition I have not identified yet".

Find the pattern:

  • Which users? Which records? Which school?
  • What time of day?
  • Under load, or always?
  • Which server, if there are several?
  • What else runs at that moment — a nightly job, a report?

A correlation id in logs is what makes this tractable. Take three failed requests, pull every log line for each, and compare against three successful ones. The difference is the condition.

Writing what you found

Finishing a debugging session means recording it, in the fix's commit message and in the pull request:

Fix fee summary showing zero for students with multiple fee accounts

FeeService.GetSummary called FirstOrDefault on FeeAccounts filtered only
by StudentId. Students with more than one academic year returned an
arbitrary account — in Ravi Kumar's case the 2023-24 one, with no
payments — so the page showed a paid amount of zero.

Filter by SchoolId and the current academic year, and order explicitly.
Added a test covering a student with two accounts.

The same pattern appears in AttendanceService.GetSummary (#151).

Fixes #147

Three things make this worth writing:

It records the cause, not the symptom, so the next person meeting a similar failure recognises it.

It names the pattern, so the same bug elsewhere gets found — the last paragraph is often more valuable than the fix.

It cites the test. A bug fixed without a test is a bug scheduled to return.

Diagnosing

SymptomFirst boundary to check
Page shows nothingNetwork — was a request sent?
Response has wrong dataAPI log — what SQL ran?
SQL is right, result wrongSSMS — does the data support it?
Works in dev, fails in prodConfig, data, version, permissions
Works when stepping, fails when runningTiming — add timestamped logs
Right after a restartCache
One in twenty requestsRace, pool, or specific data
Fails at month endTime-dependent logic
Only on one serverDeployment or configuration drift

Errors you will hit

SymptomFirst boundary to check
Page shows nothingNetwork — was a request sent?
Response has wrong dataAPI log — what SQL ran?
SQL right, result wrongSSMS — does the data support it?
Works in dev, fails in prodConfig, data, version, permissions
Works when stepping, fails when runningTiming — use timestamped logs
Right after a restartCache
One in twenty requestsRace, pool, or specific data

A bug lives at exactly one boundary. Find it by halving the system, not by reading the layer you know best.

Common mistakes

  • Reading code before checking the response
  • Guessing the layer instead of testing boundaries
  • Assuming the deployed version is the current one
  • Ignoring timezone and culture
  • Debugging a race condition with a breakpoint
  • Forgetting caches exist
  • Cache keys without the tenant
  • Treating "intermittent" as unexplainable
  • Fixing without adding a test
  • Not looking for the same pattern elsewhere

Practice

The course exercise is trace a failure end to end.

  1. Recreate the two-fee-accounts bug: give a student two accounts, use FirstOrDefault with no ordering, and trace it through all four layers.
  2. At each layer, write down whether the data was correct there.
  3. Add the three log lines and rerun. Compare the time taken with stepping.
  4. Break the frontend rendering while the API returns correct data, and confirm the trace stops at the response.
  5. Add a /version endpoint and confirm which commit is deployed.
  6. Change ASPNETCORE_ENVIRONMENT and observe which config file wins.
  7. Set the server to UTC and the client to IST, then produce an attendance report off by one day.
  8. Parse 12,000 under two cultures and compare the results.
  9. Write the read-modify-write payment update, fire two concurrent requests, and lose one.
  10. Fix it with a database-side increment and confirm both succeed.
  11. Remove an await and observe the exception disappearing.
  12. Cache a student list with a key lacking schoolId, then request the same class from two schools.
  13. Confirm the value is correct after a restart, and use that to prove it is a cache.
  14. Add a correlation id, generate twenty requests where three fail, and find the condition by comparing logs.
  15. Fix one real bug and write the commit message in full — cause, fix, test, and where else the pattern appears.

Exercises 9, 10 and 12 are the ones that only show up under concurrency. They are worth doing deliberately, because in production they arrive as "it happens sometimes".

You can now

  • Trace a request across every layer
  • Name the boundary where the data first goes wrong
  • Recognise environment, timing and cache causes
  • Use a correlation id to trace one request
  • Write up a bug so the next person recognises the pattern

Review questions

  1. What does a correct API response with a wrong page tell you?
  2. Which environment differences most often explain "works in development"?
  3. Why does a race condition often stop reproducing under a debugger?
  4. What makes an intermittent failure findable?

Next: Broken full-stack feature lab