Debugging the Full Flow
Before you start
You need: Articles 01–05. This one puts them together across the whole stack.
Time: about 50 minutes, at the keyboard.
Learning objective
Trace one request end to end across four layers and identify the exact boundary where the data stops being correct.
Topics
- The layers and their boundaries
- Tracing a request through
- The boundary method
- Environment differences
- Timing and race conditions
- Caching
- Intermittent failures
- Writing what you found
The layers
Browser
│ user action, JavaScript, fetch
▼
Network
│ HTTP request, headers, CORS, TLS
▼
API
│ routing, binding, middleware, controller
▼
Service
│ business logic
▼
Repository
│ SQL generation, parameters
▼
Database
query execution, data
A bug lives at exactly one boundary. The data is correct entering that boundary and wrong leaving it. Finding that boundary is the whole task; fixing it is usually easy.
The mistake is to start reading code at the layer you know best. Start at a boundary and halve the system.
Tracing a request through
A concrete failure: the fee summary page shows ₹0 for Ravi Kumar, who has paid ₹12,000.
1. Browser
DevTools → Network. Filter to Fetch/XHR.
GET /api/students/12/fees 200 142 ms
Response:
{ "studentId": 12, "totalFees": 15000, "paidAmount": 0, "balance": 15000 }
The API returned 0. The frontend is displaying what it was given, correctly. The frontend is now excluded — no need to read a line of JavaScript.
Had the response contained 12000 and the page still shown ₹0, the opposite conclusion would hold and the investigation would stop here.
2. API log
info: Executed DbCommand (8ms) [Parameters=[@__accountId_0='7']]
SELECT ISNULL(SUM([p].[Amount]), 0)
FROM [FeePayment] AS [p]
WHERE [p].[FeeAccountId] = @__accountId_0 AND [p].[IsCancelled] = 0
The API is running a query, with account id 7. The next question is whether that query is right, or whether 7 is the right account.
3. Database
SELECT * FROM FeeAccount WHERE StudentId = 12;
Id StudentId AcademicYear TotalFees PaidAmount
7 12 2023-24 15000 0
9 12 2024-25 15000 12000
Two accounts. The API picked the wrong one.
SELECT SUM(Amount) FROM FeePayment WHERE FeeAccountId = 9 AND IsCancelled = 0; -- 12000
SELECT SUM(Amount) FROM FeePayment WHERE FeeAccountId = 7 AND IsCancelled = 0; -- 0
4. The code
FeeAccount account = await _context.FeeAccounts
.FirstOrDefaultAsync(a => a.StudentId == studentId); // no academic year, no ordering
FirstOrDefault with no OrderBy returns whatever the database returns first, which is not guaranteed and changed when a row was inserted.
FeeAccount account = await _context.FeeAccounts
.Where(a => a.StudentId == studentId
&& a.SchoolId == schoolId
&& a.AcademicYear == currentAcademicYear)
.FirstOrDefaultAsync();
Four checks located a bug that could have taken a day of reading code. Each one halved the remaining system, and none required a hypothesis about where the fault was.
The boundary method
At each boundary, ask: is the data correct here?
| Boundary | Check | If correct | If wrong |
|---|---|---|---|
| Browser render | Does the page match the response? | Look at the response | Frontend rendering |
| Network response | Is the JSON correct? | Frontend | Look at the API |
| Controller input | Are the parameters right? | Look at the service | Binding or the client |
| Service output | Is the calculation right? | Look at the repository | Business logic |
| SQL text | Is the query right? | Look at the data | Query construction |
| Query result in SSMS | Does the data support it? | Mapping | The data itself |
Work from the outside in. Start at whichever end has cheaper evidence — usually the browser, because DevTools requires no rebuild.
Log the same identifier at every layer while you investigate:
_logger.LogInformation("Controller: studentId={StudentId} schoolId={SchoolId}", studentId, schoolId);
_logger.LogInformation("Service: found account {AccountId} year {Year}", account?.Id, account?.AcademicYear);
_logger.LogInformation("Repository: summing payments for account {AccountId}", accountId);
Three log lines often replace an hour of stepping, and they work in environments where a debugger cannot attach.
Environment differences
"It works in development" means the difference between the environments is the cause. Check them in this order:
| Difference | How to check |
|---|---|
| Configuration | Log the resolved values at startup — never the secrets |
| Data | Production has 800 students and edge cases; development has 20 clean ones |
| Version | Which commit is actually deployed? Add a /version endpoint |
| Permissions | The application's SQL login may lack a grant that yours has |
| Environment name | ASPNETCORE_ENVIRONMENT decides which appsettings file wins |
| Timezone | The server may be UTC and the developer machine IST |
| Culture | Date and decimal parsing differ by locale |
app.MapGet("/version", () => new
{
version = Assembly.GetExecutingAssembly().GetName().Version?.ToString(),
commit = Environment.GetEnvironmentVariable("GIT_COMMIT"),
environment = app.Environment.EnvironmentName
});
A /version endpoint ends the "did my change deploy?" question permanently.
Timezone and culture are the two that produce believable wrong answers. An attendance report off by one day, or 12,000 parsed as 12.000, both look like logic bugs and are not.
Production data is the most common difference. A student with two fee accounts, a name with an apostrophe, an exam with no results — development data rarely has these, which is why bugs wait until production to appear.
Timing and race conditions
Symptoms: works when stepping through, fails when running; fails once in twenty; different results on refresh.
// The list is fetched but rendered before it arrives
loadStudents();
renderStudents(students); // students is still empty
// Correct
const students = await loadStudents();
renderStudents(students);
// Two concurrent requests, both read PaidAmount = 0, both write 5000
account.PaidAmount = account.PaidAmount + payment.Amount;
await _context.SaveChangesAsync();
// Let the database do the arithmetic
await _context.Database.ExecuteSqlInterpolatedAsync(
$"UPDATE FeeAccount SET PaidAmount = PaidAmount + {payment.Amount} WHERE Id = {accountId}");
A read-modify-write across a network round trip is a lost update waiting to happen. Two receipts entered at the same moment and one disappears — with no error, no log, and a balance that does not reconcile.
A breakpoint changes timing, so a race condition often stops reproducing under a debugger. Use logging with timestamps instead.
A missing await is the most common race in .NET code:
_auditRepository.LogAsync(entry); // fire and forget
await _auditRepository.LogAsync(entry); // correct
Treat CS4014 as an error.
Caching
Caching turns "wrong data" into "wrong data that is also inconsistent". Layers to check, in order of how often they are the culprit:
| Layer | Check |
|---|---|
| Browser cache | DevTools → Disable cache, hard reload |
| Service worker | Application → Service Workers → Unregister |
| CDN | A different query string, or purge |
| Application memory cache | Restart the process and see if the value changes |
| Distributed cache (Redis) | Read the key directly |
| EF Core change tracking | AsNoTracking(), or a fresh context |
| SQL Server plan cache | OPTION (RECOMPILE) |
If the value is right after a restart and wrong before, it is a cache.
// A cache key without SchoolId serves School A's data to School B
string key = $"students:{className}"; // wrong
string key = $"students:{schoolId}:{className}"; // correct
A missing tenant segment in a cache key is a data leak that appears intermittently — only when two schools request the same class within the cache window. It is very hard to reproduce and easy to prevent.
Intermittent failures
"It happens sometimes" is a category, not a mystery. The causes are a short list:
| Cause | Signature |
|---|---|
| Race condition | Depends on load or timing; stops under a debugger |
| Cache | Correct after a restart |
| Specific data | Always the same records — it only looks random |
| Connection pool | Under load only |
| Timeout | Under load, or a slow query |
| Multiple instances | Only one server is broken — check which |
| Time-dependent | Month end, midnight, a leap day, DST |
"Random" almost always means "a condition I have not identified yet".
Find the pattern:
- Which users? Which records? Which school?
- What time of day?
- Under load, or always?
- Which server, if there are several?
- What else runs at that moment — a nightly job, a report?
A correlation id in logs is what makes this tractable. Take three failed requests, pull every log line for each, and compare against three successful ones. The difference is the condition.
Writing what you found
Finishing a debugging session means recording it, in the fix's commit message and in the pull request:
Fix fee summary showing zero for students with multiple fee accounts
FeeService.GetSummary called FirstOrDefault on FeeAccounts filtered only
by StudentId. Students with more than one academic year returned an
arbitrary account — in Ravi Kumar's case the 2023-24 one, with no
payments — so the page showed a paid amount of zero.
Filter by SchoolId and the current academic year, and order explicitly.
Added a test covering a student with two accounts.
The same pattern appears in AttendanceService.GetSummary (#151).
Fixes #147
Three things make this worth writing:
It records the cause, not the symptom, so the next person meeting a similar failure recognises it.
It names the pattern, so the same bug elsewhere gets found — the last paragraph is often more valuable than the fix.
It cites the test. A bug fixed without a test is a bug scheduled to return.
Diagnosing
| Symptom | First boundary to check |
|---|---|
| Page shows nothing | Network — was a request sent? |
| Response has wrong data | API log — what SQL ran? |
| SQL is right, result wrong | SSMS — does the data support it? |
| Works in dev, fails in prod | Config, data, version, permissions |
| Works when stepping, fails when running | Timing — add timestamped logs |
| Right after a restart | Cache |
| One in twenty requests | Race, pool, or specific data |
| Fails at month end | Time-dependent logic |
| Only on one server | Deployment or configuration drift |
Errors you will hit
| Symptom | First boundary to check |
|---|---|
| Page shows nothing | Network — was a request sent? |
| Response has wrong data | API log — what SQL ran? |
| SQL right, result wrong | SSMS — does the data support it? |
| Works in dev, fails in prod | Config, data, version, permissions |
| Works when stepping, fails when running | Timing — use timestamped logs |
| Right after a restart | Cache |
| One in twenty requests | Race, pool, or specific data |
A bug lives at exactly one boundary. Find it by halving the system, not by reading the layer you know best.
Common mistakes
- Reading code before checking the response
- Guessing the layer instead of testing boundaries
- Assuming the deployed version is the current one
- Ignoring timezone and culture
- Debugging a race condition with a breakpoint
- Forgetting caches exist
- Cache keys without the tenant
- Treating "intermittent" as unexplainable
- Fixing without adding a test
- Not looking for the same pattern elsewhere
Practice
The course exercise is trace a failure end to end.
- Recreate the two-fee-accounts bug: give a student two accounts, use
FirstOrDefaultwith no ordering, and trace it through all four layers. - At each layer, write down whether the data was correct there.
- Add the three log lines and rerun. Compare the time taken with stepping.
- Break the frontend rendering while the API returns correct data, and confirm the trace stops at the response.
- Add a
/versionendpoint and confirm which commit is deployed. - Change
ASPNETCORE_ENVIRONMENTand observe which config file wins. - Set the server to UTC and the client to IST, then produce an attendance report off by one day.
- Parse
12,000under two cultures and compare the results. - Write the read-modify-write payment update, fire two concurrent requests, and lose one.
- Fix it with a database-side increment and confirm both succeed.
- Remove an
awaitand observe the exception disappearing. - Cache a student list with a key lacking
schoolId, then request the same class from two schools. - Confirm the value is correct after a restart, and use that to prove it is a cache.
- Add a correlation id, generate twenty requests where three fail, and find the condition by comparing logs.
- Fix one real bug and write the commit message in full — cause, fix, test, and where else the pattern appears.
Exercises 9, 10 and 12 are the ones that only show up under concurrency. They are worth doing deliberately, because in production they arrive as "it happens sometimes".
You can now
- Trace a request across every layer
- Name the boundary where the data first goes wrong
- Recognise environment, timing and cache causes
- Use a correlation id to trace one request
- Write up a bug so the next person recognises the pattern
Review questions
- What does a correct API response with a wrong page tell you?
- Which environment differences most often explain "works in development"?
- Why does a race condition often stop reproducing under a debugger?
- What makes an intermittent failure findable?