Release, Deployment, and Support Basics
Before you start
You need: environments (Article 04) and testing (Article 05).
Time: about 45 minutes, plus the practice.
Learning objective
Explain how verified code reaches production safely, and what happens when something goes wrong once it is there.
Topics
- Build, deployment and release
- Versioning
- The deployment sequence
- Rollback and database migrations
- Configuration and secrets in production
- Logs, metrics and alerts
- Incident response
- Hotfixes and post-incident review
Build, deployment and release
Three words used interchangeably, meaning three different things.
| Term | Means |
|---|---|
| Build | Compiling source into a runnable artifact |
| Deployment | Putting that artifact onto a server |
| Release | Making it available to users |
Deployment and release can be separated, and separating them is powerful. Code can be deployed to production with a feature switched off, then released later by turning a flag on — for one school first, then all of them.
if (_featureFlags.IsEnabled("StudentSearch", schoolId))
{
return await _searchService.SearchStudentsAsync(schoolId, query);
}
return await _studentService.GetAllStudentsAsync(schoolId);
Turning a feature off then becomes a flag change, not a rollback. That difference matters at 9 pm.
One artifact is built once and promoted through QA, UAT and production. Rebuilding per environment means the thing tested is not the thing shipped.
Versioning
v2.4.1
│ │ │
│ │ └── PATCH — a backwards-compatible fix
│ └──── MINOR — a backwards-compatible feature
└────── MAJOR — a breaking change
| Change | Version |
|---|---|
| Fixed the class-average bug | 2.4.0 → 2.4.1 |
| Added student search | 2.4.1 → 2.5.0 |
| Renamed an API field every client depends on | 2.5.0 → 3.0.0 |
Tag every release in Git. A tag answers "what exactly is in production?" without reading deployment logs and guessing from dates.
A /version endpoint answers it from the running system:
{ "version": "2.4.1", "commit": "a3f9c1d", "environment": "Production", "deployedAt": "2026-08-27T14:20:00Z" }
This ends the "did my change actually deploy?" question permanently, which otherwise consumes an hour of every incident.
The deployment sequence
1. All tests pass on the release branch
2. Deploy to UAT; the school's staff sign off
3. Schedule the window and tell the users
4. Back up the production database
5. Apply database migrations
6. Deploy the application
7. Smoke test
8. Monitor
9. Confirm the release
| Step | Why it is there |
|---|---|
| Back up first | The only real undo for a data problem |
| Migrations before the application | New code expecting a new column fails without it |
| Smoke test | Ten minutes of checking beats a user reporting it |
| Monitor | Errors often appear under real load, not at deployment |
A smoke test is a short list of the flows that must work:
[ ] Sign in as an office administrator
[ ] Student list loads, filtered to the right school
[ ] Search a known roll number
[ ] Open a fee summary; the balance is right
[ ] Record a small test payment, then reverse it
[ ] Open the class result report; absentees show as Absent
Do not deploy on a Friday afternoon. Not superstition: if it breaks, the people who can fix it are gone, and it stays broken until Monday. Most teams deploy Tuesday to Thursday morning.
Tell users before, not after. "The portal will be unavailable from 7 to 7:30 pm" is a notice. Discovering it themselves is an incident.
Rollback and migrations
Rolling back application code is straightforward. Deploy the previous version.
Rolling back a database change is often not.
-- Forward: harmless
ALTER TABLE Student ADD IsDeleted BIT NOT NULL DEFAULT 0;
-- Backward: loses every value in that column
ALTER TABLE Student DROP COLUMN IsDeleted;
-- Forward: destructive the moment it runs
ALTER TABLE FeeAccount DROP COLUMN OldBalance;
-- Backward: the column returns, the data does not
Make migrations additive where you can. Add a column; do not drop one in the same release. Deploy code that writes to both old and new, then remove the old one a release later, once nothing reads it.
Every migration needs its rollback written and tested before the release, not improvised during an incident.
The database backup taken at step 4 is the real safety net. Verify it restores — a backup nobody has ever restored is a hope, not a plan.
Configuration and secrets in production
| Setting | Development | Production |
|---|---|---|
| Connection string | Local SQL Server | Production server, restricted login |
| Logging level | Debug | Information or Warning |
| Detailed errors | On | Off |
| CORS origins | localhost:5173 | The real domain only |
| Token lifetime | Long, for convenience | Short |
The developer exception page must never be on in production. It exposes stack traces, file paths and often the connection string to anyone who triggers an error.
Secrets live in a vault or in environment variables, never in a committed file. The application reads them at startup; nobody types them.
The application's SQL login is not your login. Yours is an administrator; the application's should have only what it needs. A missing GRANT that never appears in development is a classic production-only failure.
Logs, metrics and alerts
| Answers | Example | |
|---|---|---|
| Logs | What happened, in detail | "Fee calculation failed for student 12" |
| Metrics | How much, how often, how fast | Requests per minute; 95th-percentile response time |
| Alerts | Something needs attention now | Error rate above 1% for five minutes |
2026-08-27T14:32:10Z [ERROR] CorrelationId=8f14e45f
Fee calculation failed for student 12 in school 1
System.NullReferenceException at FeeService.CalculateBalance line 47
A correlation id ties every log line from one request together. A user reports a failure, gives you the id shown on the error page, and you retrieve exactly those lines out of millions. Without it, production debugging is guessing at timestamps.
Log the inputs. "Fee calculation failed" is unusable; "failed for student 12 in school 1" is a reproduction.
Never log a password, a token, a connection string or a parent's phone number. Logs are copied, shipped to third-party services and read by people without database access. Log the student id, not the personal data.
Alert on symptoms users feel, not on everything:
| Good alert | Bad alert |
|---|---|
| Error rate above 1% for five minutes | Any exception, ever |
| 95th-percentile response over three seconds | CPU above 50% |
| Sign-in failures spiking | A single 404 |
An alert nobody acts on trains everyone to ignore alerts, including the one that matters.
Incident response
An incident is a production problem affecting users.
| Severity | Means | Response |
|---|---|---|
| P1 | Everyone blocked; data at risk | Immediately, whatever the hour |
| P2 | A major feature broken; a workaround exists | Same day |
| P3 | A minor feature broken | Next sprint |
| P4 | Cosmetic | Backlog |
The sequence, in order:
1. Acknowledge. Someone owns it. Silence during an outage is worse than the outage.
2. Assess. Who is affected, how many, is data at risk?
3. Mitigate — restore service first. Turn off the feature flag, roll back, restart. Fixing the root cause comes second. A school whose portal is down does not care why.
4. Communicate. "We are aware of the issue with fee receipts and are working on it. Next update in 30 minutes." Then send the update, even if nothing has changed.
5. Fix. Once service is restored, find the actual cause.
6. Review. Afterwards, write up what happened.
Restoring service and finding the cause are different activities and must not be done in the wrong order. The instinct to debug first is strong and wrong.
Hotfixes
A hotfix is an urgent fix that ships without waiting for the next release.
1. Branch from what is in production — not from the development branch
2. Make the smallest possible change
3. Test it, including the case that broke
4. Review it — urgency is not an excuse to skip review
5. Deploy and verify
6. Merge back into the development branch
Step 1 is the one that goes wrong under pressure. Branching from the development branch drags in every unreleased change, so a one-line fix becomes an unplanned release of half-finished work.
Step 6 is the one that gets forgotten. Skip it and the next release reintroduces the bug you just fixed — a genuinely confusing failure. Track 16 covers both.
"Smallest possible change" means fixing the bug, not the surrounding code. Refactoring belongs in a normal release.
Post-incident review
Incident Fee receipts showed a zero balance for 40 minutes
Date 27 August 2026, 14:20–15:00
Severity P2 — receipts unusable; the office worked from the ledger
Users All office staff at 3 schools
Timeline
14:20 Release 2.4.0 deployed
14:32 First error logged
14:41 Office administrator reported it by phone
14:44 Acknowledged; investigation started
14:52 Cause identified — students with two fee accounts
14:58 Rolled back to 2.3.2
15:00 Verified; service restored
Cause FeeService selected an account with FirstOrDefault and no
academic-year filter. Students repeating a year have two
accounts; the older one was returned.
Why not caught Development data has one account per student. No test
covered a student with two.
Actions
1. Filter by SchoolId and AcademicYear, and use SingleOrDefault (NCA-147)
2. Add a test for a student with two fee accounts (NCA-148)
3. Add multi-account students to the QA data set (NCA-149)
4. Add an alert on fee-calculation errors (NCA-150)
Blameless. The question is what in the system allowed this, not who typed it. A review that assigns blame gets fewer incidents reported, not fewer incidents.
"Why was it not caught?" is the most valuable line, because its answer prevents a category of future incidents rather than one recurrence.
Every action gets a work-item number. A review producing "we should test better" changes nothing.
Where this goes wrong
| The mistake | Consequence |
|---|---|
| Using build, deployment and release as synonyms | Nobody can separate shipping code from switching a feature on |
| Deploying on a Friday afternoon | It breaks and the people who can fix it have gone |
| No database backup before a release | The only real undo is missing |
| Migrations after the application | New code queries a column that does not exist yet |
| Developer exception page on in production | Stack traces and connection strings shown to anyone |
| Logging a token or a parent's phone number | Personal data in a system with wider access |
| Debugging before restoring service | Users stay down while you investigate |
| Hotfix branched from the development branch | A one-line fix ships half-finished features |
| Hotfix never merged back | The bug returns in the next release |
Restore service first, find the cause second. The instinct to debug immediately is strong and wrong.
Common mistakes
- Using build, deployment and release as synonyms
- Rebuilding per environment
- Deploying on a Friday afternoon
- No database backup before a release
- Migrations after the application deployment
- A destructive migration with no tested rollback
- The developer exception page on in production
- Logging secrets or personal data
- No correlation id
- Alerting on everything until nobody reads alerts
- Debugging before restoring service
- Hotfix branched from the development branch
- Hotfix never merged back
- Post-incident review that assigns blame
Practice
- Define build, deployment and release, and give a School example of each.
- Describe how a feature flag lets you deploy without releasing.
- Assign version numbers to four changes — a bug fix, a new feature, a renamed API field, a security patch.
- Write the smoke test for the School portal.
- Write a deployment checklist and mark which steps are irreversible.
- Write an additive migration adding
IsDeleted, and its rollback. Say what the rollback loses. - Explain how to remove a column across two releases without data loss.
- List every configuration value that differs between development and production.
- Write a log line with a correlation id, then one without, and compare their usefulness during an incident.
- Take a log line containing a parent's phone number and rewrite it safely.
- Write three alerts worth waking someone for and three that are not.
- Assign P1–P4 to five School incidents.
- Write the incident timeline for a fee-receipt outage, in order, marking where service was restored versus where the cause was found.
- Write a hotfix procedure and mark the two steps most likely to be skipped under pressure.
- Write a blameless post-incident review for a bug you have caused, including "why was it not caught" and numbered actions.
Exercise 15 is uncomfortable and the most useful one here.
You can now
- Tell build, deployment and release apart
- Describe a safe deployment sequence and its irreversible steps
- Say why migrations run before the application deploys
- Explain what a feature flag separates
- Name what must never appear in a log
- Say what to do first when production breaks
- Write a blameless post-incident review
Review questions
- What is the difference between deployment and release, and what does separating them enable?
- Why must database migrations be applied before the application deployment?
- Why is restoring service before finding the cause the correct order?
- Why must a hotfix branch from production rather than from the development branch?
Next: Feature walkthrough