Skip to main content

Background Jobs: Fee Reminders and Receipts Without Waiting – Web API | Lesson 94

Published / updated

Introduction​

When the office takes a fee payment, the guardian should get a message: "payment received". Every evening, guardians with overdue fees should get a reminder. Both are useful, and neither should make anyone wait.

A text or email provider can take a second or more for each message. Three hundred reminders, sent one by one inside a request, would keep the office screen frozen for minutes. This lesson moves that work out of the request and into the background.

By the end of this lesson, you will be able to:

  • Explain what a hosted service and a BackgroundService are.
  • Hand work from a request to a background worker through a queue.
  • Use scoped services, such as repositories, inside a background service.
  • Run a job every day at a fixed time, and test it without waiting for that time.
  • Make a job safe to run twice, and stop one failure from stopping the whole API.

Why this matters​

Work inside a request has three problems when it is slow or depends on someone else's service.

The user waits. The clerk cannot serve the next parent until the message provider answers. On result day, when every school sends messages, providers get slow, and so does your fee counter.

A failure in the extra work fails the main work. If the provider is down, the payment was already saved, but the request answers 500. The clerk thinks it failed and takes the money again.

Nobody presses a button at six in the evening. Reminders must go out every day, on time, whether or not anyone is using the API. That needs something that runs by itself, on a clock.

The concept​

Think of the school canteen. The counter takes an order, writes a slip, and puts it on the kitchen's spike.

The counter then serves the next child at once. Meanwhile, the cook takes slips off the spike one by one and cooks.

The counter is the request, and the spike is a queue. The cook is a background worker: it lives as long as the canteen is open and works on whatever is waiting.

The lunch bell, which rings at 1 pm whether anyone asks or not, is a scheduled job.

Example: what runs where​

WorkWhere it runsWhy
Save the payment, return the receipt numberin the requestthe clerk needs the answer now
Send the "payment received" messagequeued, background workerthe clerk does not need to wait for it
Find overdue invoices at 18:00 every dayscheduled jobnobody asks; the clock does
Send each reminderqueued, background workerthe same worker as the receipts

A good test: if the user's next step does not depend on the work being finished, the work can go to the background.

Example: a hosted service in a few lines​

ASP.NET Core can run hosted services: classes it starts when the app starts and stops when the app stops. The easy way to write one is to inherit from BackgroundService and fill in ExecuteAsync:

A hosted service (not part of the SMS)
public class Heartbeat(ILogger<Heartbeat> logger) : BackgroundService
{
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
logger.LogInformation("Still running");
await Task.Delay(TimeSpan.FromMinutes(1), stoppingToken);
}
}
}

// Program.cs
builder.Services.AddHostedService<Heartbeat>();

ExecuteAsync runs in the background for the whole life of the app. stoppingToken is cancelled when the app shuts down, so Task.Delay ends early and the loop finishes. While it waits, no thread is used.

Example: a queue with Channel​

.NET has a ready-made, thread-safe queue for this: System.Threading.Channels. Writers add at the back; a reader waits at the front and wakes up as soon as something arrives.

A channel in four lines
Channel<string> channel = Channel.CreateBounded<string>(100);
await channel.Writer.WriteAsync("Remind guardian 21"); // a request adds work
await foreach (string job in channel.Reader.ReadAllAsync(stoppingToken)) // the worker waits for work
Console.WriteLine(job);

Bounded means at most 100 items wait at once. When it is full, a writer waits for room, so a flood of work cannot use up all the server's memory.

Example: when is the next run?​

A daily job needs one calculation: how long to sleep until the next 18:00.

NowNext runWhy
Monday 09:00Monday 18:00today's run is still ahead
Monday 18:00Tuesday 18:00today's run is happening now
Monday 22:30Tuesday 18:00today's run has passed

The job sleeps until the next run, does its work, and then works out the next run again.

Apply it to the SMS​

The plan has five parts: a table that remembers reminders, a queue, a worker that sends, a service that finds overdue invoices, and an evening job.

Step 1: remember every reminder​

database/094-fee-reminders.sql
-- Lesson 94: one row for every fee reminder the school sends.
-- The background job (and the office's "send now" button) records a reminder here before sending it.
-- UQ_FeeReminders_Invoice_Date allows one reminder per invoice per day, so two servers running the same
-- job, or the job and the button together, can never remind a guardian twice on one day.
-- CreatedBy is the office user who pressed the button; NULL means the evening job (no signed-in user).
-- Safe to run more than once.
USE SchoolDb;
GO

IF OBJECT_ID(N'dbo.FeeReminders', N'U') IS NULL
BEGIN
CREATE TABLE dbo.FeeReminders
(
Id INT IDENTITY(1, 1) NOT NULL CONSTRAINT PK_FeeReminders PRIMARY KEY,
FeeInvoiceId INT NOT NULL CONSTRAINT FK_FeeReminders_FeeInvoices REFERENCES dbo.FeeInvoices (Id),
ReminderDate DATE NOT NULL,
CreatedAt DATETIME2(0) NOT NULL CONSTRAINT DF_FeeReminders_CreatedAt DEFAULT SYSDATETIME(),
CreatedBy INT NULL CONSTRAINT DF_FeeReminders_CreatedBy DEFAULT (CAST(SESSION_CONTEXT(N'UserId') AS INT)),
CONSTRAINT UQ_FeeReminders_Invoice_Date UNIQUE (FeeInvoiceId, ReminderDate)
);
END
GO

-- Check: the table and its constraints.
SELECT name, type_desc
FROM sys.objects
WHERE parent_object_id = OBJECT_ID(N'dbo.FeeReminders')
ORDER BY type_desc, name;

-- Check: the invoices a reminder would go to today (overdue, not paid in full).
SELECT InvoiceNumber, DueDate, Status
FROM dbo.FeeInvoices
WHERE Status IN ('Unpaid', 'PartiallyPaid') AND DueDate < CAST(SYSDATETIME() AS DATE)
ORDER BY InvoiceNumber;
GO

The unique key is the important line: one reminder per invoice per day. Its job becomes clear in Step 6.

CreatedBy uses the same SESSION_CONTEXT default as Lesson 88. A reminder sent from the office button records the clerk; one from the evening job records NULL, because no one is signed in.

A correction: this script first set CreatedAt with SYSDATETIME(), the server's local time. Every other table uses SYSUTCDATETIME(), as Lesson 8 requires, so the course review caught it. database/097-fee-reminders-utc.sql moves the default to UTC and converts any rows already written.

Step 2: the message and the queue​

A queued message carries only the invoice number and the reason:

GuardianMessage.cs (the first lines)
public sealed record GuardianMessage(string InvoiceNumber, MessageReason Reason);

public enum MessageReason
{
PaymentReceived,
FeeReminder
}

The worker reads the guardian and the balance when it sends. So a reminder queued at 18:00 and sent a minute later shows the balance at 18:01, not an old copy.

src/SchoolApp.Api/BackgroundJobs/GuardianMessageQueue.cs
using System.Threading.Channels;
using SchoolApp.Core.Interfaces;
using SchoolApp.Core.Notifications;

namespace SchoolApp.Api.BackgroundJobs;

// Lesson 94: the queue, built on a Channel: a thread-safe line where writers add at the back and a reader
// waits at the front. Registered as a singleton, so every request and the worker share the same line.
// Bounded: at most 10,000 messages wait at once. When it is full, a writer waits for room (FullMode.Wait)
// instead of using more and more memory.
// It lives in memory: messages still waiting when the app stops are lost (see the lesson's "In real projects").
public sealed class GuardianMessageQueue : IGuardianMessageQueue
{
private readonly Channel<GuardianMessage> _channel = Channel.CreateBounded<GuardianMessage>(
new BoundedChannelOptions(10_000)
{
FullMode = BoundedChannelFullMode.Wait,
SingleReader = true // only GuardianMessageWorker reads
});

public ValueTask QueueAsync(GuardianMessage message, CancellationToken cancellationToken = default) =>
_channel.Writer.WriteAsync(message, cancellationToken);

public IAsyncEnumerable<GuardianMessage> ReadAllAsync(CancellationToken cancellationToken) =>
_channel.Reader.ReadAllAsync(cancellationToken);
}

The queue is a singleton: every request and the worker must share the same line. A queue registered as scoped would give each request its own empty line that nobody ever reads.

Step 3: the worker​

src/SchoolApp.Api/BackgroundJobs/GuardianMessageWorker.cs
using SchoolApp.Api.Logging;
using SchoolApp.Core.Interfaces;
using SchoolApp.Core.Notifications;
using SchoolApp.Core.Services;

namespace SchoolApp.Api.BackgroundJobs;

// Lesson 94: a hosted service. ASP.NET Core starts it with the app and stops it with the app; in between,
// ExecuteAsync runs in the background, outside every request. This one waits for queued messages and
// sends them one at a time.
public sealed class GuardianMessageWorker : BackgroundService
{
private readonly IGuardianMessageQueue _queue;
private readonly IServiceScopeFactory _scopes;
private readonly ILogger<GuardianMessageWorker> _logger;

// The worker lives as long as the app (like a singleton), so it cannot take scoped services such as a
// repository in its constructor. It asks for a scope factory and makes a fresh scope for each message.
public GuardianMessageWorker(IGuardianMessageQueue queue, IServiceScopeFactory scopes, ILogger<GuardianMessageWorker> logger)
{
_queue = queue;
_scopes = scopes;
_logger = logger;
}

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
// Waits (without using a thread) until a message arrives; ends when the app stops.
await foreach (GuardianMessage message in _queue.ReadAllAsync(stoppingToken))
{
try
{
await using AsyncServiceScope scope = _scopes.CreateAsyncScope();
GuardianMessageService messages = scope.ServiceProvider.GetRequiredService<GuardianMessageService>();
MessageResult result = await messages.SendAsync(message, stoppingToken);

if (result.Outcome == MessageOutcome.Sent)
{
SchoolLog.GuardianMessageSent(_logger, message.Reason, message.InvoiceNumber, result.GuardianId, result.Channel);
}
else
{
SchoolLog.GuardianMessageSkipped(_logger, message.Reason, message.InvoiceNumber, result.Outcome);
}
}
catch (Exception exception) when (!stoppingToken.IsCancellationRequested)
{
// One bad message must not stop the worker: log it and go on to the next one.
SchoolLog.GuardianMessageFailed(_logger, exception, message.Reason, message.InvoiceNumber);
}
}
}
}

Look at the constructor. It does not ask for a repository, because a hosted service is created once and lives as long as the app. Our repositories are scoped: one per request (Lesson 69).

If you put a scoped repository in the constructor, the app refuses to start. The real message is "Cannot consume scoped service ... from singleton 'Microsoft.Extensions.Hosting.IHostedService'."

Instead, the worker asks for IServiceScopeFactory and creates a fresh scope for each message, just like a new request would. When the scope ends, its database connection and services are cleaned up.

The try/catch around each message means one bad message is logged and skipped. The worker then carries on with the next one.

Step 4: sending one message​

src/SchoolApp.Core/Services/GuardianMessageService.cs
using SchoolApp.Core.Interfaces;
using SchoolApp.Core.Notifications;

namespace SchoolApp.Core.Services;

// Lesson 94: sends one queued message. The pieces come from earlier lessons: the notices from Lesson 40
// write the text, the factory from Lesson 56 picks email or phone, and the senders from Lesson 41 send it.
// (Those senders still only build the text. Using a real SMS or email provider means one new
// INotificationSender class; nothing here changes.)
public class GuardianMessageService
{
private readonly IFeeReminderRepository _reminders;

public GuardianMessageService(IFeeReminderRepository reminders)
{
_reminders = reminders;
}

public async Task<MessageResult> SendAsync(GuardianMessage message, CancellationToken cancellationToken = default)
{
FeeContact? contact = await _reminders.GetContactAsync(message.InvoiceNumber, cancellationToken);
if (contact is null)
{
return new MessageResult(MessageOutcome.NoPrimaryContact);
}

// Checked again now: the guardian may have paid between queuing and sending.
if (message.Reason == MessageReason.FeeReminder && contact.Balance <= 0)
{
return new MessageResult(MessageOutcome.NothingDue);
}

INotificationSender sender = NotificationSenderFactory.For(contact.Guardian);
FeeNotice notice = sender.Channel == "Email" ? new EmailFeeNotice() : new PhoneFeeNotice();
string text = notice.Compose(contact.Guardian, contact.StudentName, contact.Balance);
sender.Send(contact.Guardian, text);

return new MessageResult(MessageOutcome.Sent, contact.Guardian.Id, sender.Channel);
}
}

This is where earlier lessons meet:

  • FeeNotice from Lesson 40 writes the text.
  • NotificationSenderFactory from Lesson 56 chooses email or phone.
  • The senders from Lesson 41 send it.

Those senders still only build the text: the SMS has no real text-message account yet. Extra 8 connects one. It needs a new, async interface, because a call over the network must not block a thread.

The balance check matters. If a guardian pays in full after the reminder was queued but before it is sent, the reminder is skipped.

Step 5: a receipt after every payment​

FeeInvoicesController.cs, Pay (the Lesson 94 lines)
SchoolLog.PaymentRecorded(_logger, payment.ReceiptNumber, payment.Amount, invoiceNumber); // Lesson 90
// Lesson 94: the guardian's receipt message is sent in the background; the clerk does not wait for it.
// CancellationToken.None: the payment is saved, so the message must be queued even if the client has gone.
await _messages.QueueAsync(new GuardianMessage(invoiceNumber, MessageReason.PaymentReceived), CancellationToken.None);
return CreatedAtAction(nameof(Get), new { invoiceNumber },
ApiResponse.Ok(payment.ToDto(payment.FeeInvoice!), "Payment recorded."));

Queuing takes microseconds. The clerk's answer no longer depends on the message provider at all.

Step 6: finding overdue invoices, safely twice​

src/SchoolApp.Core/Services/FeeReminderService.cs
using SchoolApp.Core.Interfaces;
using SchoolApp.Core.Notifications;
using SchoolApp.Core.Rules;

namespace SchoolApp.Core.Services;

// Lesson 94: finds overdue invoices and queues one reminder for each. It does not send anything itself:
// sending is the background worker's job, so this finishes in moments even for thousands of invoices.
public class FeeReminderService
{
private readonly IFeeReminderRepository _reminders;
private readonly IGuardianMessageQueue _queue;

public FeeReminderService(IFeeReminderRepository reminders, IGuardianMessageQueue queue)
{
_reminders = reminders;
_queue = queue;
}

// Returns how many reminders were queued.
public async Task<int> QueueRemindersAsync(DateOnly today, CancellationToken cancellationToken = default)
{
IReadOnlyList<string> invoiceNumbers =
await _reminders.GetInvoicesToRemindAsync(today, SchoolRules.DaysBetweenFeeReminders, cancellationToken);

int queued = 0;
foreach (string invoiceNumber in invoiceNumbers)
{
// Record first, then queue. If someone else recorded this invoice today, skip it.
if (await _reminders.TryRecordAsync(invoiceNumber, today, cancellationToken))
{
// CancellationToken.None: once recorded, the reminder must reach the queue even if the caller gives up.
await _queue.QueueAsync(new GuardianMessage(invoiceNumber, MessageReason.FeeReminder), CancellationToken.None);
queued++;
}
}
return queued;
}
}

The query (in the repository below) picks invoices that are Unpaid or PartiallyPaid, past their due date, and not reminded in the last 7 days (SchoolRules.DaysBetweenFeeReminders).

Example: why "record first, then queue"​

Imagine the API runs on two servers, and both start the evening job at 18:00. Or the evening job runs while a clerk presses the office button. Both find the same overdue invoices.

Each tries to insert today's row for an invoice. The unique key lets exactly one succeed; the other gets error 2627, TryRecordAsync returns false, and that invoice is skipped. So each guardian gets one reminder, not two.

No lock is needed, and no "check first" query. The database is the referee.

Show src/SchoolApp.Data/Dapper/DapperFeeReminderRepository.cs
src/SchoolApp.Data/Dapper/DapperFeeReminderRepository.cs
using Dapper;
using Microsoft.Data.SqlClient;
using SchoolApp.Core.Entities;
using SchoolApp.Core.Interfaces;
using SchoolApp.Core.Notifications;

namespace SchoolApp.Data.Dapper;

// Lesson 94: the queries behind fee reminders and payment receipts.
public class DapperFeeReminderRepository : IFeeReminderRepository
{
private readonly SqlConnectionFactory _connections;

public DapperFeeReminderRepository(SqlConnectionFactory connections)
{
_connections = connections;
}

public async Task<IReadOnlyList<string>> GetInvoicesToRemindAsync(DateOnly today, int daysBetweenReminders,
CancellationToken cancellationToken = default)
{
await using SqlConnection connection = _connections.Create();
IEnumerable<string> invoiceNumbers = await connection.QueryAsync<string>(new CommandDefinition("""
SELECT i.InvoiceNumber
FROM dbo.FeeInvoices AS i
WHERE i.Status IN ('Unpaid', 'PartiallyPaid')
AND i.DueDate < @Today
AND NOT EXISTS (SELECT 1
FROM dbo.FeeReminders AS r
WHERE r.FeeInvoiceId = i.Id
AND r.ReminderDate > DATEADD(DAY, -@DaysBetween, @Today))
ORDER BY i.InvoiceNumber;
""", new { Today = today, DaysBetween = daysBetweenReminders }, cancellationToken: cancellationToken));
return invoiceNumbers.ToList();
}

// The unique key decides: if another server inserted today's row a moment earlier, this INSERT fails
// with error 2627 and the reminder is skipped. No locks, no checking first.
public async Task<bool> TryRecordAsync(string invoiceNumber, DateOnly reminderDate, CancellationToken cancellationToken = default)
{
await using SqlConnection connection = _connections.Create();
try
{
int rows = await connection.ExecuteAsync(new CommandDefinition("""
INSERT INTO dbo.FeeReminders (FeeInvoiceId, ReminderDate)
SELECT Id, @ReminderDate FROM dbo.FeeInvoices WHERE InvoiceNumber = @InvoiceNumber;
""", new
{
ReminderDate = reminderDate,
InvoiceNumber = new DbString { Value = invoiceNumber, IsAnsi = true, Length = 20 }
}, cancellationToken: cancellationToken));
return rows == 1;
}
catch (SqlException exception) when (SqlErrors.IsUniqueKeyViolation(exception, "UQ_FeeReminders_Invoice_Date"))
{
return false;
}
}

public async Task<FeeContact?> GetContactAsync(string invoiceNumber, CancellationToken cancellationToken = default)
{
await using SqlConnection connection = _connections.Create();
ContactRow? row = await connection.QuerySingleOrDefaultAsync<ContactRow>(new CommandDefinition("""
SELECT i.InvoiceNumber,
s.FirstName + ' ' + s.LastName AS StudentName,
i.TotalAmount - i.DiscountAmount - ISNULL(p.Paid, 0) AS Balance,
g.Id AS GuardianId, g.FirstName, g.LastName, g.Phone, g.Email
FROM dbo.FeeInvoices AS i
JOIN dbo.Enrollments AS e ON e.Id = i.EnrollmentId
JOIN dbo.Students AS s ON s.Id = e.StudentId
JOIN dbo.StudentGuardians AS sg ON sg.StudentId = s.Id AND sg.IsPrimaryContact = 1
JOIN dbo.Guardians AS g ON g.Id = sg.GuardianId AND g.IsDeleted = 0
OUTER APPLY (SELECT SUM(fp.Amount) AS Paid
FROM dbo.FeePayments AS fp
WHERE fp.FeeInvoiceId = i.Id AND fp.IsDeleted = 0) AS p
WHERE i.InvoiceNumber = @InvoiceNumber;
""", new { InvoiceNumber = new DbString { Value = invoiceNumber, IsAnsi = true, Length = 20 } },
cancellationToken: cancellationToken));

if (row is null)
{
return null;
}
Guardian guardian = new Guardian(row.FirstName, row.LastName, row.Phone) { Id = row.GuardianId, Email = row.Email };
return new FeeContact(row.InvoiceNumber, row.StudentName, row.Balance, guardian);
}

private sealed class ContactRow
{
public string InvoiceNumber { get; set; } = "";
public string StudentName { get; set; } = "";
public decimal Balance { get; set; }
public int GuardianId { get; set; }
public string FirstName { get; set; } = "";
public string LastName { get; set; } = "";
public string Phone { get; set; } = "";
public string? Email { get; set; }
}
}

Step 7: the evening job​

src/SchoolApp.Api/BackgroundJobs/FeeReminderJob.cs
using Microsoft.Extensions.Options;
using SchoolApp.Api.Logging;
using SchoolApp.Api.Options;
using SchoolApp.Core.Services;

namespace SchoolApp.Api.BackgroundJobs;

// Lesson 94: a scheduled job. Every day at RunAt (18:00), it queues a reminder for each overdue invoice.
// The clock is a TimeProvider (Lesson 79), so a test can move time forward instead of waiting until evening.
public sealed class FeeReminderJob : BackgroundService
{
private readonly IServiceScopeFactory _scopes;
private readonly TimeProvider _time;
private readonly FeeReminderOptions _options;
private readonly ILogger<FeeReminderJob> _logger;

public FeeReminderJob(IServiceScopeFactory scopes, TimeProvider time, IOptions<FeeReminderOptions> options,
ILogger<FeeReminderJob> logger)
{
_scopes = scopes;
_time = time;
_options = options.Value;
_logger = logger;
}

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
if (!_options.Enabled)
{
SchoolLog.FeeReminderJobOff(_logger);
return;
}

while (!stoppingToken.IsCancellationRequested)
{
DateTimeOffset now = _time.GetLocalNow();
DateTimeOffset nextRun = NextRun(now, _options.RunAt);

// Sleeps until the run time. No thread is used while waiting, and stopping the app ends the wait.
// The wait starts before the log line, so anyone who reads "next run at" (a test, too) knows the
// timer is already set.
Task untilNextRun = Task.Delay(nextRun - now, _time, stoppingToken);
SchoolLog.FeeReminderJobScheduled(_logger, nextRun);
await untilNextRun;
await RunOnceAsync(DateOnly.FromDateTime(nextRun.DateTime), stoppingToken);
}
}

private async Task RunOnceAsync(DateOnly today, CancellationToken stoppingToken)
{
try
{
await using AsyncServiceScope scope = _scopes.CreateAsyncScope();
FeeReminderService reminders = scope.ServiceProvider.GetRequiredService<FeeReminderService>();
int queued = await reminders.QueueRemindersAsync(today, stoppingToken);
SchoolLog.FeeRemindersQueued(_logger, queued, today);
}
catch (Exception exception) when (!stoppingToken.IsCancellationRequested)
{
// The database may be down for a moment. Without this catch, the exception would stop the whole app
// (the default since .NET 6). Log it; tomorrow's run finds every invoice that is still overdue.
SchoolLog.FeeReminderJobFailed(_logger, exception, today);
}
}

// Today at runAt if that is still ahead, otherwise tomorrow at runAt.
public static DateTimeOffset NextRun(DateTimeOffset now, TimeOnly runAt)
{
DateTimeOffset todayAtRunAt = new DateTimeOffset(now.Date.Add(runAt.ToTimeSpan()), now.Offset);
return todayAtRunAt > now ? todayAtRunAt : todayAtRunAt.AddDays(1);
}
}

The job never reads DateTime.Now. It asks TimeProvider, the same clock as Lesson 79, so a test can replace it with a clock that jumps forward a whole day in no time.

Task.Delay(nextRun - now, _time, stoppingToken) sleeps until 18:00 without holding a thread. When the app stops, the delay ends at once and the loop finishes.

Example: what a missing catch does​

Before .NET 6, an exception in a background service failed silently: the job just stopped, and nobody noticed. Since .NET 6, the default is the opposite. We checked it with a service that throws after 1.5 seconds:

Output
fail: Microsoft.Extensions.Hosting.Internal.Host[9]
BackgroundService failed
System.InvalidOperationException: database busy
crit: Microsoft.Extensions.Hosting.Internal.Host[10]
The HostOptions.BackgroundServiceExceptionBehavior is configured to StopHost. A BackgroundService has
thrown an unhandled exception, and the IHost instance is stopping.
info: Microsoft.Hosting.Lifetime[0]
Application is shutting down...

The whole API stopped, fee counter included, because the reminder job had a problem. That is why RunOnceAsync catches everything except shutdown, logs it, and tries again the next evening.

Step 8: register it all​

Program.cs (the Lesson 94 lines)
// Lesson 94: work that runs in the background, outside any request.
builder.Services.AddOptions<FeeReminderOptions>()
.BindConfiguration(FeeReminderOptions.SectionName)
.ValidateOnStart();
builder.Services.AddSingleton<IGuardianMessageQueue, GuardianMessageQueue>(); // one queue for the whole app
builder.Services.AddScoped<GuardianMessageService>();
builder.Services.AddScoped<FeeReminderService>();
builder.Services.AddHostedService<GuardianMessageWorker>(); // sends whatever is queued
builder.Services.AddHostedService<FeeReminderJob>(); // every day at FeeReminders:RunAt
appsettings.json (the Lesson 94 section)
"FeeReminders": {
"Enabled": true,
"RunAt": "18:00"
}

RunAt is the server's local time. A school that closes later can change it without touching code.

Step 9: the office button, and a real run​

The office sometimes wants the reminders now, not at 18:00. POST /api/fee-reminders calls the same FeeReminderService and answers 202 Accepted: "the work is accepted and will be done", not "it is done".

Show src/SchoolApp.Api/Controllers/FeeRemindersController.cs
src/SchoolApp.Api/Controllers/FeeRemindersController.cs
using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Mvc;
using SchoolApp.Api.Models;
using SchoolApp.Api.Security;
using SchoolApp.Core.Dtos;
using SchoolApp.Core.Services;

namespace SchoolApp.Api.Controllers;

// Lesson 94: the office can send today's fee reminders now, instead of waiting for the evening job.
// The request only queues them; the background worker sends them. So the answer is 202 Accepted:
// "the work is accepted and will be done", not 200 "the work is done".
[ApiController]
[Authorize(Policy = Policies.Office)]
[Route("api/fee-reminders")]
[Produces("application/json")]
[ProducesErrorResponseType(typeof(ApiResponse))]
public class FeeRemindersController : ControllerBase
{
private readonly FeeReminderService _reminders;
private readonly TimeProvider _time;

public FeeRemindersController(FeeReminderService reminders, TimeProvider time)
{
_reminders = reminders;
_time = time;
}

/// <summary>Queues today's fee reminders for every overdue invoice; they are sent in the background.</summary>
/// <remarks>An invoice gets at most one reminder a week, whether from this button or the evening job.</remarks>
/// <param name="cancellationToken">Cancelled when the client gives up.</param>
/// <response code="202">The reminders were queued; the answer says how many.</response>
[HttpPost]
[ProducesResponseType(StatusCodes.Status202Accepted)]
public async Task<ActionResult<ApiResponse<FeeRemindersQueuedDto>>> Send(CancellationToken cancellationToken)
{
DateOnly today = DateOnly.FromDateTime(_time.GetLocalNow().DateTime);
int queued = await _reminders.QueueRemindersAsync(today, cancellationToken);
return Accepted(ApiResponse.Ok(new FeeRemindersQueuedDto(today, queued),
$"{queued} fee reminders queued. They are sent in the background."));
}
}

When the API started, the job logged its plan. Then we signed in as a teacher and as the accounts user and pressed the button:

Output
FeeReminderJob[3004] Fee reminders: next run at 10/10/2026 18:00:00 +05:30

POST /api/fee-reminders (teacher) 403
POST /api/fee-reminders (accounts) 202
{"status":"success","message":"10 fee reminders queued. They are sent in the background.","data":{"date":"2026-10-10","queued":10}}
POST /api/fee-reminders (accounts) 202
{"status":"success","message":"0 fee reminders queued. They are sent in the background.","data":{"date":"2026-10-10","queued":0}}

GuardianMessageWorker[3001] FeeReminder message for invoice INV-2026-000002 sent to guardian 3 by Phone
GuardianMessageWorker[3001] FeeReminder message for invoice INV-2026-000003 sent to guardian 5 by Phone
...
GuardianMessageWorker[3001] FeeReminder message for invoice INV-2026-000012 sent to guardian 21 by Phone

Ten overdue invoices got ten reminders. The second press found them already reminded this week and queued none. FeeReminders held ten rows with CreatedBy = 4, the accounts user.

Then a payment of 5,000 on Kiara's invoice:

Output
POST /api/fee-invoices/INV-2026-000012/payments 201 X-Response-Time-ms: 155
FeeInvoicesController[1001] Payment RCP-2026-000075 of 5000 recorded on invoice INV-2026-000012
GuardianMessageWorker[3001] PaymentReceived message for invoice INV-2026-000012 sent to guardian 21 by Phone

The log shows invoice numbers and guardian ids only, never phone numbers (the Lesson 90 rule). Afterwards, the test payment and the reminder rows were removed, so the sample data is unchanged.

Step 10: tests that do not wait until evening​

The unit tests use FakeTimeProvider (package Microsoft.Extensions.TimeProvider.Testing). The test sets the clock to 17:59, starts the job, and moves the clock one minute. The job runs at once.

Show tests/SchoolApp.Tests/Unit/FeeReminderTests.cs
tests/SchoolApp.Tests/Unit/FeeReminderTests.cs
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Time.Testing;
using SchoolApp.Api.BackgroundJobs;
using SchoolApp.Api.Options;
using SchoolApp.Core.Entities;
using SchoolApp.Core.Interfaces;
using SchoolApp.Core.Notifications;
using SchoolApp.Core.Services;
using SchoolApp.Tests.Api;
using SchoolApp.Tests.Fakes;

namespace SchoolApp.Tests.Unit;

// Lesson 94: the fee reminder job, the reminder service, and the message service, without a database.
public class FeeReminderTests
{
private static readonly TimeOnly SixPm = new TimeOnly(18, 0);

[Theory]
[InlineData("2026-10-12T09:00:00+05:30", "2026-10-12T18:00:00+05:30")] // morning: this evening
[InlineData("2026-10-12T18:00:00+05:30", "2026-10-13T18:00:00+05:30")] // exactly 18:00: it has just run, so tomorrow
[InlineData("2026-10-12T22:30:00+05:30", "2026-10-13T18:00:00+05:30")] // late evening: tomorrow
public void NextRun_IsTheNext1800(string now, string expected)
{
Assert.Equal(DateTimeOffset.Parse(expected), FeeReminderJob.NextRun(DateTimeOffset.Parse(now), SixPm));
}

// The job with a fake clock: one minute of fake time passes in no time at all.
private static (FeeReminderJob Job, FakeTimeProvider Time, ListLoggerProvider Logs, ServiceProvider Services) CreateJob(
FakeFeeReminderRepository repository, bool enabled = true)
{
FakeTimeProvider time = new FakeTimeProvider(new DateTimeOffset(2026, 10, 12, 17, 59, 0, TimeSpan.Zero));
time.SetLocalTimeZone(TimeZoneInfo.Utc);

ServiceCollection services = new ServiceCollection();
services.AddSingleton<IFeeReminderRepository>(repository);
services.AddSingleton<IGuardianMessageQueue, FakeGuardianMessageQueue>();
services.AddScoped<FeeReminderService>();
ServiceProvider provider = services.BuildServiceProvider();

ListLoggerProvider logs = new ListLoggerProvider();
ILogger<FeeReminderJob> logger = LoggerFactory.Create(logging => logging.AddProvider(logs)).CreateLogger<FeeReminderJob>();
FeeReminderJob job = new FeeReminderJob(provider.GetRequiredService<IServiceScopeFactory>(), time,
Microsoft.Extensions.Options.Options.Create(new FeeReminderOptions { Enabled = enabled, RunAt = SixPm }), logger);
return (job, time, logs, provider);
}

[Fact]
public async Task Job_WaitsUntil1800_ThenQueuesReminders()
{
FakeFeeReminderRepository repository = new FakeFeeReminderRepository();
repository.Overdue.AddRange(["INV-1", "INV-2"]);
(FeeReminderJob job, FakeTimeProvider time, ListLoggerProvider logs, ServiceProvider services) = CreateJob(repository);
await using (services)
{
await job.StartAsync(CancellationToken.None);
await logs.WaitForAsync(3004); // the job has set its timer for 18:00
Assert.False(repository.Asked.Reader.TryRead(out _)); // 17:59: nothing yet

time.Advance(TimeSpan.FromMinutes(1)); // 18:00

DateOnly asked = await repository.Asked.Reader.ReadAsync().AsTask().WaitAsync(TimeSpan.FromSeconds(5));
Assert.Equal(new DateOnly(2026, 10, 12), asked);
var queued = Assert.Single(await logs.WaitForAsync(3005));
Assert.StartsWith("Fee reminders: 2 queued for ", queued.Message);
await job.StopAsync(CancellationToken.None);
}
}

[Fact]
public async Task Job_SurvivesAFailedRun()
{
FakeFeeReminderRepository repository = new FakeFeeReminderRepository { FailWith = new TimeoutException("database busy") };
(FeeReminderJob job, FakeTimeProvider time, ListLoggerProvider logs, ServiceProvider services) = CreateJob(repository);
await using (services)
{
await job.StartAsync(CancellationToken.None);
await logs.WaitForAsync(3004);
time.Advance(TimeSpan.FromMinutes(1));

Assert.Single(await logs.WaitForAsync(3006)); // the failure is logged...
Assert.Equal(2, (await logs.WaitForAsync(3004, 2)).Count); // ...and tomorrow's run is scheduled
Assert.False(job.ExecuteTask!.IsCompleted); // the job is still running
await job.StopAsync(CancellationToken.None);
}
}

[Fact]
public async Task Job_SwitchedOff_NeverRuns()
{
FakeFeeReminderRepository repository = new FakeFeeReminderRepository();
(FeeReminderJob job, FakeTimeProvider time, ListLoggerProvider logs, ServiceProvider services) = CreateJob(repository, enabled: false);
await using (services)
{
await job.StartAsync(CancellationToken.None);
Assert.Single(await logs.WaitForAsync(3007));
time.Advance(TimeSpan.FromDays(2));

Assert.False(repository.Asked.Reader.TryRead(out _));
await job.StopAsync(CancellationToken.None);
}
}

[Fact]
public async Task QueueReminders_SkipsInvoicesAlreadyRemindedToday()
{
FakeFeeReminderRepository repository = new FakeFeeReminderRepository();
repository.Overdue.AddRange(["INV-1", "INV-2", "INV-3"]);
repository.AlreadyRecorded.Add("INV-2"); // another server recorded it a moment ago
FakeGuardianMessageQueue queue = new FakeGuardianMessageQueue();

int queued = await new FeeReminderService(repository, queue).QueueRemindersAsync(new DateOnly(2026, 10, 12));

Assert.Equal(2, queued);
Assert.Equal(["INV-1", "INV-3"], queue.Messages.Select(m => m.InvoiceNumber));
Assert.All(queue.Messages, m => Assert.Equal(MessageReason.FeeReminder, m.Reason));
}

private static FeeContact Contact(string invoiceNumber, decimal balance, string? email = null) =>
new FeeContact(invoiceNumber, "Kiara Thomas", balance,
new Guardian("Joseph", "Thomas", "9800000021") { Id = 21, Email = email });

[Theory]
[InlineData(null, "Phone")]
[InlineData("joseph@example.test", "Email")]
public async Task Send_PicksTheChannelFromTheGuardian(string? email, string expectedChannel)
{
FakeFeeReminderRepository repository = new FakeFeeReminderRepository();
repository.Contacts["INV-12"] = Contact("INV-12", 39500m, email);

MessageResult result = await new GuardianMessageService(repository)
.SendAsync(new GuardianMessage("INV-12", MessageReason.FeeReminder));

Assert.Equal(new MessageResult(MessageOutcome.Sent, 21, expectedChannel), result);
}

[Fact]
public async Task Send_PaidMeanwhile_SkipsTheReminder_ButStillSendsTheReceipt()
{
FakeFeeReminderRepository repository = new FakeFeeReminderRepository();
repository.Contacts["INV-12"] = Contact("INV-12", 0m);
GuardianMessageService messages = new GuardianMessageService(repository);

MessageResult reminder = await messages.SendAsync(new GuardianMessage("INV-12", MessageReason.FeeReminder));
MessageResult receipt = await messages.SendAsync(new GuardianMessage("INV-12", MessageReason.PaymentReceived));

Assert.Equal(MessageOutcome.NothingDue, reminder.Outcome);
Assert.Equal(MessageOutcome.Sent, receipt.Outcome);
}

[Fact]
public async Task Send_NoPrimaryContact_IsSkipped()
{
MessageResult result = await new GuardianMessageService(new FakeFeeReminderRepository())
.SendAsync(new GuardianMessage("INV-99", MessageReason.PaymentReceived));

Assert.Equal(MessageOutcome.NoPrimaryContact, result.Outcome);
}
}

The first version of these tests failed: the job had not set its timer yet when the test moved the clock. In .NET 10, all of ExecuteAsync runs in the background, even the part before the first await.

The fix was to start Task.Delay before writing the "next run at" log line. Now the test waits for that line before moving the clock, so the timer is always set by then.

The API tests run the real worker against SchoolDb. For reminders, they fix "today" at 2026-07-15, after the sample invoices' due date. That gives exactly ten reminders, whatever the real date is.

Show tests/SchoolApp.Tests/Api/BackgroundJobTests.cs
tests/SchoolApp.Tests/Api/BackgroundJobTests.cs
using System.Net;
using System.Net.Http.Headers;
using System.Net.Http.Json;
using System.Text.Json;
using Dapper;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.AspNetCore.TestHost;
using Microsoft.Data.SqlClient;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using SchoolApp.Api.Security;
using SchoolApp.Data;
using SchoolApp.Tests.Fakes;

namespace SchoolApp.Tests.Api;

// Lesson 94: fee reminders and payment receipts, sent by the background worker of a real API in memory.
[Collection("Api")]
public class BackgroundJobTests
{
private static readonly SqlConnectionFactory Connections = SqlConnectionFactory.FromEnvironment();

// A fixed "today" after the sample invoices' due date (2026-06-30). The ten overdue sample invoices
// are exactly the ones a reminder goes to, whatever the real date is when the tests run.
private static readonly DateOnly ReminderDay = new DateOnly(2026, 7, 15);
private const string KiaraInvoice = "INV-2026-000012"; // unpaid, 39,500; primary contact: guardian 21

private readonly ApiFactory _factory;

public BackgroundJobTests(ApiFactory factory)
{
_factory = factory;
}

// The API with its log captured and, for reminders, the clock fixed at ReminderDay.
private async Task<(WebApplicationFactory<Program> App, HttpClient Client)> StartAsync(ListLoggerProvider logs, string role,
bool fixTheClock = false)
{
WebApplicationFactory<Program> app = _factory.WithWebHostBuilder(builder =>
{
builder.ConfigureLogging(logging => logging.AddProvider(logs));
if (fixTheClock)
{
builder.ConfigureTestServices(services => services.AddSingleton<TimeProvider>(
new FixedTimeProvider(new DateTimeOffset(ReminderDay, new TimeOnly(12, 0), TimeSpan.Zero))));
}
});
// The token comes from the normal API. A token made on the fixed clock would already be expired by the
// real one, which is what the token check uses.
HttpClient client = app.CreateClient();
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer",
await ApiFactory.GetTokenAsync(_factory.CreateClient(), ApiFactory.UserNameFor(role)));
return (app, client);
}

private static async Task<JsonElement> DataAsync(HttpResponseMessage response) =>
(await response.Content.ReadFromJsonAsync<JsonElement>()).GetProperty("data");

[Fact]
public async Task Office_QueuesReminders_TheWorkerSendsThem_OncePerWeek()
{
ListLoggerProvider logs = new ListLoggerProvider();
(WebApplicationFactory<Program> app, HttpClient client) = await StartAsync(logs, Roles.Accountant, fixTheClock: true);
await using SqlConnection connection = Connections.Create();
try
{
HttpResponseMessage first = await client.PostAsync("/api/fee-reminders", null);
HttpResponseMessage second = await client.PostAsync("/api/fee-reminders", null);

Assert.Equal(HttpStatusCode.Accepted, first.StatusCode);
JsonElement firstData = await DataAsync(first);
Assert.Equal((10, "2026-07-15"), (firstData.GetProperty("queued").GetInt32(), firstData.GetProperty("date").GetString()));
Assert.Equal(0, (await DataAsync(second)).GetProperty("queued").GetInt32()); // already reminded this week

// The worker sends them a moment later. The log has invoice and guardian ids, never phone numbers.
var sent = await logs.WaitForAsync(3001, 10);
Assert.Equal(10, sent.Count);
Assert.Contains(sent, e => e.Message == $"FeeReminder message for invoice {KiaraInvoice} sent to guardian 21 by Phone");

// One row per invoice, recorded by the accountant (Lesson 88 fills CreatedBy).
(int Rows, int Users, string? UserName) recorded = await connection.QuerySingleAsync<(int, int, string?)>("""
SELECT COUNT(*), COUNT(DISTINCT r.CreatedBy), MAX(u.UserName)
FROM dbo.FeeReminders AS r LEFT JOIN dbo.Users AS u ON u.Id = r.CreatedBy
WHERE r.ReminderDate = @ReminderDay;
""", new { ReminderDay });
Assert.Equal((10, 1, "test.accountant"), recorded);
}
finally
{
await connection.ExecuteAsync("DELETE FROM dbo.FeeReminders WHERE ReminderDate = @ReminderDay;", new { ReminderDay });
await app.DisposeAsync();
}
}

[Fact]
public async Task Teacher_CannotSendReminders()
{
HttpClient teacher = await _factory.CreateSignedInClientAsync(Roles.Teacher);

HttpResponseMessage response = await teacher.PostAsync("/api/fee-reminders", null);

Assert.Equal(HttpStatusCode.Forbidden, response.StatusCode);
}

[Fact]
public async Task Payment_AnswersAtOnce_AndTheReceiptFollows()
{
ListLoggerProvider logs = new ListLoggerProvider();
(WebApplicationFactory<Program> app, HttpClient client) = await StartAsync(logs, Roles.Accountant);
await using SqlConnection connection = Connections.Create();
(DateTime? UpdatedAt, int? UpdatedBy) before = await connection.QuerySingleAsync<(DateTime?, int?)>(
"SELECT UpdatedAt, UpdatedBy FROM dbo.FeeInvoices WHERE InvoiceNumber = @KiaraInvoice;", new { KiaraInvoice });
string? receipt = null;
try
{
HttpResponseMessage response = await client.PostAsJsonAsync($"/api/fee-invoices/{KiaraInvoice}/payments",
new { amount = 1, paymentMode = "Cash" });

Assert.Equal(HttpStatusCode.Created, response.StatusCode);
receipt = (await DataAsync(response)).GetProperty("receiptNumber").GetString();
var sent = Assert.Single(await logs.WaitForAsync(3001));
Assert.Equal($"PaymentReceived message for invoice {KiaraInvoice} sent to guardian 21 by Phone", sent.Message);
}
finally
{
// Put the invoice back as it was: the test payment removed, status and update columns restored.
await connection.ExecuteAsync("""
DELETE FROM dbo.FeePayments WHERE ReceiptNumber = @Receipt;
UPDATE dbo.FeeInvoices SET Status = 'Unpaid', UpdatedAt = @UpdatedAt, UpdatedBy = @UpdatedBy
WHERE InvoiceNumber = @KiaraInvoice;
""", new { Receipt = receipt ?? "", before.UpdatedAt, before.UpdatedBy, KiaraInvoice });
await app.DisposeAsync();
}
}
}

One more surprise: with the clock fixed in July, sign-in failed with 401. The token was made on the July clock, but the token check uses the real clock, so the token looked expired.

The test now gets its token from the normal API. The suite has 161 tests, all passing.

Example: what is lost when the API stops​

The queue lives in memory. If the API stops while messages are waiting, those messages are lost.

For the SMS, that is acceptable. A lost receipt message is a small thing (the paper receipt is the real record). A lost reminder means the guardian is reminded a week later.

When every message matters, use the outbox pattern: save the message in a table, in the same transaction as the payment. A worker reads unsent rows, sends them, and marks them sent, so nothing is lost after a restart.

For many jobs with retries and a dashboard, teams use a library such as Hangfire or Quartz.NET. They store jobs in a database and make sure only one server runs each one.

Common mistakes​

  1. Slow work inside the request. The user waits, and a provider failure fails the request. Fix: queue it; answer at once.
  2. Task.Run in a controller ("fire and forget"). The request's scoped services are disposed when the response is sent, and nobody sees the errors. Fix: a queue and a hosted service.
  3. A scoped repository in a hosted service's constructor. The app refuses to start. Fix: IServiceScopeFactory and a new scope for each piece of work.
  4. No try/catch in ExecuteAsync. One error stops the whole API. Fix: catch, log, and carry on.
  5. Assuming the job runs once. Two servers, or a button and a job, double the messages. Fix: a unique key that lets only one record win.
  6. DateTime.Now in a scheduled job. You cannot test it without waiting. Fix: TimeProvider.

In real projects​

From 16 years of .NET projects

A school's fee API sent the "payment received" text message inside the payment request. On result day, the text provider took eight seconds to answer, and the office screen timed out after five.

The payment had been saved, but the clerk saw an error and took it again. By evening there were 23 double payments to refund, and a long queue of angry parents.

We moved the message to a background queue, so the payment answered in a fraction of a second. Later, we added an outbox table, so even a restart in the middle of the day lost no message.

Best practices​

  • Keep requests short: do the work the user needs, queue the rest.
  • Register the queue as a singleton, and create a new scope for each piece of background work.
  • Catch and log every error inside a background loop; never let one stop the app.
  • Make jobs safe to run twice, with a unique key or a lock.
  • Read the time from TimeProvider, and test schedules with FakeTimeProvider.
  • Log ids, not phone numbers or emails.

Recap​

  • A hosted service starts and stops with the app; BackgroundService.ExecuteAsync runs in the background.
  • A bounded Channel passes work from requests to one worker, which sends receipts and reminders.
  • Background services make a new scope for scoped services such as repositories.
  • FeeReminderJob runs daily at 18:00 on TimeProvider; FeeReminders and its unique key stop double reminders.
  • An unhandled exception in a background service stops the whole API, so every run is caught and logged.

Homework​

Library module task: Add background work to a single-file library API.

  1. POST /loans/{id}/return marks the loan returned and queues a "thank you" message. It answers at once.
  2. A worker sends queued messages. Sending is simulated: it takes one second and writes a log line.
  3. A job runs every day at 09:00 and queues a reminder for each overdue, unreturned loan, at most one per loan per day. POST /reminders runs the same check now and answers 202.

A correct result looks like:

Output
POST /loans/1/return 200 (answers in a fraction of a second)
POST /reminders 202 {"queued":1}
POST /reminders 202 {"queued":0}
Overdue check: next run at ... 09:00:00
Sent: Thank you, Aarav, for returning "Malgudi Days".
Sent: Reminder for Diya: "Wings of Fire" was due on 2026-10-09.
Show the solution
library-assignment/094-library-reminders.cs
// Library homework, Lesson 94: background work for the library.
// 1. Returning a book answers at once; the "thank you" message is queued and sent by a background worker.
// 2. Every day at 09:00 a scheduled job queues a reminder for each overdue loan (at most one per loan per day).
// POST /reminders runs the same check now, so you can try it without waiting for the morning.
// Sending a message is simulated and takes one second, to show that no request waits for it.
// Run with: dotnet run library-assignment/094-library-reminders.cs
#:sdk Microsoft.NET.Sdk.Web
#:property PublishAot=false
using System.Threading.Channels;

WebApplicationBuilder builder = WebApplication.CreateBuilder(args);
builder.Services.AddSingleton(TimeProvider.System);
builder.Services.AddSingleton<Library>();
builder.Services.AddSingleton<MessageQueue>();
builder.Services.AddHostedService<MessageWorker>();
builder.Services.AddHostedService<OverdueJob>();
WebApplication app = builder.Build();

app.MapGet("/loans", (Library library) => library.Loans);

app.MapPost("/loans/{id:int}/return", async (int id, Library library, MessageQueue queue) =>
{
Loan? loan = library.Loans.SingleOrDefault(l => l.Id == id);
if (loan is null || loan.Returned)
{
return Results.NotFound();
}
loan.Returned = true;
await queue.QueueAsync($"Thank you, {loan.Member}, for returning \"{loan.Title}\".");
return Results.Ok(loan); // the message is sent later, by the worker
});

app.MapPost("/reminders", async (Library library, MessageQueue queue, TimeProvider time) =>
{
int queued = await library.QueueRemindersAsync(DateOnly.FromDateTime(time.GetLocalNow().DateTime), queue);
return Results.Accepted(value: new { queued });
});

app.Run("http://localhost:5094");

public class Loan
{
public int Id { get; init; }
public string Title { get; init; } = "";
public string Member { get; init; } = "";
public DateOnly DueDate { get; init; }
public bool Returned { get; set; }
public DateOnly? LastReminded { get; set; }
}

public class Library(TimeProvider time)
{
private readonly Lock _lock = new();
private static DateOnly Today(TimeProvider time) => DateOnly.FromDateTime(time.GetLocalNow().DateTime);

public List<Loan> Loans { get; } =
[
new Loan { Id = 1, Title = "Malgudi Days", Member = "Aarav", DueDate = Today(time).AddDays(-3) },
new Loan { Id = 2, Title = "Wings of Fire", Member = "Diya", DueDate = Today(time).AddDays(-1) },
new Loan { Id = 3, Title = "The Jungle Book", Member = "Kabir", DueDate = Today(time).AddDays(7) }
];

// The job and POST /reminders may run at the same moment, so the check-and-mark happens under a lock:
// the in-memory version of the SMS's unique key on FeeReminders.
public async Task<int> QueueRemindersAsync(DateOnly today, MessageQueue queue)
{
List<Loan> due;
lock (_lock)
{
due = Loans.Where(l => !l.Returned && l.DueDate < today && l.LastReminded != today).ToList();
due.ForEach(l => l.LastReminded = today);
}
foreach (Loan loan in due)
{
await queue.QueueAsync($"Reminder for {loan.Member}: \"{loan.Title}\" was due on {loan.DueDate:yyyy-MM-dd}.");
}
return due.Count;
}
}

public class MessageQueue
{
private readonly Channel<string> _channel = Channel.CreateBounded<string>(100);

public ValueTask QueueAsync(string message) => _channel.Writer.WriteAsync(message);

public IAsyncEnumerable<string> ReadAllAsync(CancellationToken cancellationToken) => _channel.Reader.ReadAllAsync(cancellationToken);
}

public class MessageWorker(MessageQueue queue, ILogger<MessageWorker> logger) : BackgroundService
{
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
await foreach (string message in queue.ReadAllAsync(stoppingToken))
{
try
{
await Task.Delay(TimeSpan.FromSeconds(1), stoppingToken); // a slow SMS or email provider
logger.LogInformation("Sent: {Message}", message);
}
catch (Exception exception) when (!stoppingToken.IsCancellationRequested)
{
logger.LogError(exception, "Could not send a message");
}
}
}
}

public class OverdueJob(Library library, MessageQueue queue, TimeProvider time, ILogger<OverdueJob> logger) : BackgroundService
{
private static readonly TimeOnly RunAt = new TimeOnly(9, 0);

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
DateTimeOffset now = time.GetLocalNow();
DateTimeOffset todayAtRunAt = new DateTimeOffset(now.Date.Add(RunAt.ToTimeSpan()), now.Offset);
DateTimeOffset nextRun = todayAtRunAt > now ? todayAtRunAt : todayAtRunAt.AddDays(1);
logger.LogInformation("Overdue check: next run at {NextRun}", nextRun);
await Task.Delay(nextRun - now, time, stoppingToken);
try
{
int queued = await library.QueueRemindersAsync(DateOnly.FromDateTime(nextRun.DateTime), queue);
logger.LogInformation("Overdue check: {Queued} reminders queued", queued);
}
catch (Exception exception) when (!stoppingToken.IsCancellationRequested)
{
logger.LogError(exception, "Overdue check failed; it runs again tomorrow");
}
}
}
}

The library's lock plays the part of the SMS's unique key: whether the job or the button gets there first, a loan is reminded once a day.

Quiz​

  1. Why does the payment endpoint queue the receipt message instead of sending it?

    1. Queues are more secure
    2. So the clerk gets the answer at once, and a slow or failing provider cannot fail the payment
    3. Messages must be sent at night
    4. Controllers cannot send messages
    Show answer

    Answer: B. So the clerk gets the answer at once, and a slow or failing provider cannot fail the payment The payment is the work the clerk needs; the message can follow a moment later.

  2. Why does GuardianMessageWorker take IServiceScopeFactory instead of a repository?

    1. Repositories are slow
    2. The worker lives as long as the app, and repositories are scoped, so it makes a new scope for each message
    3. To save memory
    4. Repositories cannot be used in the background
    Show answer

    Answer: B. The worker lives as long as the app, and repositories are scoped, so it makes a new scope for each message A scoped service in a hosted service's constructor stops the app from starting.

  3. Two servers run the evening job at 18:00. What stops guardians getting two reminders?

    1. A lock in C#
    2. The unique key on FeeReminders (invoice, date): only one insert succeeds
    3. The queue
    4. Nothing
    Show answer

    Answer: B. The unique key on FeeReminders (invoice, date): only one insert succeeds The other insert gets error 2627, and that invoice is skipped.

  4. What happens since .NET 6 if a BackgroundService throws an unhandled exception?

    1. Nothing
    2. The job restarts
    3. The whole app stops
    4. The exception is ignored
    Show answer

    Answer: C. The whole app stops The default is StopHost, so every run in the SMS catches and logs its errors.

  5. How do the tests check the 18:00 job without waiting until evening?

    1. They change the computer's clock
    2. FakeTimeProvider: the test moves the job's clock forward
    3. They run the tests at 18:00
    4. They call ExecuteAsync twice
    Show answer

    Answer: B. FakeTimeProvider: the test moves the job's clock forward The job asks TimeProvider for the time, so a test can replace the clock.

Interview questions​

Questions are grouped by experience level. Click a question to see a model answer. All questions: Web API interview questions.

Fresher (0-2 yrs)​

F What is a hosted service in ASP.NET Core, and what is BackgroundService? concept

A hosted service is a class the app starts when it starts and stops when it stops (IHostedService, registered with AddHostedService). BackgroundService is a base class for one: you override ExecuteAsync, which runs in the background for the life of the app, and its stoppingToken is cancelled at shutdown.

See the example · Permalink

F Which work should move out of a request into the background? concept

Work the user's next step does not depend on, especially slow work or work that calls another service: sending messages, emails, and reports. In the SMS, saving a payment stays in the request; the receipt message is queued. Moving it also stops a slow or failing provider from failing the payment itself.

See the example · Permalink

F When should an endpoint answer 202 Accepted instead of 200 OK? concept

When it has accepted the work but not finished it, because the work continues in the background. 200 says "done, here is the result"; 202 says "accepted, it will be done". In the SMS, POST /api/fee-reminders answers 202 with how many reminders were queued; the worker sends them afterwards.

See the example · Permalink

Mid (3-5 yrs)​

M How do you pass work from a controller to a background worker in ASP.NET Core? coding

Register a queue as a singleton, for example a bounded Channel of messages. The controller writes a message and returns at once; a BackgroundService reads the channel with ReadAllAsync and processes each message. Bounded with FullMode.Wait means a flood of work waits for room instead of using up memory.

See the example · Permalink

M Your BackgroundService needs a scoped repository. How do you use it? coding

Not through the constructor: a hosted service is a singleton, and the app refuses to start with "Cannot consume scoped service ... from singleton IHostedService". Inject IServiceScopeFactory, create a scope for each piece of work (CreateAsyncScope), and resolve the repository from that scope; disposing the scope cleans it up.

See the example · Permalink

M What happens if ExecuteAsync throws an unhandled exception, and how do you guard against it? scenario

Since .NET 6 the default BackgroundServiceExceptionBehavior is StopHost: the whole app shuts down, every endpoint included. Catch exceptions inside the loop (except cancellation at shutdown), log them, and carry on with the next message or the next scheduled run.

See the example · Permalink

M How would you write and test a job that must run every day at 18:00? coding

A BackgroundService loop: get the time from TimeProvider, work out the next 18:00, await Task.Delay(next - now, timeProvider, stoppingToken), run, repeat. Because the clock is injected, a test uses FakeTimeProvider: set it to 17:59, start the job, Advance one minute, and check the job ran, without waiting.

See the example · Permalink

Senior (6-9 yrs)​

S The API runs on two servers and both run the evening reminder job. How do you stop duplicate reminders? scenario

Let the database decide. Record each reminder in a table with a unique key on (invoice, date) before sending, and send only if the insert succeeded; the second server gets a duplicate-key error and skips that invoice. Alternatives are a distributed lock (such as sp_getapplock) or a scheduler that runs each job on one server.

How to answer: Name the problem (every instance runs every hosted service), then give an idempotent design and say why it needs no coordination between servers.

See the example · Permalink

S Why is an in-memory queue risky, and what is the outbox pattern? design

Messages waiting in memory are lost when the app stops or crashes. With an outbox, the message is saved in a table in the same transaction as the business change (such as the payment); a worker reads unsent rows, sends them, and marks them sent. Nothing is lost on restart, but sending is at least once, so receivers must handle duplicates.

How to answer: Explain what is lost and when, describe the outbox steps, and mention the at-least-once trade-off.

See the example · Permalink

Architect / Lead (10+ yrs)​

A How would you design background processing for a school platform that sends millions of messages a day? design

Separate the API from the workers: the API writes messages to an outbox table or a durable broker (Azure Service Bus, RabbitMQ), and a scalable pool of worker services sends them. Add retries with backoff, a dead-letter queue for messages that keep failing, idempotency keys so retries never double-send, rate limits per provider, and dashboards for queue length and failures. Schedules run in one place (Hangfire, Quartz.NET, or a cloud scheduler) so each job runs once.

How to answer: Cover durability, scaling the workers, retries and dead letters, idempotency, scheduling, and monitoring.

See the example · Permalink

Resources​

  • Code for this lesson: git tag lesson-94-background-jobs (code repository coming soon)
  • Files: src/SchoolApp.Api/BackgroundJobs/FeeReminderJob.cs, src/SchoolApp.Api/BackgroundJobs/GuardianMessageWorker.cs, src/SchoolApp.Api/BackgroundJobs/GuardianMessageQueue.cs, src/SchoolApp.Core/Services/FeeReminderService.cs, src/SchoolApp.Core/Services/GuardianMessageService.cs, database/094-fee-reminders.sql, tests/SchoolApp.Tests/Unit/FeeReminderTests.cs, tests/SchoolApp.Tests/Api/BackgroundJobTests.cs, library-assignment/094-library-reminders.cs
  • Video: coming soon

What's next​

The API now does work on its own. Next, how do you know it is healthy: is it running, can it reach SQL Server, is the message worker alive?

Health checks answer that. Lesson 95: Health checks

Want live sessions, code reviews, and interview preparation? Join the next batch at Sahasra Technologies.