32. Parallel Programming in C#

Level: Advanced
Goal: Speed up CPU-heavy work by using all CPU cores at once.

Imagine calculating grades for 10,000 students:

• Old way (one by one): 10 seconds
• Parallel way (all cores): 2 seconds (on 4-core CPU)

Parallel spreads work across CPU cores. Perfect for calculation-heavy work.

ℹ️ What You'll Learn

• Parallel vs async - when each is right
• Parallel.For and Parallel.ForEach for loop parallelism
• PLINQ - parallel LINQ queries
• Thread-safe collections (ConcurrentBag, ConcurrentDictionary)
• ParallelOptions for controlling parallelism
• Overhead and when parallel hurts performance
• CPU-bound vs I/O-bound work
• Measuring and tuning parallel code

Parallel Programming Concepts Reference

Concept	Purpose	Use Case	Thread-Safe	Example
Parallel.For	Loop with index, parallel iterations	CPU-bound loops	Wrong Requires synchronization	Parallel.For(0, count, i => Work(i))
Parallel.ForEach	Loop collection items, parallel	CPU-bound collection processing	Wrong Requires synchronization	Parallel.ForEach(items, item => Work(item))
PLINQ	AsParallel() on LINQ	Query parallelization	Wrong Requires synchronization	items.AsParallel().Where(...).ToList()
ParallelOptions	Configure Max cores, CancellationToken	Control parallelism degree	N/A	MaxDegreeOfParallelism = 4
ConcurrentBag<T>	Thread-safe collection, unordered	Collect results from parallel work	OK Thread-safe	bag.Add(item) from multiple threads
ConcurrentDictionary<K,V>	Thread-safe key-value store	Shared results from parallel work	OK Thread-safe	dict.TryAdd(key, value)
Partitioner<T>	Split collection for better balance	Large collections, custom partitioning	OK Safe when used correctly	Partitioner.Create(items, true)
Task.Run + Parallel	Wrap blocking parallel work in Task	Non-blocking parallel operations	Wrong Requires synchronization	await Task.Run(() => Parallel.ForEach(...))

---

Quick Definitions

• Parallel programming - Running work on multiple CPU cores simultaneously
• Parallel.For - Loop each iteration on different core
• Parallel.ForEach - Loop collection items on different cores
• PLINQ - Parallel LINQ queries with AsParallel()
• ParallelOptions - Configure max cores and cancellation
• ConcurrentBag<T> - Thread-safe collection (unordered)
• ConcurrentDictionary<K,V> - Thread-safe key-value store
• Partitioner - Split collection for balanced work distribution
• Data parallelism - Distribute data across cores
• CPU-bound - Work using calculations (suitable for parallel)

---

When to Use Parallel (and When NOT to)

OK Use Parallel for:

• CPU-bound work (calculations, report generation, grade point computation)
• Large collections (100+ items) with independent processing
• Batch operations that dont share state
• Math-heavy algorithms that utilize multi-core processors

Wrong Dont use Parallel for:

• I/O-bound work (database calls, HTTP requests, file reads) - use async/await instead
• Small loops with simple work (overhead > benefit)
• Shared mutable state without synchronization
• Work that requires specific ordering

// Wrong BAD - I/O bound, use async instead
var students = GetAllStudents();
Parallel.ForEach(students, student =>
{
    var grades = LoadFromDatabase(student.Id);  // I/O blocking thread!
});

// OK GOOD - use async for I/O
var students = GetAllStudents();
var tasks = students.Select(s => LoadFromDatabaseAsync(s.Id));
await Task.WhenAll(tasks);

// Wrong BAD - tiny loop, overhead > gain
Parallel.For(0, 5, i => Console.WriteLine(i));  // 5 items not worth parallelizing

// OK GOOD - large CPU-bound loop
var results = new double[1000000];
Parallel.For(0, results.Length, i =>
{
    results[i] = ExpensiveCalculation(i);  // Worth parallelizing
});

Rule of thumb: Collection size work per item > parallelism overhead? Then parallelize.

---

Parallel vs Async - Key Difference

Parallel: Uses multiple cores simultaneously. Thread allocated per item. For CPU-bound.

Async: Frees thread during I/O wait. Single thread handles many concurrent I/O. For I/O-bound.

// PARALLEL - use all CPU cores for calculation
var grades = new double[10000];
Parallel.For(0, grades.Length, i =>
{
    grades[i] = CalculateGradePoints(i);  // CPU work on multiple cores
});

// ASYNC - frees thread during I/O, handles many concurrent requests
async Task ProcessAllStudentsAsync()
{
    var tasks = students.Select(s => 
        LoadFromDatabaseAsync(s.Id)  // Thread free during DB I/O
    );
    await Task.WhenAll(tasks);  // One thread handles many waits
}

// DON'T MIX - parallel blocking threads + async overhead = slow
var results = new List<int>();
Parallel.ForEach(students, student =>
{
    var score = LoadFromDatabase(student.Id);  // Wrong Blocks thread on I/O
    results.Add(score);  // Wrong Race condition on results
});

In web apps: Always use async/await for I/O. Use Parallel for CPU calculations inside async methods if needed.

---

Parallel.ForEach - Process Collection in Parallel

var students = GetAllStudents();  // 500 students

// Sequential - one at a time, 100ms per student
foreach (var student in students)
    GenerateReportCard(student);   // 500  100ms = 50 seconds

// Parallel - on 8-core machine, roughly 8 at a time
Parallel.ForEach(students, student =>
    GenerateReportCard(student));  // ~50s / 8 = ~6 seconds

// With ParallelOptions - control degree and cancellation
var options = new ParallelOptions
{
    MaxDegreeOfParallelism = Environment.ProcessorCount,  // use all cores
    CancellationToken      = cancellationToken
};

Parallel.ForEach(students, options, student =>
{
    try
    {
        GenerateReportCard(student);
    }
    catch (OperationCanceledException)
    {
        // Cancelled, stop processing
    }
});

// Limit to specific core count (e.g., leave 2 cores free)
var limitedOptions = new ParallelOptions
{
    MaxDegreeOfParallelism = Environment.ProcessorCount - 2
};

Parallel.ForEach(students, limitedOptions, student =>
    GenerateReportCard(student));

Note: Iteration order is NOT guaranteed. If order matters, track separately or use PLINQ with .AsOrdered().

Parallel.For - Index-Based

var results = new double[students.Count];

Parallel.For(0, students.Count, i =>
{
    results[i] = CalculateGradePoints(students[i]);
});

// Each index processed in parallel - results array populated concurrently

PLINQ - Parallel LINQ

var students = GetAllStudents();

// Normal LINQ - sequential
var topStudents = students
    .Where(s => s.Percentage >= 75)
    .OrderByDescending(s => s.Percentage)
    .ToList();

// PLINQ - parallel
var topStudentsFast = students
    .AsParallel()                          // enable parallel
    .WithDegreeOfParallelism(4)            // limit cores
    .Where(s => s.Percentage >= 75)
    .OrderByDescending(s => s.Percentage)
    .ToList();

// Preserve order (slower but ordered output)
var ordered = students
    .AsParallel()
    .AsOrdered()
    .Where(s => s.Percentage >= 35)
    .Select(s => new { s.Name, s.Percentage })
    .ToList();

School Management - Batch Report Generation

💻 Try It — Console App

💡 Paste into Program.cs and press F5⌥ GitHub

public class BatchReportService
{
    public async Task<BatchResult> GenerateAllReportsAsync(
        List<Student> students,
        CancellationToken ct = default)
    {
        var success = new ConcurrentBag<string>();  // thread-safe collection
        var failed  = new ConcurrentBag<string>();
        var sw      = System.Diagnostics.Stopwatch.StartNew();

        var opts = new ParallelOptions
        {
            MaxDegreeOfParallelism = Environment.ProcessorCount,
            CancellationToken      = ct
        };

        await Task.Run(() =>
        {
            Parallel.ForEach(students, opts, student =>
            {
                try
                {
                    string report = BuildReportCard(student);
                    // ConcurrentBag is thread-safe
                    success.Add(student.RollNumber);
                }
                catch (Exception ex)
                {
                    failed.Add($"{student.RollNumber}: {ex.Message}");
                }
            });
        }, ct);

        sw.Stop();

        return new BatchResult
        {
            TotalProcessed = students.Count,
            Successful     = success.Count,
            Failed         = failed.Count,
            FailedItems    = failed.ToList(),
            ElapsedMs      = sw.ElapsedMilliseconds
        };
    }

    private static string BuildReportCard(Student s)
    {
        Thread.SpinWait(10000);   // simulate CPU work
        return $"Report: {s.Name} | {s.ClassName} | {s.Percentage:F1}%";
    }
}

public record BatchResult(
    int          TotalProcessed,
    int          Successful,
    int          Failed,
    List<string> FailedItems,
    long         ElapsedMs);

---

When You'll Use This in SMS

SMS parallelizes compute-heavy operations:

// Calculate grades for 10,000 students in parallel
var students = GetAllStudents();  // 10,000 records
var result = new ConcurrentBag<StudentGrade>();

Parallel.ForEach(students, student =>
{
    var grade = CalculateGradeParallel(student);  // CPU-heavy
    result.Add(grade);  // Thread-safe collection
});

// Generate all report cards in parallel (I/O after calculation)
var opts = new ParallelOptions { MaxDegreeOfParallelism = 4 };
Parallel.ForEach(students, opts, student =>
{
    string report = BuildReportCard(student);  // CPU work
    SaveReportToFile(student.Id, report);       // Then I/O
});

// PLINQ for filtering/sorting large result sets
var topStudents = allResults
    .AsParallel()
    .Where(s => s.Percentage >= 75)
    .OrderByDescending(s => s.Percentage)
    .ToList();

Real impact: Without parallelism = 10,000 grades in 30 seconds. With parallelism (4 cores) = 8 seconds. Speed gain = CPU cores work complexity.

---

Try This Now

1. Create Parallel.ForEach() to calculate grades for 1000 students
2. Use ConcurrentBag to collect results
3. Compare: sequential vs parallel execution time
4. Add ParallelOptions to limit max cores to 2
5. Measure and log ElapsedMilliseconds difference

---

ℹ️ Video Tutorial

Parallel programming explained: Parallel.For, Parallel.ForEach, PLINQ, ConcurrentBag, when NOT to parallelize. Video coming soon. Subscribe to NexCoding YouTube for updates.

---

Common Mistakes

Wrong Using Parallel for I/O-bound work (blocks threads):

Parallel.ForEach(students, student =>
{
    var data = _db.GetStudentGrades(student.Id);  // I/O blocks thread!
    ProcessGrades(data);
});

OK Use async for I/O:

var tasks = students.Select(s => GetStudentGradesAsync(s.Id));
await Task.WhenAll(tasks);

Wrong Shared mutable state without synchronization (race condition):

var total = 0;
Parallel.ForEach(students, student =>
{
    total += student.Percentage;  // Race condition! Lost increments
});

OK Use thread-safe collection or lock:

// Option 1 - thread-safe collection
var bag = new ConcurrentBag<double>();
Parallel.ForEach(students, student =>
{
    bag.Add(student.Percentage);
});
double total = bag.Sum();

// Option 2 - lock
var total = 0.0;
var _lock = new object();
Parallel.ForEach(students, student =>
{
    lock (_lock) { total += student.Percentage; }
});

Wrong Parallelizing tiny loops (overhead > benefit):

Parallel.For(0, 10, i => DoQuickWork(i));  // Not worth it!

OK Only parallelize large, expensive work:

if (students.Count > 100 && IsExpensiveWork())
    Parallel.ForEach(students, ProcessStudentExpensive);
else
    foreach (var s in students)
        ProcessStudentExpensive(s);

Wrong Not measuring performance (assumes parallel is faster):

// Parallelized without profiling - might be slower!
Parallel.ForEach(students, student => QuickCalculation(student));

OK Measure before and after:

var sw = System.Diagnostics.Stopwatch.StartNew();
Parallel.ForEach(students, student => ExpensiveCalculation(student));
sw.Stop();
Console.WriteLine($"Parallel: {sw.ElapsedMilliseconds}ms");

Wrong Modifying source collection during iteration:

var list = GetStudents().ToList();
Parallel.ForEach(list, student =>
{
    if (student.Percentage < 35)
        list.Remove(student);  // Wrong Undefined behavior
});

OK Create results collection, don't modify source:

var list = GetStudents().ToList();
var failed = new ConcurrentBag<Student>();
Parallel.ForEach(list, student =>
{
    if (student.Percentage < 35)
        failed.Add(student);
});

---

Best Practices

1. Use Parallel for CPU-bound work only - Large calculations, report generation
2. Use async/await for I/O - Database, file, HTTP (don't block threads)
3. Measure before optimizing - Parallelism has overhead; profile first
4. Use ConcurrentBag or ConcurrentDictionary - For collecting results
5. Avoid lock contention - Don't hold locks long, use Interlocked when possible
6. Don't parallelize tiny loops - Only worth it for 100+ items with expensive work
7. Control degree of parallelism - MaxDegreeOfParallelism for resource control
8. Support CancellationToken - Let caller cancel long operations
9. Use PLINQ for queries - AsParallel() for filtering, grouping, sorting
10. Document why parallel is needed - Help future developers understand tradeoff
11. Test thread safety - Run under load, race conditions hide in normal testing
12. Consider Partitioner for large collections - Better load balancing than default

---

🎯 Q1: What's the difference between Parallel and async/await?

Parallel: Uses multiple cores. Each thread processes one item. For CPU-bound.

async/await: Frees thread during I/O. One thread handles many waits. For I/O-bound.

// Parallel - all cores, CPU work
Parallel.ForEach(items, item => HeavyCalculation(item));

// async - free threads, I/O work
await Task.WhenAll(items.Select(item => GetDataAsync(item.Id)));

Mixing both: parallel blocking threads on I/O = slow. Use async for I/O, parallel for CPU.

🎯 Q2: When does Parallel.ForEach actually slow things down?

Overhead of creating/managing parallel tasks > benefit of parallelism when:

1. Small collection (< 100 items)
2. Each item = quick work (< 10ms)
3. High contention on shared state (lots of locks)

// Slow with parallel - overhead > benefit
Parallel.ForEach(items.Take(5), item => SimpleMath(item));

// Fast with parallel - item count  cost > overhead
Parallel.ForEach(items.Take(10000), item => ExpensiveCalculation(item));

Profile with stopwatch. If sequential faster = don't parallelize.

🎯 Q3: What is PLINQ and how is it different from LINQ?

PLINQ = Parallel LINQ. Uses AsParallel() to parallelize query operations.

// LINQ - sequential
var passed = students
    .Where(s => s.Percentage >= 35)
    .OrderBy(s => s.Name)
    .ToList();

// PLINQ - parallel Where and OrderBy
var passed = students
    .AsParallel()
    .Where(s => s.Percentage >= 35)
    .OrderBy(s => s.Name)  // Parallel sort on multi-core
    .ToList();

Use PLINQ for expensive filtering/grouping/sorting on large collections. Check if actually faster with profiling.

🎯 Q4: What is a thread-safe collection and when do I need one?

Thread-safe collection = multiple threads can safely add/remove without corruption. Examples: ConcurrentBag, ConcurrentDictionary, ConcurrentQueue.

Need one when: multiple parallel tasks share results collection.

// NOT thread-safe - race condition
var results = new List<int>();
Parallel.ForEach(items, item =>
{
    results.Add(item * 2);  // Multiple threads write, results corrupt
});

// Thread-safe
var results = new ConcurrentBag<int>();
Parallel.ForEach(items, item =>
{
    results.Add(item * 2);  // Safe
});

Without thread-safe collection, use lock or Interlocked.

🎯 Q5: How do I control how many cores Parallel uses?

Use ParallelOptions.MaxDegreeOfParallelism.

var opts = new ParallelOptions
{
    MaxDegreeOfParallelism = 4  // Use only 4 cores
};

Parallel.ForEach(students, opts, student => ProcessStudent(student));

Useful when: want to leave cores free for other apps, prevent overloading server.

Environment.ProcessorCount = total cores available. Limit to ProcessorCount - 2 to keep system responsive.

🎯 Q6: How do I measure if Parallel is actually faster?

Use Stopwatch to time sequential vs parallel.

var sw = System.Diagnostics.Stopwatch.StartNew();
foreach (var s in students)
    CalculateGrade(s);
sw.Stop();
Console.WriteLine($"Sequential: {sw.ElapsedMilliseconds}ms");

sw.Restart();
Parallel.ForEach(students, s => CalculateGrade(s));
sw.Stop();
Console.WriteLine($"Parallel: {sw.ElapsedMilliseconds}ms");

Rule: Only use Parallel if it's measurably faster under realistic conditions. Profile first, optimize second.

---

🤖Use AI to Learn Faster

Use ChatGPT, Claude, or Copilot to go deeper on C# parallel programming. Try these prompts:

• "What is the difference between parallel programming and async/await in C#?"
• "When does Parallel.ForEach actually make things slower?"
• "What is PLINQ and how is it different from regular LINQ?"
• "What is a thread-safe collection in C# and when do I need one?"

💡 Tip: After reading this article, paste your own code into AI and ask "What could go wrong here and why?" — fastest way to find edge cases and deepen understanding.

Next Article

Reflection and Attributes ->