Background Services in .NET: IHostedService, Workers, and Queue Processing

Background Services in .NET: IHostedService, Workers, and Queue Processing

Introduction

Background services handle long-running operations outside HTTP request lifecycles: scheduled tasks, queue processing, data synchronization, and monitoring. This guide covers IHostedService and BackgroundService fundamentals, worker service applications, scheduled tasks with Quartz.NET and Hangfire, queue processing patterns with channels and message queues, and graceful shutdown for reliable operation.

IHostedService Fundamentals

Basic Implementation

Simple hosted service:

public class DataSyncService : IHostedService
{
    private readonly ILogger<DataSyncService> _logger;
    private Timer? _timer;
    
    public DataSyncService(ILogger<DataSyncService> logger)
    {
        _logger = logger;
    }
    
    public Task StartAsync(CancellationToken cancellationToken)
    {
        _logger.LogInformation("Data sync service starting");
        
        // Execute every 5 minutes
        _timer = new Timer(DoWork, null, TimeSpan.Zero, TimeSpan.FromMinutes(5));
        
        return Task.CompletedTask;
    }
    
    private void DoWork(object? state)
    {
        _logger.LogInformation("Syncing data at {Time}", DateTime.UtcNow);
        // Perform sync operation
    }
    
    public Task StopAsync(CancellationToken cancellationToken)
    {
        _logger.LogInformation("Data sync service stopping");
        
        _timer?.Change(Timeout.Infinite, 0);
        _timer?.Dispose();
        
        return Task.CompletedTask;
    }
}

// Registration
builder.Services.AddHostedService<DataSyncService>();

BackgroundService Base Class

Simplified background work:

public class EmailQueueService : BackgroundService
{
    private readonly ILogger<EmailQueueService> _logger;
    private readonly IServiceScopeFactory _scopeFactory;
    
    public EmailQueueService(
        ILogger<EmailQueueService> logger,
        IServiceScopeFactory scopeFactory)
    {
        _logger = logger;
        _scopeFactory = scopeFactory;
    }
    
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        _logger.LogInformation("Email queue service is starting");
        
        while (!stoppingToken.IsCancellationRequested)
        {
            try
            {
                await ProcessEmailQueueAsync(stoppingToken);
                await Task.Delay(TimeSpan.FromSeconds(10), stoppingToken);
            }
            catch (OperationCanceledException)
            {
                // Expected during shutdown
                break;
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "Error processing email queue");
                await Task.Delay(TimeSpan.FromMinutes(1), stoppingToken);
            }
        }
        
        _logger.LogInformation("Email queue service is stopping");
    }
    
    private async Task ProcessEmailQueueAsync(CancellationToken cancellationToken)
    {
        using var scope = _scopeFactory.CreateScope();
        var emailService = scope.ServiceProvider.GetRequiredService<IEmailService>();
        var repository = scope.ServiceProvider.GetRequiredService<IEmailQueueRepository>();
        
        var pendingEmails = await repository.GetPendingEmailsAsync(10);
        
        foreach (var email in pendingEmails)
        {
            if (cancellationToken.IsCancellationRequested)
                break;
            
            await emailService.SendAsync(email);
            await repository.MarkAsSentAsync(email.Id);
        }
    }
}

Worker Service Applications

Creating a Worker

Project template:

dotnet new worker -n MyWorkerService
cd MyWorkerService
dotnet run

Worker.cs:

public class Worker : BackgroundService
{
    private readonly ILogger<Worker> _logger;
    private readonly IConfiguration _configuration;
    
    public Worker(ILogger<Worker> logger, IConfiguration configuration)
    {
        _logger = logger;
        _configuration = configuration;
    }
    
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        var interval = _configuration.GetValue<int>("WorkerInterval", 60);
        
        while (!stoppingToken.IsCancellationRequested)
        {
            _logger.LogInformation("Worker running at: {time}", DateTimeOffset.Now);
            
            await DoWorkAsync(stoppingToken);
            
            await Task.Delay(TimeSpan.FromSeconds(interval), stoppingToken);
        }
    }
    
    private async Task DoWorkAsync(CancellationToken cancellationToken)
    {
        // Your background work here
        await Task.Delay(1000, cancellationToken);
    }
}

Program.cs:

var builder = Host.CreateApplicationBuilder(args);

builder.Services.AddHostedService<Worker>();

// Add dependencies
builder.Services.AddScoped<IDataService, DataService>();
builder.Services.AddSingleton<IMessageQueue, RabbitMqQueue>();

// Configure logging
builder.Logging.AddConsole();
builder.Logging.AddEventLog();

var host = builder.Build();
await host.RunAsync();

Windows Service Deployment

Configuration:

<Project Sdk="Microsoft.NET.Sdk.Worker">
  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
    <OutputType>exe</OutputType>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.Extensions.Hosting" Version="8.0.0" />
    <PackageReference Include="Microsoft.Extensions.Hosting.WindowsServices" Version="8.0.0" />
  </ItemGroup>
</Project>

Enable Windows Service:

var builder = Host.CreateApplicationBuilder(args);

builder.Services.AddWindowsService(options =>
{
    options.ServiceName = "MyWorkerService";
});

builder.Services.AddHostedService<Worker>();

var host = builder.Build();
await host.RunAsync();

Install service:

# Publish
dotnet publish -c Release -o C:\Services\MyWorkerService

# Create service
sc create MyWorkerService binPath="C:\Services\MyWorkerService\MyWorkerService.exe"

# Start service
sc start MyWorkerService

# Stop service
sc stop MyWorkerService

# Delete service
sc delete MyWorkerService

Scheduled Tasks

Quartz.NET Integration

Installation:

dotnet add package Quartz
dotnet add package Quartz.Extensions.Hosting

Configuration:

builder.Services.AddQuartz(q =>
{
    // Register jobs
    q.AddJob<DataCleanupJob>(opts => opts.WithIdentity("DataCleanup"));
    q.AddJob<ReportGenerationJob>(opts => opts.WithIdentity("ReportGeneration"));
    
    // Trigger for data cleanup (daily at 2 AM)
    q.AddTrigger(opts => opts
        .ForJob("DataCleanup")
        .WithIdentity("DataCleanup-trigger")
        .WithCronSchedule("0 0 2 * * ?"));
    
    // Trigger for report generation (hourly)
    q.AddTrigger(opts => opts
        .ForJob("ReportGeneration")
        .WithIdentity("ReportGeneration-trigger")
        .WithSimpleSchedule(x => x
            .WithIntervalInHours(1)
            .RepeatForever()));
});

builder.Services.AddQuartzHostedService(options =>
{
    options.WaitForJobsToComplete = true;
});

Job implementation:

public class DataCleanupJob : IJob
{
    private readonly ILogger<DataCleanupJob> _logger;
    private readonly IServiceScopeFactory _scopeFactory;
    
    public DataCleanupJob(
        ILogger<DataCleanupJob> logger,
        IServiceScopeFactory scopeFactory)
    {
        _logger = logger;
        _scopeFactory = scopeFactory;
    }
    
    public async Task Execute(IJobExecutionContext context)
    {
        _logger.LogInformation("Starting data cleanup job");
        
        using var scope = _scopeFactory.CreateScope();
        var repository = scope.ServiceProvider.GetRequiredService<IDataRepository>();
        
        var cutoffDate = DateTime.UtcNow.AddDays(-90);
        var deletedCount = await repository.DeleteOldRecordsAsync(cutoffDate);
        
        _logger.LogInformation(
            "Data cleanup completed. Deleted {Count} records", 
            deletedCount);
    }
}

Hangfire Alternative

Installation:

dotnet add package Hangfire
dotnet add package Hangfire.SqlServer

Configuration:

builder.Services.AddHangfire(config =>
{
    config.UseSqlServerStorage(
        builder.Configuration.GetConnectionString("HangfireConnection"));
});

builder.Services.AddHangfireServer();

var app = builder.Build();

app.UseHangfireDashboard("/hangfire", new DashboardOptions
{
    Authorization = new[] { new HangfireAuthorizationFilter() }
});

// Schedule recurring jobs
RecurringJob.AddOrUpdate(
    "data-cleanup",
    () => CleanupOldData(),
    Cron.Daily(2));

RecurringJob.AddOrUpdate(
    "send-reports",
    () => SendDailyReports(),
    Cron.Daily(9));

Queue Processing with Channels

Producer-Consumer Pattern

Channel-based queue:

public interface IBackgroundTaskQueue
{
    ValueTask QueueAsync(Func<CancellationToken, ValueTask> workItem);
    ValueTask<Func<CancellationToken, ValueTask>> DequeueAsync(CancellationToken cancellationToken);
}

public class BackgroundTaskQueue : IBackgroundTaskQueue
{
    private readonly Channel<Func<CancellationToken, ValueTask>> _queue;
    
    public BackgroundTaskQueue(int capacity = 100)
    {
        var options = new BoundedChannelOptions(capacity)
        {
            FullMode = BoundedChannelFullMode.Wait
        };
        _queue = Channel.CreateBounded<Func<CancellationToken, ValueTask>>(options);
    }
    
    public async ValueTask QueueAsync(Func<CancellationToken, ValueTask> workItem)
    {
        if (workItem is null)
            throw new ArgumentNullException(nameof(workItem));
        
        await _queue.Writer.WriteAsync(workItem);
    }
    
    public async ValueTask<Func<CancellationToken, ValueTask>> DequeueAsync(
        CancellationToken cancellationToken)
    {
        return await _queue.Reader.ReadAsync(cancellationToken);
    }
}

// Registration
builder.Services.AddSingleton<IBackgroundTaskQueue, BackgroundTaskQueue>();
builder.Services.AddHostedService<QueueProcessorService>();

Queue processor:

public class QueueProcessorService : BackgroundService
{
    private readonly IBackgroundTaskQueue _taskQueue;
    private readonly ILogger<QueueProcessorService> _logger;
    
    public QueueProcessorService(
        IBackgroundTaskQueue taskQueue,
        ILogger<QueueProcessorService> logger)
    {
        _taskQueue = taskQueue;
        _logger = logger;
    }
    
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        _logger.LogInformation("Queue processor service is starting");
        
        await foreach (var workItem in GetWorkItemsAsync(stoppingToken))
        {
            try
            {
                await workItem(stoppingToken);
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "Error processing work item");
            }
        }
    }
    
    private async IAsyncEnumerable<Func<CancellationToken, ValueTask>> GetWorkItemsAsync(
        [EnumeratorCancellation] CancellationToken cancellationToken)
    {
        while (!cancellationToken.IsCancellationRequested)
        {
            var workItem = await _taskQueue.DequeueAsync(cancellationToken);
            yield return workItem;
        }
    }
}

Usage in API:

[ApiController]
[Route("api/[controller]")]
public class OrdersController : ControllerBase
{
    private readonly IBackgroundTaskQueue _taskQueue;
    private readonly IOrderService _orderService;
    
    [HttpPost]
    public async Task<IActionResult> CreateOrder(Order order)
    {
        // Save order synchronously
        var created = await _orderService.CreateAsync(order);
        
        // Queue background tasks
        await _taskQueue.QueueAsync(async ct =>
        {
            await SendConfirmationEmailAsync(created.Id, ct);
        });
        
        await _taskQueue.QueueAsync(async ct =>
        {
            await UpdateInventoryAsync(created.Items, ct);
        });
        
        return CreatedAtAction(nameof(GetOrder), new { id = created.Id }, created);
    }
}

Message Queue Integration

RabbitMQ Consumer

Installation:

dotnet add package RabbitMQ.Client

Consumer service:

public class RabbitMqConsumerService : BackgroundService
{
    private readonly ILogger<RabbitMqConsumerService> _logger;
    private readonly IServiceScopeFactory _scopeFactory;
    private IConnection? _connection;
    private IModel? _channel;
    
    public RabbitMqConsumerService(
        ILogger<RabbitMqConsumerService> logger,
        IServiceScopeFactory scopeFactory)
    {
        _logger = logger;
        _scopeFactory = scopeFactory;
        
        InitializeRabbitMQ();
    }
    
    private void InitializeRabbitMQ()
    {
        var factory = new ConnectionFactory
        {
            HostName = "localhost",
            UserName = "guest",
            Password = "guest",
            DispatchConsumersAsync = true
        };
        
        _connection = factory.CreateConnection();
        _channel = _connection.CreateModel();
        
        _channel.QueueDeclare(
            queue: "orders",
            durable: true,
            exclusive: false,
            autoDelete: false);
        
        _channel.BasicQos(prefetchSize: 0, prefetchCount: 1, global: false);
    }
    
    protected override Task ExecuteAsync(CancellationToken stoppingToken)
    {
        stoppingToken.Register(() => _logger.LogInformation("Consumer stopping"));
        
        var consumer = new AsyncEventingBasicConsumer(_channel);
        
        consumer.Received += async (model, ea) =>
        {
            try
            {
                var body = ea.Body.ToArray();
                var message = Encoding.UTF8.GetString(body);
                
                await ProcessMessageAsync(message, stoppingToken);
                
                _channel.BasicAck(ea.DeliveryTag, multiple: false);
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "Error processing message");
                _channel.BasicNack(ea.DeliveryTag, multiple: false, requeue: true);
            }
        };
        
        _channel.BasicConsume(
            queue: "orders",
            autoAck: false,
            consumer: consumer);
        
        return Task.CompletedTask;
    }
    
    private async Task ProcessMessageAsync(string message, CancellationToken cancellationToken)
    {
        using var scope = _scopeFactory.CreateScope();
        var orderService = scope.ServiceProvider.GetRequiredService<IOrderService>();
        
        var order = JsonSerializer.Deserialize<Order>(message);
        await orderService.ProcessOrderAsync(order, cancellationToken);
    }
    
    public override void Dispose()
    {
        _channel?.Close();
        _connection?.Close();
        base.Dispose();
    }
}

Azure Service Bus Integration

Installation:

dotnet add package Azure.Messaging.ServiceBus

Consumer service:

public class ServiceBusConsumerService : BackgroundService
{
    private readonly ILogger<ServiceBusConsumerService> _logger;
    private readonly IServiceScopeFactory _scopeFactory;
    private ServiceBusProcessor? _processor;
    
    public ServiceBusConsumerService(
        ILogger<ServiceBusConsumerService> logger,
        IServiceScopeFactory scopeFactory,
        IConfiguration configuration)
    {
        _logger = logger;
        _scopeFactory = scopeFactory;
        
        var connectionString = configuration["ServiceBus:ConnectionString"];
        var queueName = configuration["ServiceBus:QueueName"];
        
        var client = new ServiceBusClient(connectionString);
        _processor = client.CreateProcessor(queueName, new ServiceBusProcessorOptions
        {
            MaxConcurrentCalls = 10,
            AutoCompleteMessages = false
        });
        
        _processor.ProcessMessageAsync += ProcessMessageAsync;
        _processor.ProcessErrorAsync += ProcessErrorAsync;
    }
    
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        await _processor!.StartProcessingAsync(stoppingToken);
        
        // Keep service running
        await Task.Delay(Timeout.Infinite, stoppingToken);
    }
    
    private async Task ProcessMessageAsync(ProcessMessageEventArgs args)
    {
        using var scope = _scopeFactory.CreateScope();
        var orderService = scope.ServiceProvider.GetRequiredService<IOrderService>();
        
        var body = args.Message.Body.ToString();
        var order = JsonSerializer.Deserialize<Order>(body);
        
        await orderService.ProcessOrderAsync(order, args.CancellationToken);
        await args.CompleteMessageAsync(args.Message);
    }
    
    private Task ProcessErrorAsync(ProcessErrorEventArgs args)
    {
        _logger.LogError(args.Exception, "Service Bus error");
        return Task.CompletedTask;
    }
    
    public override async Task StopAsync(CancellationToken cancellationToken)
    {
        await _processor!.StopProcessingAsync(cancellationToken);
        await base.StopAsync(cancellationToken);
    }
}

Graceful Shutdown

Handling Cancellation

Proper cleanup:

public class LongRunningService : BackgroundService
{
    private readonly ILogger<LongRunningService> _logger;
    
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        try
        {
            while (!stoppingToken.IsCancellationRequested)
            {
                await ProcessBatchAsync(stoppingToken);
            }
        }
        catch (OperationCanceledException)
        {
            // Expected during shutdown
            _logger.LogInformation("Service is stopping gracefully");
        }
        finally
        {
            await CleanupResourcesAsync();
        }
    }
    
    private async Task ProcessBatchAsync(CancellationToken cancellationToken)
    {
        // Pass cancellation token to all async operations
        await FetchDataAsync(cancellationToken);
        await ProcessDataAsync(cancellationToken);
        await SaveResultsAsync(cancellationToken);
        
        // Check cancellation before delays
        await Task.Delay(TimeSpan.FromMinutes(1), cancellationToken);
    }
    
    private async Task CleanupResourcesAsync()
    {
        _logger.LogInformation("Cleaning up resources");
        // Close connections, flush buffers, etc.
    }
}

Shutdown Timeout

Configuration:

var builder = Host.CreateApplicationBuilder(args);

builder.Services.Configure<HostOptions>(options =>
{
    options.ShutdownTimeout = TimeSpan.FromSeconds(30);
});

builder.Services.AddHostedService<Worker>();

var host = builder.Build();
await host.RunAsync();

Best Practices

1. Use IServiceScopeFactory: Create scoped services in background tasks
2. Handle Cancellation: Respect CancellationToken in all async operations
3. Implement Idempotency: Handle duplicate message processing
4. Add Retry Logic: Use Polly for transient failures
5. Monitor Performance: Track queue depth and processing times
6. Log Extensively: Background errors are easy to miss
7. Test Shutdown: Ensure graceful cleanup under all conditions

Troubleshooting

Service Won't Stop:

// ❌ Ignores cancellation
while (true)
{
    await Task.Delay(1000);
}

// ✅ Respects cancellation
while (!stoppingToken.IsCancellationRequested)
{
    await Task.Delay(1000, stoppingToken);
}

Memory Leaks:

// ✅ Always dispose scopes
using var scope = _scopeFactory.CreateScope();
var service = scope.ServiceProvider.GetRequiredService<IService>();
// Scope disposed automatically

Key Takeaways

• IHostedService and BackgroundService enable long-running background operations
• Worker services run as standalone applications or Windows Services
• Quartz.NET and Hangfire provide robust scheduling capabilities
• Channels enable efficient producer-consumer queue processing
• Graceful shutdown handling prevents data loss and resource leaks

Next Steps

• Implement health checks for background services
• Add distributed locking with Redis for multi-instance deployments
• Explore Akka.NET for actor-based processing
• Use Application Insights to monitor background job performance

Additional Resources

• Background Tasks in ASP.NET Core
• Worker Services
• Quartz.NET Documentation
• Hangfire Documentation

---

Background work, foreground results.