Acta. Durable execution for .NET

learn actamake work survive a crash

How to compensate when the last attempt fails

A failing step retries through its job, and a catch on EndsJob compensates before the failure ends the job.

A how-to for Acta, durable execution for .NET. New to Acta? Run your first durable job in five minutes.

Checked every release learn/200-durable-execution/210-step-retry
Read as

1Give the job its retry budget

A step retries through its job: a body that throws fails the attempt, MaxAttempts and Backoff retry it, and the retry replays the steps that already succeeded, so the slot is reserved once.

step-retry.csexcerpt
[Job("fetch-forecast", MaxAttempts = 3, Backoff = "1s")]
public async Task Handle(FetchForecast input, JobContext context, CancellationToken ct)
{
    await context.RunStepAsync(
        "reserve-slot",
        async inner =>
        {
            Console.WriteLine($"[{input.City}] report slot reserved");
            await Task.CompletedTask;
        },
        ct
    );

    try
    {
        await context.RunStepAsync(
            "call-weather-api",
            async inner =>
            {
                Console.WriteLine($"[{input.City}] calling weather API (attempt {context.ExecutionNumber})");
                await Task.Delay(50, inner);
                if (input.City == "Atlantis" || context.ExecutionNumber < 3)
                {
                    throw new InvalidOperationException("weather API timeout");
                }
                Console.WriteLine($"[{input.City}] forecast: 18C, light rain");
            },
            ct
        );
    }
    catch (Exception ex) when (context.EndsJob(ex))
    {
        // This failure ends the job, so give the slot back first. Best effort: a crash on the last attempt
        // skips this block, which is why the release is a step of its own and safe to repeat.
        await context.RunStepAsync(
            "release-slot",
            async inner =>
            {
                Console.WriteLine($"[{input.City}] report slot released; the job fails for good");
                await Task.CompletedTask;
            },
            ct
        );
        throw;
    }

    Console.WriteLine($"[{input.City}] forecast stored");
}

2Compensate in a catch that fires only when the failure ends the job

EndsJob is true when this failure ends the job: on the last attempt, or on any attempt for one that is not retried.

step-retry.csexcerpt
try
{
    await context.RunStepAsync(
        "call-weather-api",
        async inner =>
        {
            Console.WriteLine($"[{input.City}] calling weather API (attempt {context.ExecutionNumber})");
            await Task.Delay(50, inner);
            if (input.City == "Atlantis" || context.ExecutionNumber < 3)
            {
                throw new InvalidOperationException("weather API timeout");
            }
            Console.WriteLine($"[{input.City}] forecast: 18C, light rain");
        },
        ct
    );
}
catch (Exception ex) when (context.EndsJob(ex))
{
    // This failure ends the job, so give the slot back first. Best effort: a crash on the last attempt
    // skips this block, which is why the release is a step of its own and safe to repeat.
    await context.RunStepAsync(
        "release-slot",
        async inner =>
        {
            Console.WriteLine($"[{input.City}] report slot released; the job fails for good");
            await Task.CompletedTask;
        },
        ct
    );
    throw;
}

Console.WriteLine($"[{input.City}] forecast stored");

3Run it and check the output

From a clone of the Acta repository, with the .NET 10 SDK:

Terminal
dotnet run --project learn/200-durable-execution/210-step-retry
Its output must contain
[Ljubljana] forecast stored
[Atlantis] report slot released; the job fails for good

Checked every release. Before each release, Acta Verify runs this program and checks that its output contains these lines.

This page shows the code. The rules it relies on, starting with the handler running again from its first line, are in the handler contract.

The whole program

Every file of learn/200-durable-execution/210-step-retry, as it is compiled.

UseLocalDatabase is a helper of the Acta repository, not of the packages. In your own app, register the database with UseSqlite, UsePostgres or UseSqlServer, as in how to run your first job.

step-retry.cs95 lines
step-retry.cs
// How to compensate when the last attempt fails
// A failing step retries through its job, and a `catch` on `EndsJob` compensates before the failure ends the job.
using Acta;
using Acta.Learn.StepRetry;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;

var builder = Host.CreateApplicationBuilder(args);

builder.Services.UseActa(j =>
{
    j.UseLocalDatabase(builder.Configuration);
    j.Run<StepRetryJobs>("step-retry");
});

// The steps fail on purpose to show the retries; quiet the framework retry logging.
builder.Logging.AddFilter("Acta", LogLevel.Error);

using var host = builder.Build();
await host.StartAsync();

var jobs = host.Services.GetRequiredService<IJobs>();

// Ljubljana's forecast arrives on the third attempt; Atlantis's never does, so its last attempt compensates.
await jobs.EnqueueAsync(new FetchForecast("Ljubljana"));
await jobs.EnqueueAsync(new FetchForecast("Atlantis"));
Console.WriteLine("Enqueued. Worker is running - press Ctrl+C to stop.");

await host.WaitForShutdownAsync();
await host.StopAsync();

namespace Acta.Learn.StepRetry
{
    public sealed record FetchForecast(string City);

    public sealed class FetchForecastJob
    {
        // learn: 1. Give the job its retry budget
        // A step retries through its job: a body that throws fails the attempt, MaxAttempts and Backoff retry it,
        // and the retry replays the steps that already succeeded, so the slot is reserved once.
        [Job("fetch-forecast", MaxAttempts = 3, Backoff = "1s")]
        // learn: end
        public async Task Handle(FetchForecast input, JobContext context, CancellationToken ct)
        {
            await context.RunStepAsync(
                "reserve-slot",
                async inner =>
                {
                    Console.WriteLine($"[{input.City}] report slot reserved");
                    await Task.CompletedTask;
                },
                ct
            );

            try
            {
                await context.RunStepAsync(
                    "call-weather-api",
                    async inner =>
                    {
                        Console.WriteLine($"[{input.City}] calling weather API (attempt {context.ExecutionNumber})");
                        await Task.Delay(50, inner);
                        if (input.City == "Atlantis" || context.ExecutionNumber < 3)
                        {
                            throw new InvalidOperationException("weather API timeout");
                        }
                        Console.WriteLine($"[{input.City}] forecast: 18C, light rain");
                    },
                    ct
                );
            }
            // learn: 2. Compensate in a catch that fires only when the failure ends the job
            // EndsJob is true when this failure ends the job: on the last attempt, or on any attempt for one that is not retried.
            catch (Exception ex) when (context.EndsJob(ex))
            {
                // This failure ends the job, so give the slot back first. Best effort: a crash on the last attempt
                // skips this block, which is why the release is a step of its own and safe to repeat.
                await context.RunStepAsync(
                    "release-slot",
                    async inner =>
                    {
                        Console.WriteLine($"[{input.City}] report slot released; the job fails for good");
                        await Task.CompletedTask;
                    },
                    ct
                );
                throw;
            }
            // learn: end

            Console.WriteLine($"[{input.City}] forecast stored");
        }
    }
}

Where to go next

New to Acta?

Acta is durable execution for .NET: workflows and jobs that survive restarts, as a library inside your application with its durable state in your SQL database.