One-Shot APIs Hide Their Setup Cost

This is the fifth post in the Campfire series. The last one was about work we didn’t need. This one is about work we did need, done in a way that repeated its setup on every call.

The numbers were measured during development, one fix at a time, on an Intel i9-11900K running Linux with the app on four CPUs. They aren’t the final benchmark.

Four crypto operations per request

Campfire keeps people signed in with two cookies, and the C# port has to read and write them exactly as Rails does, so nobody gets signed out when an install switches from Rails. On every signed-in GET, that means four operations:

  1. Verify the HMAC on the signed session_token cookie.
  2. Sign it again for the response.
  3. Decrypt the session cookie, which is AES-GCM.
  4. Encrypt it again, because Rails’ cookie store writes a loaded session back on every response.

The code used the one-shot APIs, which are the clean way to write this in .NET: HMACSHA1.HashData(key, data) for the HMAC, and a new AesGcm(key) that was disposed after a single encrypt or decrypt.

What the profiler showed

On the room page, RestoreAuthenticationAsync was 6,778 of the controller’s 13,625 CPU samples, about half of all the work the controller did. Many of those samples had no managed method under them. They sat in methods like FindSessionByCookieAsync (1,730 samples), CookieSession.Commit (2,028) and MessageEncryptor.Decrypt and Encrypt (780 and 566), plus 532 in EvpCipherCreatePartial, which is the .NET wrapper creating an OpenSSL cipher context.

Samples with no managed callee usually mean native code, and here the native code was OpenSSL. On Linux, .NET’s crypto goes through OpenSSL, and with OpenSSL 3 a one-shot call sets up a fresh context every time: fetch the algorithm, allocate the context, load the key, run the operation, free it all. For a cookie of a few hundred bytes, the setup is as much work as the operation.

A microbenchmark on the image’s runtime (OpenSSL 3.5.7, messages of 160 to 300 bytes) put numbers on it:

Operation One-shot Reused context
HMAC-SHA1 1,203 ns 580 ns
AES-GCM encrypt 1,734 ns 649 ns

The lesson

A one-shot API is a good default because it’s hard to misuse: nothing to dispose, nothing shared between threads. But it has to do its setup on every call, and the documentation won’t tell you how much that costs on your platform. When you call one on every request, measure the setup separately from the work.

The same applies well beyond crypto: a new HttpClient per call, a Regex constructed in a method, a serializer options object built for each call, a database connection opened without pooling.

The fix

ThreadContexts keeps one HMAC-SHA1, one HMAC-SHA256 and one AES-GCM context per thread, and reuses it for as long as the thread asks for the same key:

static T For<T>(ref Slot<T>? slot, byte[] key, Func<byte[], T> create, int variant = 0)
    where T : class, IDisposable
{
    if (slot is { } held && held.Variant == variant && held.Key.AsSpan().SequenceEqual(key))
    {
        return held.Context;
    }
    slot?.Context.Dispose();
    slot = null;
    var copy = (byte[])key.Clone();
    slot = new Slot<T>(copy, variant, create(copy));
    return slot.Context;
}

The slots are [ThreadStatic], so no context is ever shared between threads, and no lock is needed. Keys are compared by content, not by reference, because each request builds its cookie jar with its own copies of the keys. A different key replaces the held context. The HMAC uses IncrementalHash, whose GetHashAndReset leaves it ready for the next message.

The output is byte for byte what the one-shot APIs produced, and the tests check exactly that: HMAC digests match the one-shot ones as keys and digests alternate on a thread, a key changed in place isn’t mistaken for the held one, AES-GCM still decrypts after a tag mismatch, two encryptors with different keys on one thread keep their own keys, and eight threads sharing a verifier and an encryptor get correct round trips.

After the change, FindSessionByCookieAsync’s own samples fell from 1,730 to 73.

The result

Route Quick A/B Benchmark harness
Room page +11.7% +13.8%
Messages page +11.8% +11.9%
Sidebar +13.0% +10.1%
Search +12.5% +13.3%
Post a message +0.4% 0.0%

The run with a writer posting 10 messages a second in the background gained 12 to 14% on the same pages.

Why three times the prediction?

The microbenchmark saved about 600 to 1,100 ns per operation. Scaled up to four operations per request, that predicted roughly a third of the gain we measured.

My guess is contention. The microbenchmark ran on one thread. In the server, four threads were creating and freeing contexts at the same time, and OpenSSL 3 takes locks shared by the whole process during some of that setup. If those locks were contended, each one-shot call cost more in the server than it did on one thread, and reusing contexts removed that contention as well as the setup. I haven’t verified this; it fits the numbers, and that’s all I can say for it.

The general point stands either way: a single-threaded microbenchmark measures the work, not the contention, so it can underestimate a fix by a lot. The A/B on the real server under real concurrency is the number to trust.

Check your own app

  • List the crypto your requests do: cookie signing and encryption, token validation, password checks, hashing for ETags or cache keys. Count the operations per request.
  • Look for samples in native code with no managed callee under authentication or cookie methods.
  • Check whether you’re on Linux with OpenSSL 3. Windows uses CNG, and the cost of a one-shot call is different there.
  • If you cache contexts per thread, compare keys by content, never share a context between threads, and test that the output matches the one-shot API exactly.
  • Measure the fix on the server, under its real concurrency.

Try it in LINQPad

This script compares the one-shot calls with reused per-thread contexts, running on every core at once. Add System.Security.Cryptography and System.Threading.Tasks to the query’s namespaces.

var key = RandomNumberGenerator.GetBytes(32);
var message = RandomNumberGenerator.GetBytes(250); // about the size of a session cookie
var mac = new byte[20];
var nonce = new byte[12];
var cipher = new byte[message.Length];
var tag = new byte[16];

// One-shot: every call sets up a new context, uses it once and throws it away.
// Reused: one context per thread, kept for as long as the key stays the same.
var hmac = new ThreadLocal<IncrementalHash>(() => IncrementalHash.CreateHMAC(HashAlgorithmName.SHA1, key));
var aes = new ThreadLocal<AesGcm>(() => new AesGcm(key, 16));

new
{
    HmacOneShot = Time(() => HMACSHA1.HashData(key, message, mac)),
    HmacReused = Time(() => { hmac.Value!.AppendData(message); hmac.Value.GetHashAndReset(mac); }),
    AesGcmOneShot = Time(() => { using var gcm = new AesGcm(key, 16); gcm.Encrypt(nonce, message, cipher, tag); }),
    AesGcmReused = Time(() => aes.Value!.Encrypt(nonce, message, cipher, tag)),
}.Dump($"Nanoseconds per call, {Environment.ProcessorCount} threads");

double Time(Action call, int perThread = 200_000)
{
    var threads = Environment.ProcessorCount;
    Parallel.For(0, threads, _ => { for (var i = 0; i < 1_000; i++) call(); }); // warm up every thread
    var clock = Stopwatch.StartNew();
    Parallel.For(0, threads, new ParallelOptions { MaxDegreeOfParallelism = threads }, _ =>
    {
        for (var i = 0; i < perThread; i++) call();
    });
    return Math.Round(clock.Elapsed.TotalNanoseconds / perThread); // wall time per call on each thread
}

Reusing a nonce is fine for a timing test and never fine for real encryption. On my Windows machine, with 16 threads, the HMAC barely changed (about 2,100 ns against 1,900) and AES-GCM went from about 3,300 ns to 1,000. Windows uses CNG, not OpenSSL, so copy the code into a console app and run it on Linux to see the OpenSSL 3 numbers.

The next post is about the two biggest allocations in the app, and why removing them barely mattered.

csharp dotnet performance cryptography openssl hmac aes-gcm linqpad