I run a public API at MySafeInfo that anyone can call, no signup required, with API keys for heavier use. A while back, I looked at where the database was spending its time. More key lookups than I expected were for keys that did not exist. Typos, long expired keys, and bots guessing at random. The valid keys were cached and cost nothing. The garbage went straight to SQL Server, every single time.
The cause was one innocent looking line in my caching code. You have probably written this method yourself:
public async Task<ApiKey?> GetKeyAsync(string key)
{
if (_cache.TryGetValue(key, out ApiKey? cached))
return cached;
var apiKey = await _repository.GetKeyAsync(key);
if (apiKey is not null)
_cache.Set(key, apiKey, TimeSpan.FromMinutes(5));
return apiKey;
}
See it? Only successful lookups get cached. When the repository comes back empty, the method skips the cache and returns null, which means the next request for that same bad key does the whole trip again. A cache like this protects you from your best traffic and leaves you wide open to your worst.
The fix is called negative caching: store the miss too. IMemoryCache is happy to hold a null, so the change is small:
public async Task<ApiKey?> GetKeyAsync(string key)
{
if (_cache.TryGetValue(key, out ApiKey? cached))
return cached; // may be null, and that is fine
var apiKey = await _repository.GetKeyAsync(key);
var options = new MemoryCacheEntryOptions()
.SetAbsoluteExpiration(apiKey is null
? TimeSpan.FromMinutes(1)
: TimeSpan.FromMinutes(5))
.SetSize(1); // pairs with the size limit below
_cache.Set(key, apiKey, options);
return apiKey;
}
Now a bad key costs one database trip per minute instead of one per request. Notice the misses get a shorter lifetime than the hits, and there is a reason for the asymmetry. A cached hit going slightly stale is usually harmless. A cached miss going stale is a customer who just created a key and gets told it does not exist. Keeping the negative TTL short means new keys start working within a minute while the database still sleeps through the bot noise. Tune both numbers to your own traffic; the asymmetry is the part that matters.
Negative caching solves repeated misses, but it cannot help when a client invents a new key for every request. The short negative TTL handles repeated bad keys. A size limit protects the cache from the other kind of traffic: an endless stream of unique junk.
Be careful where you apply that limit. Microsoft's guidance on limiting cache size warns that setting SizeLimit on the shared cache registered by AddMemoryCache means every entry in that cache must specify a size, including entries created by code you may not control. I use a dedicated cache for API keys instead, and _cache in the fixed version above is that instance:
public sealed class ApiKeyCache
{
public MemoryCache Cache { get; } = new(
new MemoryCacheOptions
{
SizeLimit = 10_000
});
}
// in Program.cs
builder.Services.AddSingleton<ApiKeyCache>();
Give each cache entry a size of 1, which simply tells the cache to count it as one item. With a limit of 10,000, this dedicated cache can hold up to 10,000 entries. Valid keys and cached misses both count. Once the cache is full, new entries are not added, so it cannot grow without limit. This is a count of entries, not a 10,000-byte limit.
What happens when the cache reaches 10,000 entries? New entries are simply not added until existing entries expire or are removed. The request still works because it falls back to the database, but that particular result receives no caching benefit. The size limit protects memory; rate limiting is still needed to protect the database from a client sending a constant stream of unique random keys.
None of this is new. That is the point. Negative caching is a familiar pattern in DNS, HTTP, and other systems that answer the same questions all day. But it is easy to overlook in application code because that null check feels perfectly reasonable when you write it. Take a look at your own caching code. If you find that innocent little if statement, check what it is costing you.