Table of Contents

Azure Blob Storage

Coming in 4.0

Resourcefull.AzureBlob stores resources in a blob container of an Azure Storage account. Files stream in and out with flat memory use, and every name keeps its versions, content hashes and metadata, as with the database providers. It's tested on every change against Azurite, Microsoft's local emulator for Azure Storage.

dotnet add package Resourcefull.AzureBlob

using Azure.Storage.Blobs;
using Resourcefull;
using Resourcefull.AzureBlob;

builder.Services.AddResourcefull(options => options.UseAzureBlobStorage("files", azure =>
    azure.Client = new BlobServiceClient(builder.Configuration.GetConnectionString("Storage"))));

The blob container, files here, must exist; Resourcefull doesn't create it. Every Resourcefull container is kept inside it, so Resourcefull's container names aren't bound by Azure's rules for naming blob containers.

What it supports

Uploads, downloads (with ranges), listing, versions, create-only and conditional writes, conditional deletes, copies, moves and renames, container and version metadata, container policies, export and import, expiry and upload sessions. It passes the same contract tests as the other providers, so switching to it, or from it, takes only the line that picks the provider. Provider features compares the providers.

It works the same way as the S3-compatible provider, and shares its code: only the calls to the store differ.

Connecting to an account

Give UseAzureBlobStorage a BlobServiceClient for the account. Any way of creating one works:

  • A connection string, as above. For Azurite on your own machine, it's UseDevelopmentStorage=true.

  • Microsoft Entra ID, with no secret to store, using the Azure.Identity package:

    using Azure.Identity;

options.UseAzureBlobStorage("files", azure => azure.Client =
    new BlobServiceClient(new Uri("https://myaccount.blob.core.windows.net"), new DefaultAzureCredential()));

  • A client registered in your application's services, such as one Aspire or Microsoft.Extensions.Azure registers. Leave Client unset, and Resourcefull uses the registered BlobServiceClient. If there's none either, the first use of Resourcefull throws InvalidOperationException, saying to set one.

Keep connection strings and keys out of source control, in user secrets, environment variables or Key Vault.

Permissions

Resourcefull only reads, writes, lists and deletes blobs: it never creates or deletes blob containers, or changes the account's settings. With Microsoft Entra ID, the narrowest built-in role that allows that is Storage Blob Data Contributor; give it to the application on the blob container rather than the whole account.

Completing an upload session has Azure copy the session's segments into place, reading them through a read-only link the provider signs. With an account key, as in a connection string, it signs with that key and needs nothing more. With Microsoft Entra ID, it signs with a user delegation key, which needs permission on the storage account itself: give the application the Storage Blob Delegator role on the account, which allows only that. A client made from a URL with a SAS in it reads with that SAS, which then needs read permission too. A client given a SAS as an AzureSasCredential can't sign links, so it can't complete sessions: put the SAS in the URL instead.

Sharing a blob container

Every blob Resourcefull writes has a name starting with resourcefull/, so the blob container can hold other things too. To keep several stores in one blob container, such as one per environment, give each a prefix of its own:

options.UseAzureBlobStorage("files", azure =>
{
    azure.Client = client;
    azure.KeyPrefix = "staging/resourcefull/";
});

Don't let anything else write, change or delete blobs under Resourcefull's prefix, and give every instance of the application the same blob container and prefix. A prefix can be up to 190 characters long; an empty one puts Resourcefull's blobs at the top of the blob container.

Account settings

  • Blob versioning isn't needed, and is better left off for the account: Resourcefull rewrites a small blob for every change to a name, and with versioning on, Azure keeps every old copy of it, and bills for them.
  • Soft delete keeps deleted blobs, and with versioning off the old copy of every overwritten one, for the number of days you choose, so they still cost storage after Resourcefull deletes or rewrites them. Since every change to a name rewrites its head, that adds up on busy names. It's a safety net if you want one; just count it in the bill.
  • Hierarchical namespace (Data Lake Storage) isn't supported: it treats / in blob names as folders, which changes the order blobs are listed in.

How it stores things

You don't need this to use the provider, but it explains its costs and what you'll see in the blob container.

  • Each container has a record, {prefix}c/{name}, holding a random ID and the container's metadata. Everything the container holds is kept under that ID, at {prefix}d/{id}/.
  • Each name has a head, a small JSON blob listing its versions' attributes in the order they were stored; the last is current. Every change to a name (an upload, a delete or a metadata change) reads the head and writes it back only if nobody changed it meanwhile (If-Match with the ETag it read, or If-None-Match: * for a new name). When somebody did, it reads it again and retries. Each write carries a random ID, so when an answer is lost and Azure refuses the resent write, Resourcefull can tell its own write landed.
  • Each version that expires has an entry in an index under {prefix}x/, keyed by when it expires, so the expiry sweep lists only what's due rather than reading every head. The entry goes in just before the head write that sets the expiry, so a version never has an expiry the index doesn't know about. Each entry's key holds a random token that the version records, so no two writes share a key. Changing an expiry writes a new entry, then deletes the old one once the head no longer names it; deleting a version deletes its entry with its content. An entry those deletes miss, such as one a failed write left, is dropped by a sweep that finds it due and over an hour old; a younger one may belong to a write still in progress.
  • Each version in a bin 4.0 stays in its name's head, separately from the name's versions, and its content stays where it was. It has two entries, written just before the head write that puts it in the bin: one under {prefix}d/{id}/t/, newest deletion first, whose key holds its content's length, for listings of the bin and the usage figures, and one under {prefix}t/, keyed by when it's purged, for the sweep. Both are checked against the head when read, and one whose version is no longer in the bin is dropped once it's over an hour old, as expiry entries are; the daily scan for content no head names also drops stale bin entries that no listing reaches. A name left with only a bin has a mark under {prefix}d/{id}/e/, so the usage figures can leave it out of the names, and the same scan fixes any mark a failed write left wrong.
  • Each version's content is a blob of its own, uploaded before the head names it, and deleted after the head stops naming it.
  • A move writes an entry for it under {prefix}m/, marks both names' heads with its ID, then commits that entry with one conditional write: until then, every read finds the versions at the source, and from then at the destination, so they're never in both places or in neither. Across containers, Azure first copies each version's content under the destination container's ID, in parts of up to 1 GiB, so the destination's usage counts it and deleting the source container doesn't take it; within a container, nothing is copied. Each version that expires gets an expiry entry for its new name. A write to either name while the move is still pending abandons it, so the mover's commit is refused and it starts again. Once a move is decided, the mover clears the marks, deletes the entry and, across containers, the content left behind.
  • Each upload session has a record, {prefix}u/{container ID}/{upload ID}!/r, holding the version's attributes and the list of its segments, and one blob per segment beside it. Sessions are kept apart from the containers' data, so they don't count toward a container's usage. An append stores its content in segments of up to 64 MiB, adding each to the record as soon as it's stored, so an append that fails partway keeps what it stored. Each change to the record is a conditional write, as for heads, so two appends from the same offset can't both land. A last segment shorter than 8 MiB is read back and stored again with the start of the next append, so every segment but the last is at least 8 MiB, which keeps a session's segment count, and so its blocks, down. Completing marks the record sealed, so nothing more can be appended, then has Azure copy each segment into a block of the version's content blob (Put Block From URL) and commits the blocks; the bytes don't pass through your application. Then it adds the version to the head and deletes the session.

In blob names, the space, the punctuation from ! to /, [ and \ are escaped, and a ! marks where each name ends, so a name such as ../report.pdf or a//b is kept exactly as given. The escaping keeps the names' order, so listings come straight from the order Azure lists blobs in. Against Azurite, that's .NET's ordinal order, which the contract tests check. Azure documents only that it lists blobs alphabetically, uppercase letters first, so names with characters beyond U+FFFF, such as emoji, or from U+E000 to U+FFFF, might list in a different order there.

What stays behind

  • A deleted container keeps its record, marked deleted, and a name whose versions are all deleted keeps an empty head. Both are a few bytes each, and are reused if the name is created again.
  • Deleting a container hides everything in it at once, then deletes its blobs a thousand at a time. If that stops partway, deleting the container again finishes it, and so does creating a container of that name again. Its versions' expiry entries are kept apart from it, under {prefix}x/, so they stay until they're due, and a sweep then drops them.
  • A write interrupted after its content was uploaded, such as by a crash, leaves that content named by no head, and so does a delete that fails to delete its content. It counts toward the container's stored size until the expiry sweep deletes it: once a day, a sweep reads every head and deletes the content that it and the sweep the day before both found named by no head and over a day old. Two scans are needed because one reads the heads one after another, so a version renamed meanwhile could look unnamed to it. The same daily scan also finishes purging any container whose delete stopped partway.
  • An upload of 8 MiB or more that fails partway leaves the blocks it sent uncommitted, and so does a completion whose join fails. Azure discards uncommitted blocks on its own after a week.
  • A move that stops partway, such as by a crash, leaves its marks and its entry. Every read still finds the versions in one place, at the cost of one more read for a name a move marked, and the first expiry sweep once the move has made no progress for an hour finishes or undoes it. Until then, the resource counts under both names, in each one's container's number of resources. A move across containers writes its entry again every 15 minutes while it copies, so a long copy doesn't make the move look stopped.
  • A deleted container's upload sessions stay under {prefix}u/ until the next expiry sweep, which deletes them. So does a session's segment whose delete failed, once it's an hour old.

Costs

  • An upload smaller than 8 MiB reads the container's record, sends the content in one request, reads and writes the head, then reads the container's record again to catch a container deleted meanwhile. A create-only or conditional upload, which includes the default mode, also reads the head before sending the content. From 8 MiB up, content goes up in 8 MiB blocks, each read into a buffer first, so each upload in progress holds 8 MiB of memory. Content larger than 390 GiB, the most one upload's blocks hold, goes up in pieces of 390 GiB. The store then joins them, one copy request per GiB, and the pieces are deleted. Until they're joined, a small record under {prefix}p/ lists them, written again every 15 minutes, so the daily scan for unnamed content leaves them alone. An upload holds up to about 48 TiB; a larger one throws ResourceUploadLimitException, which ASP.NET Core's handler answers with 413 Content Too Large.
  • A download reads the container's record and the head, then streams the content; a range asks Azure for only that range.
  • A listing page reads the container's record, lists the heads, then reads each one, several at a time, since the attributes are inside them.
  • Usage, which container policies check before each write they limit, comes from listings alone: the content blobs' sizes, and the number of non-empty heads. That's one list request per five thousand blobs in the container.
  • An expiry sweep with nothing due is three list requests (of expiry entries, upload sessions and moves) and one read, of the record of the last scan for unnamed content. Once a day, the sweep runs that scan: it lists every container's record, head and content blob, and reads every move's entry, record and head, several at a time. Each expired version it finds costs three reads to find it (its index entry, its container's record and its head), two more to delete it (the container's record and the head again), a write of the head, and two deletes, of its content and its entry. A version with an expiry costs one more small blob, and an upload with an expiry one more write, for its entry. Changing an expiry reads the container's record and the head, writes the new entry (none when clearing it), reads and writes the head, then deletes the old entry.
  • A rename reads both containers' records and both heads, writes the move's entry, marks the source head, writes the destination's, and commits the entry; then it reads and writes each head again to clear the marks, deletes the entry, and reads the destination container's record: about eight reads, six writes and two deletes. A version with an expiry costs one more write, for its new entry. A move to another container also has the store copy each version's content, one request per GiB, and writes the move's entry again every 15 minutes while it copies and after each version; then it deletes the old blobs. While a move is unfinished, a read of either name costs one more read, of the move's entry.
  • An upload session costs, to start, a read of the container's record and a write of the session's record. Each append reads both, then for each segment it stores (one per 64 MiB) uploads the segment and reads and writes the record. An append that starts after a segment shorter than 8 MiB first reads that segment back, so many small appends each cost up to 8 MiB more, each way. Completing reads the content once to hash it, as with every provider; then it reads and writes the record to seal it, reads the head, has Azure copy each segment into a block of the version's content (several at once) and commits them, and writes the head; then it deletes the session's blobs. A session holds up to about 48 TiB, as an upload does. A blob holds at most 50,000 blocks, so once a session's segments take that many, the next append first has Azure join those stored since the last such join into one segment. That copies them, though the bytes stay in the store, and each byte is copied about once. With appends in multiples of 64 MiB, the first join comes after about 3 TiB, and later ones sooner, since a joined segment counts as one part for each GiB it holds. An append past about 48 TiB throws ResourceUploadLimitException, which ASP.NET Core's handler answers with 413 Content Too Large. The expiry sweep lists every session, and reads the record of each, to find those that nothing was appended to for a day.

Health checks

The generic health check, AddResourcefullCheck(), makes one read through the provider, so it reports whether the account can be reached and the blob container exists and is readable with the credentials given.

Telemetry

Besides the operations every provider records (see Telemetry), the provider counts resourcefull.azureblob.retries: writes to a name's head, or to an upload session's record, retried because another writer changed it first. A few are normal when writers share names; a steady rise means many writers are contending for the same names. It counts resourcefull.azureblob.moves_recovered: moves the expiry sweep finished or undid because their mover stopped first, such as after a crash or a lost connection. It counts resourcefull.azureblob.session_joins: joins of an upload session's segments, made once they took as many parts as the store joins, each of which delays an append while the store copies what was appended since the last join. It also counts resourcefull.azureblob.stale_expiries: expiry entries a sweep dropped because their version no longer existed or had another expiry; resourcefull.azureblob.stale_bin_entries: bin and purge entries dropped because their version was no longer in its bin; and resourcefull.azureblob.abandoned_uploads: upload sessions nothing was appended to for a day, or whose container was deleted, and content no head names, which the expiry sweep deleted. The Azure SDK can also trace its own requests; see its documentation for OpenTelemetry.

Edit this page