Skip to content

Content-Addressable Storage: SHA256 as the Key

Content-Addressable Storage: SHA256 as the Key

Written by:

Igor Gorovyy
DevOps Engineer Lead & Senior Solutions Architect

LinkedIn

In Meadow, every blob is stored by its SHA256 hash. The file name = the hash of its content. This gives you deduplication (identical content is stored once) and verification (hash doesn't match = data is corrupted).

Directory structure

data/
  blobs/
    sha256/
      a1b2c3d4e5f6...   <- layer blob
      f6e5d4c3b2a1...   <- config blob
  repositories/
    myapp/
      manifests/
        latest           <- manifest JSON
        latest.content-type
        sha256_abc...    <- manifest by digest
  uploads/               <- temp files

Writing a blob

func (s *Storage) PutBlob(r io.Reader) (string, int64, error) {
    s.mu.Lock()
    defer s.mu.Unlock()

    // Write to a temp file and compute the hash simultaneously
    tmpFile, _ := os.CreateTemp(
        filepath.Join(s.baseDir, "uploads"), "blob-*")
    tmpPath := tmpFile.Name()

    h := sha256.New()
    size, err := io.Copy(io.MultiWriter(tmpFile, h), r)
    tmpFile.Close()
    if err != nil {
        os.Remove(tmpPath)
        return "", 0, err
    }

    digest := "sha256:" + hex.EncodeToString(h.Sum(nil))
    _, alg, hexStr := parseDigest(digest)
    blobPath := filepath.Join(s.baseDir, "blobs", alg, hexStr)

    os.MkdirAll(filepath.Dir(blobPath), 0755)
    // Atomic rename
    if err := os.Rename(tmpPath, blobPath); err != nil {
        copyFilePath(tmpPath, blobPath)
        os.Remove(tmpPath)
    }

    return digest, size, nil
}

Here's the neat part: io.MultiWriter(tmpFile, h) writes to the file and computes the SHA256 at the same time. No need to read the file twice.

Atomic write: we first write to a temp file (in uploads/), then os.Rename moves it to the final location. If the process crashes mid-write, the temp file stays in uploads, but the blob won't be corrupted.

graph LR
    A["HTTP Body"] --> B["io.MultiWriter"]
    B --> C["temp file<br/>uploads/blob-xxx"]
    B --> D["sha256 hasher"]
    D --> E["digest = sha256:a1b2c3..."]
    C -->|"os.Rename (atomic)"| F["blobs/sha256/a1b2c3..."]

Checking existence

func (s *Storage) HasBlob(digest string) bool {
    _, alg, hex := parseDigest(digest)
    path := filepath.Join(s.baseDir, "blobs", alg, hex)
    _, err := os.Stat(path)
    return err == nil
}

Before uploading, the client sends a HEAD request. If the blob already exists, the upload can be skipped. That's deduplication: if two images share the same layer, it's stored once.

Manifest storage

Manifests are stored both by tag and by digest:

func (s *Storage) PutManifest(repo, ref string,
    data []byte, contentType string) (string, error) {
    h := sha256.Sum256(data)
    digest := "sha256:" + hex.EncodeToString(h[:])

    // Store by tag
    path := s.manifestPath(repo, ref)
    os.WriteFile(path, data, 0644)
    os.WriteFile(path+".content-type",
        []byte(contentType), 0644)

    // And by digest
    digestPath := s.manifestPath(repo, digest)
    os.WriteFile(digestPath, data, 0644)
    os.WriteFile(digestPath+".content-type",
        []byte(contentType), 0644)

    return digest, nil
}

The Content-Type is stored in a sidecar file (.content-type). OCI manifests and Docker manifests have different media types.

Why content-addressable is better

  1. Deduplication — same layer = one file
  2. Verification — download a blob, compute the hash, compare with the digest
  3. Immutability — a blob can't be "updated", you can only add a new one
  4. Caching — if the digest is the same, the content is guaranteed to be the same

Honestly about the downsides

There's no garbage collection. A deleted manifest leaves blobs on disk. Over time, storage grows. Docker Registry has a garbage collector that removes unreferenced blobs.

  • os.Rename is only atomic within a single filesystem. If uploads/ and blobs/ live on different volumes, the rename fails — which is why the code has a copy fallback. But copy isn't atomic: a crash mid-copy leaves a corrupted blob.
  • Content-addressing on its own doesn't protect against collision attacks on a weak hash. SHA256 is safe for now, but SHA-1 (which Git used for years) has already been broken — which is why OCI is pinned to SHA256, not to "any hash".

💡 Fun facts

  • Content-addressable storage isn't a Docker invention. Git has done the same thing since 2005: every object is stored by the hash of its content. Docker essentially took the same idea and applied it to image layers.
  • Splitting the hash into subdirectories (blobs/sha256/ab/abcd... in real registries) isn't aesthetics — it's an escape from a filesystem limit: tens of thousands of files in one directory cripple performance on ext4 and many other filesystems.
  • The very property that gives you deduplication makes it impossible to quietly "overwrite" a tag in place: change the content, and the digest changes. That's exactly why a supply-chain attack is repointing a tag at a different digest, not swapping the blob itself.
  • The .content-type sidecar file is needed because the filesystem doesn't store a MIME type. A Docker manifest and an OCI manifest are byte-different even for the same image — and the client picks the format via the Accept header.
  • The term "content-addressable" didn't come from storage — it came from hardware: content-addressable memory (CAM) in network switches looks up a row by its content, not its address. Addressing data by its content predates Docker by decades.
  • Docker only moved to content-addressable image IDs in version 1.10 (February 2016). Before that, image IDs were random UUIDs — the source of the infamous "same image, different ID" confusion around docker save/load.
  • The same idea scales to an entire network: IPFS addresses every file by a CID (a hash of its content), Nix addresses every build artifact by a hash of its inputs, and ZFS/Btrfs keep a checksum per block. Content-addressing isn't a registry feature — it's a whole class of architectures.
  • SHA256 is 256 bits = 64 hex characters. The odds of a collision across any realistic number of blobs are so small (the birthday bound) that it's cheaper to assume a hardware failure than a hash clash.

What I figured out while digging into this

At first I thought io.MultiWriter(tmpFile, h) was just a tidy trick to avoid reading the file twice. Then it hit me: without it you literally can't compute the digest honestly from a stream — because I have to write the data to disk BEFORE I know its hash, i.e. BEFORE I know the final file name. Hence the whole dance with the temp file and atomic rename. It's not an optimization — it's the only correct order of operations.

What could be improved

  • Add garbage collection: mark-and-sweep over manifests to delete unreferenced blobs (the way registry garbage-collect does).
  • Reference-count blobs so a deduplicated layer isn't deleted while at least one image still references it.
  • Verify the digest on read (GetBlob) too, not just on write — content-addressing won't catch bit rot on disk by itself until someone explicitly re-checks the hash.

Try it yourself

# After a push, check the storage:
ls /tmp/meadow-data/blobs/sha256/ | head -5
ls /tmp/meadow-data/repositories/
# Blob size:
du -sh /tmp/meadow-data/blobs/

# The exact same principle in Git — watch it compute a digest:
echo -n "hello meadow" | git hash-object --stdin
# and pull the object back out by hash:
git cat-file -p <hash>

Storage is clear. Next up — Pull from Docker Hub: auth, manifest lists, and multi-arch.

Resources

Source code for the series: github.com/igorgorovoy/sheep-shepherd-meadow

Previous: OCI Distribution Spec | Next: Pull from Docker Hub