Skip to content

JSON Serialization for State Persistence

JSON Serialization for State Persistence

Written by:

Igor Gorovyy
DevOps Engineer Lead & Senior Solutions Architect

LinkedIn

JSON is everywhere in Sheep and Shepherd: state.json for every container, manifest.json for every image, BoltDB stores resources as JSON bytes, the entire API uses JSON request/response. One format for everything.

Four Places Where JSON Lives

graph TB
    subgraph "1. Files on disk"
        STATE["containers/abc/state.json<br/>container state"]
        MANIFEST["images/def/manifest.json<br/>image metadata"]
    end

    subgraph "2. BoltDB"
        DB["shepherd.db<br/>key: bytes -> value: JSON bytes"]
    end

    subgraph "3. HTTP API"
        REQ["Request body: JSON"]
        RESP["Response body: JSON"]
    end

    subgraph "4. OCI Registry"
        OCI_MAN["OCI manifest: JSON"]
        OCI_CFG["Image config: JSON"]
    end

Container state.json

Every container state change gets written to disk:

func (m *Manager) saveState(c *Container) error {
    dir := filepath.Join(m.baseDir, "containers", c.ID)
    os.MkdirAll(dir, 0755)

    data, err := json.MarshalIndent(c, "", "  ")
    if err != nil {
        return err
    }

    return os.WriteFile(
        filepath.Join(dir, "state.json"), data, 0644)
}

MarshalIndent with two spaces. You can open the file and read it:

{
  "id": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4",
  "name": "web",
  "image": "nginx",
  "command": ["nginx", "-g", "daemon off;"],
  "state": "running",
  "pid": 12345,
  "exit_code": 0,
  "created_at": "2026-04-28T10:30:00.123456Z",
  "started_at": "2026-04-28T10:30:01.456789Z",
  "config": {
    "hostname": "a1b2c3d4e5f6",
    "env": ["PORT=8080", "WORKER_COUNT=4"],
    "work_dir": "",
    "memory": 268435456,
    "cpu_shares": 0,
    "cpu_quota": 0,
    "pids_limit": 100
  },
  "rootfs": "/var/lib/sheep/overlay/a1b2c3d4.../merged",
  "mounts": [
    {
      "source": "/data",
      "target": "/app/data",
      "readonly": true
    }
  ],
  "network": {
    "ip_address": "10.20.0.2",
    "gateway": "10.20.0.1",
    "bridge": "sheep0",
    "veth_host": "vetha1b2c3d4",
    "veth_guest": "eth0",
    "mac_address": ""
  }
}

JSON Struct Tags

Go uses tags to control serialization:

type Container struct {
    ID        string    `json:"id"`
    Name      string    `json:"name"`
    State     State     `json:"state"`
    Pid       int       `json:"pid"`
    ExitCode  int       `json:"exit_code"`
    CreatedAt time.Time `json:"created_at"`
    StartedAt time.Time `json:"started_at,omitempty"`
    StoppedAt time.Time `json:"stopped_at,omitempty"`
    Mounts    []Mount   `json:"mounts,omitempty"`
    Network   *NetworkSettings `json:"network,omitempty"`
}

json:"exit_code" - snake_case in JSON, CamelCase in Go. A convention the Go community adopted.

omitempty - don't include the field if the value is zero. StartedAt won't appear in JSON for a created container. Network (pointer) won't appear if nil. This keeps the JSON clean.

Here's a neat thing about time.Time: Go serializes it as an RFC3339 string automatically. "2026-04-28T10:30:00.123456Z" - a human-readable format.

Loading on Startup

func (m *Manager) loadExisting() error {
    dir := filepath.Join(m.baseDir, "containers")
    entries, err := os.ReadDir(dir)
    if err != nil {
        if os.IsNotExist(err) {
            return nil // first run, no containers yet
        }
        return err
    }

    for _, e := range entries {
        if !e.IsDir() { continue }

        data, err := os.ReadFile(
            filepath.Join(dir, e.Name(), "state.json"))
        if err != nil { continue }

        var c Container
        if err := json.Unmarshal(data, &c); err != nil {
            continue // skip corrupted files
        }

        // Check if previously running containers are still alive
        if c.State == StateRunning && c.Pid > 0 {
            if !isProcessAlive(c.Pid) {
                c.State = StateStopped
                c.Pid = 0
            }
        }

        m.containers[c.ID] = &c
    }
    return nil
}

Corrupted state.json? Skip it. Container says "running" but the process is gone? Set it to stopped. Defensive deserialization.

BoltDB - JSON as Bytes

BoltDB stores []byte key -> []byte value. We serialize Go structs to JSON:

func (s *Store) put(bucket []byte, key []byte, v any) error {
    data, err := json.Marshal(v)
    if err != nil {
        return err
    }
    return s.db.Update(func(tx *bolt.Tx) error {
        return tx.Bucket(bucket).Put(key, data)
    })
}

func (s *Store) get(bucket []byte, key []byte, v any) error {
    return s.db.View(func(tx *bolt.Tx) error {
        data := tx.Bucket(bucket).Get(key)
        if data == nil {
            return fmt.Errorf("not found")
        }
        return json.Unmarshal(data, v)
    })
}

func (s *Store) list(bucket []byte, prefix string,
    fn func([]byte) error) error {
    return s.db.View(func(tx *bolt.Tx) error {
        b := tx.Bucket(bucket)
        return b.ForEach(func(k, v []byte) error {
            if prefix == "" ||
                strings.HasPrefix(string(k), prefix+"/") {
                return fn(v) // v is JSON bytes
            }
            return nil
        })
    })
}

The key for pods: default/web-0 (namespace/name). The value: the entire Pod as JSON.

API - Streaming JSON

// Request: streaming decode from body
func (api *APIServer) createPod(w http.ResponseWriter,
    r *http.Request) {
    var pod Pod
    if err := json.NewDecoder(r.Body).Decode(&pod); err != nil {
        httpError(w, http.StatusBadRequest,
            "invalid pod: %v", err)
        return
    }
    // ...
}

// Response: streaming encode to writer
func respondJSON(w http.ResponseWriter, status int, v any) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(status)
    json.NewEncoder(w).Encode(v)
}

NewDecoder(r.Body) reads from the HTTP body directly, without buffering everything into memory. NewEncoder(w) writes directly to the HTTP response.

Image manifest.json

func (im *ImageManager) saveMetadata(id string, img *Image) error {
    data, err := json.Marshal(img)
    if err != nil {
        return err
    }
    return os.WriteFile(
        filepath.Join(im.baseDir, id, "manifest.json"),
        data, 0644)
}

Here we use Marshal without Indent - the manifest isn't for human reading, only for the machine. Smaller file.

Why JSON Instead of protobuf/msgpack

JSON Protobuf
Schema Implicit (struct tags) Explicit (.proto file)
Readability cat file.json protoc --decode
Size Larger Smaller (~30-50%)
Speed Slower Faster (~2-5x)
Dependencies stdlib google.golang.org/protobuf

Kubernetes uses protobuf for internal communication and JSON for the API. We use JSON everywhere because: - encoding/json is in stdlib - cat state.json for debugging - For dozens of objects the speed difference is invisible

What We Simplified

json.Marshal/Unmarshal use reflection, which is slow. For thousands of serializations per second, look into json.NewEncoder (streaming) or code generation libraries like easyjson. For Sheep with dozens of containers, reflection is fine.

💡 Fun facts

  • JSON was invented by Douglas Crockford in the early 2000s - not as a new format, but as a subset of JavaScript's object syntax. He deliberately left out comments and versioning so the format couldn't be "extended" and break compatibility.
  • Go serializes a map with sorted keys - deterministically, unlike many languages. So two Marshal calls on the same map produce byte-for-byte identical output, which is handy for diffs and hashing.
  • A []byte in Go marshals not as an array of numbers but as a base64 string. It's a surprise almost everyone trips over - binary data in JSON always costs ~33% in size overhead.
  • Kubernetes stores all objects in etcd, and although the API serves JSON, internally it uses protobuf for speed. JSON there is the "human" facade over a binary core - exactly the same "one format for convenience, another for speed" idea.

What I figured out while digging into this

The biggest insight: omitempty is treacherous. It drops a field when the value is "zero", but in Go "zero" and "unset" are the same thing. So replicas: 0 disappears from the JSON just like an unset field, and then you can't tell "I want zero replicas" from "I didn't specify". I saw this on my own state.json and finally understood why Kubernetes APIs reach for pointers (*int32) so often instead of plain numbers - it's the only way nil differs from zero. A tiny detail in a tag, but the semantics are completely different.

What to watch out for

  • omitempty doesn't distinguish "zero" from "unset". If a zero value is meaningful to you (0 replicas, false, empty string), use a pointer or a custom Marshaler.
  • json.Unmarshal silently ignores unknown fields. An old state.json with a field that no longer exists in the struct loads without error - convenient, but it hides schema drift. Decoder.DisallowUnknownFields() turns on strict checking.
  • MarshalIndent on every state write means extra allocations and a bigger file. On frequent state updates it adds up.
  • No schema versioning: if you rename a field tomorrow, old files on disk silently lose their value on the very first load.

What could be improved

  • Add a schemaVersion field to every state.json and manifest, so a struct change lets you migrate old files instead of silently losing data.
  • Atomic writes: write to state.json.tmp and os.Rename - otherwise a crash mid-WriteFile leaves a half-written, corrupt JSON.
  • Replace reflection-based Marshal with generated code (easyjson/ffjson) on hot paths if profiling shows serialization became a bottleneck.
  • Consider custom MarshalJSON/UnmarshalJSON for types like State, to validate values on input rather than accepting any string.

Try It Yourself

# Look at a container's state.json:
cat /var/lib/sheep/containers/*/state.json | jq .
# Image manifest:
cat /var/lib/sheep/images/*/manifest.json | jq .
# API response:
curl -s localhost:9876/api/v1/pods | jq '.[0]'

Starting the final series - Architecture & Design.

Resources

Source code for the series: github.com/igorgorovoy/sheep-shepherd-meadow

Previous: CLI Without Frameworks | Next: Docker vs Sheep