Home Assistant OS Upgrade and Rollback

Q: Can I roll back HA OS itself after a bad update?

Yes, but only if you still have the old boot partition. HA OS keeps the previous install in the 'B' slot by default; a failed update usually auto-rolls to B on next boot. If the previous slot was overwritten (e.g. two updates ago), you fall back to a NandUSB-Save-Manager full-restore or reflash.

Q: Why does addon upgrade order matter?

HA Core reads discovery topics from Mosquitto at startup. If Mosquitto upgrades and briefly loses state while HA Core is also restarting, HA may drop MQTT entities and only repopulate them after discovery re-fires. Upgrading the broker first, letting it settle, then HA Core avoids the empty-entity window.

Q: Is HA Core version independent of HA OS version?

Yes. HA OS is the appliance (Linux + Docker + Supervisor). HA Core is a Python container running inside it. A given HA OS release supports a *range* of Core versions. They can drift, but only within the range the OS was tested with. Pinning Core and OS separately is a real ops move when you're fighting a regression.

Q: What breaks first in a bad HA update?

In my experience, in order of frequency: (1) a custom integration stopped by a Core API change, (2) addon startup ordering issues after restart, (3) HA Core migration error on the database, (4) full bootloop from config partition corruption. The first two are addon/backup recoveries; the last two are full-restore recoveries.

Q: How do I keep a stable HA version for months?

Set the Supervisor auto-upgrade policy to Manual for both HA Core and HA OS. Then install a fixed addon version via the addon's own UI (HACS does this for Core integrations). Auto-upgrades are a footgun in a 24/7 deployment.

Q: Does backing up /config alone cover addons?

It covers addon *config files* — not the addon's own data directory (Mosquitto's `mosquitto.db`, Z2M's network state). Those live under `/usr` and are only captured by Supervisor snapshots or NandUSB-Save-Manager.

My HA server has been on production for three years — roughly fourteen Core upgrades and three OS upgrades. Six of those sixteen required a rollback or hot-fix within forty-eight hours. Every one of those six taught me something the official changelog didn't mention.

The single most important thing I've learned: "Update now" is not one operation. It's four chained operations, and any one of them can fail independently. A successful update in every other part of the system does not make a failed fourth step acceptable. This is my runbook for treating that chain the way it deserves.

1. What Actually Happens When You Click "Update Now"

The button in Settings → System → Updates triggers four chained steps inside the Supervisor:

Boot config migration — the Supervisor reads the target HA OS version and stages a new boot config under /usr/local/share/rauc/mounts/host. Skipped if you're only updating Core.
Addon container migration — every installed addon image is pulled if its version hash changed. This includes addon data snapshots (Mosquitto mosquitto.db, Z2M zigbee2mqtt.db, Matter Server .matter-server).
State restore — the new HA Core container boots and runs homeassistant core run. The Python migration code reads SQLite and applies pending schema changes. This is where most bad upgrades break.
Addon restart cascade — the Supervisor restarts every addon in dependency order. Mosquitto up first, then MQTT clients, then Z2M, then HA Core's addon integrations reconnect.

Failures at each step behave differently. Step 1 auto-rolls back to the previous OS slot. Step 2 leaves partially-migrated addons. Step 3 can corrupt the database. Step 4 just leaves addons in "Failed" state until you intervene.

2. The 6-Step Pre-Upgrade Checklist

Six steps, roughly twenty minutes of hands-on work, catches ~80% of failure modes before they happen.

Step 1 — Check the target version against my addon stack.

# What versions am I on right now?
ha supervisor info | jq '{os, homeassistant, docker, supervisor, host_os}'

# What's my addon list?
ha addons list | jq -r '.addons[] | [.name, .version] | @tsv' | column -t

If an addon I depend on (Mosquitto, Z2M, Matter Server, ESPHome) has a "breaking change" tag in that release, I stop and read it before touching anything.

Step 2 — Take a Supervisor snapshot.

ha supervisor backup create \
  --name "pre-upgrade-$(date +%Y%m%d)" \
  --password "$BACKUP_PASS"

The only snapshot that captures addon data. A plain tar /config does not.

Step 3 — Tar /config off-box.

docker exec homeassistant tar czf /mnt/nas/ha/ha-config-$(date +%Y%m%d).tar.gz -C /config .

The Supervisor snapshot is local. If the disk dies mid-restore, that snapshot dies with it. This layer lives on NAS.

Step 4 — Screenshot every addon's current state. Version, config, port, environment variables. Addons auto-upgrade silently by default; a screenshot is the only evidence you had before.

Step 5 — Verify the addon order I'm about to trigger. See chapter 3.

Step 6 — Set my phone out of silent. A production HA server that needs human intervention at 3 AM is not fun without vibration.

3. Addon Upgrade Order

I upgrade addons in this specific order and it is not arbitrary:

Mosquitto Broker — first, always. HA Core's MQTT integration reads discovery topics on startup. If Mosquitto is on a fresh image with a clean data directory, HA Core will see "no entities" for the first few minutes. Starting with Mosquitto up and stable means by the time HA Core comes up, discovery is populated.
Network infrastructure addons — DNSMasq, Pi-hole, WireGuard, AdGuard. These need to be up before anything else starts making DNS requests.
Zigbee2MQTT / Matter Server — these take 30–60 seconds to rejoin the radio network. Get them up before HA Core so the HA Core UI doesn't briefly show "Zigbee: no controller".
HACS — because HACS manages third-party integrations. If HACS is upgraded while HA Core is running an old HACS, you get version mismatch errors in the HACS UI.
Home Assistant Core itself — last, once the substrate is stable.
Everything else — Home Assistant Cast, Node-RED, ESPHome, etc. These don't have ordering constraints.

The auto-upgrade setting on addons is the single most common cause of "I upgraded HA Core and something broke" reports I've seen. Someone's Mosquitto addon auto-upgraded at 3 AM, then someone clicked the Core upgrade button at 9 AM, and the two upgrades compounded. I disable auto-upgrade on every addon and set explicit versions.

4. Three Real Upgrade Failures

4.1 Core 2025.11 broke a custom integration

I had ha-dynamic-themes running on Core 2025.10.6. 2025.11 changed the ThemeManager API to require an async call. My custom theme broke on startup with AttributeError: 'ThemeManager' object has no attribute 'apply_theme'.

Rollback: I restored the previous Supervisor snapshot into a side-by-side instance (same box, different /config), verified the old theme worked, then pinned Core to 2025.10.12 and installed the updated integration. That gave me 48 hours to fix the actual bug before retaking Core 2025.11.

Lesson: pinned versions in HACS are the cheapest rollback you'll ever have. If your addon is behind a bugfix, pin it.

4.2 Bootloop after OS update

On the October OS update I got past step 1, partway through step 2, then the box stopped responding to SSH. The Supervisor log: Cannot allocate memory for addon 'core_homeassistant'.

Diagnosis: pulled the SD, booted from USB, read /var/log/journal. The config partition had a bad block the update's write amplification finally hit.

Recovery (Rank 3 in chapter 5): mount the SD, run fsck.ext4 -fy on /config, copy config back, reflash the previous OS version, restore my NandUSB-Save-Manager snapshot. Downtime: ~4 hours.

Prevention: HA OS on a SATA SSD with nightly SMART, and a full /usr + /config partition snapshot to NAS every Sunday. The SD card was the most underrated upgrade failure mode in my first two years.

4.3 Addon auto-upgrade killed Mosquitto during HA update

My Mosquitto addon was set to auto-upgrade. At 2:47 AM Tuesday it fetched a new image. At 2:53 AM I was updating Core. The two upgrades intersected; Mosquitto restarted, HA Core tried to restart, and the Supervisor's dependency graph got confused. Mosquitto ended up "Failed" with Address already in use: port 1883 — the old container still held the socket.

supervisorctl restart mosquitto cleared it, but I lost ~20 MQTT entities until discovery re-fired. Fix: auto_update: false on Mosquitto, and only trigger Core upgrades after a manual Mosquitto restart window.

5. Rollback Paths, Ranked

Ordered from least disruptive to most:

Rank 1 — Addon snapshot (30 seconds). Every addon with a data directory gets a snapshot. If Z2M drops your Zigbee network, restore that snapshot and Z2M is back. Only useful when the failure is isolated to one addon.

Rank 2 — HA config backup (5 minutes). Restore /config from the off-box tarball. Resets HA Core state, automations, entities — but not addon data. Use when a bad config edit broke something, not a bad version.

Rank 3 — NandUSB-Save-Manager full restore (30 minutes). My main tool. Install once, configure it to auto-backup to a USB stick or NAS every Sunday. To restore: mount the stick, run nandusbsm restore /var/config/nandusbsm/latest, reboot. Restores OS, Core, addons, addon data.

Rank 4 — Reflash from installer (1 hour). Download the HA OS image, write it with balenaEtcher or dd, boot, restore your NandUSB-Save-Manager snapshot on the fresh install. Use when the boot partition itself is corrupt.

Rank 5 — Accept it and upgrade forward. Sometimes the cleanest rollback is to skip past the broken version. If 2025.11 has a fix in 2025.11.3, upgrading forward is faster than rolling back to 2025.10 and re-upgrading in three weeks.

Rule: try Rank 1 first, escalate one level at a time, stop as soon as the failure is gone.

6. Core vs. HA OS Version Drift

HA OS and HA Core are versioned independently. A given HA OS release supports a range of Core versions — but only the versions the OS was tested with.

The symptom: my box was on HA OS 13.3 and Core 2025.11.2. When I upgraded Core to 2025.12, HA Core booted fine but the Supervisor started logging unsupported core version, features may be limited. Some features — the new Matter bridge, improved Zigbee OTA — were silently degraded because the Supervisor wouldn't coordinate with a Core version newer than the OS had been tested against.

Fix: upgrade the OS first. Rule of thumb: upgrade OS before Core when the Core version is at or near the top of the supported range for your current OS. Running Core 2025.6 on OS 12.0 is fine; running Core 2025.12 on OS 12.0 is a problem.

7. Staging Upgrades in Production: Four Rules

My HA server runs 24/7: Z2M for forty-plus devices, Mosquitto for a dozen MQTT clients, ~200 automations. Four rules:

Rule 1 — Upgrade window. Tuesday 23:00–02:00 local. Never weekends. Never working hours. The humans I can call are asleep, but I'm awake.

Rule 2 — Comms. Post to the home-automation channel: Starting HA upgrade at 23:15, ETA 45 min. Do not touch automations until 01:00. Announce every start and every completion. This catches the "someone edited an automation at 23:30" false-alarm class of incidents.

Rule 3 — Addon order. Chapter 3. Always.

Rule 4 — Verification. Chapter 8. All six smoke tests pass before I declare done.

8. Post-Upgrade Smoke Test: Six Checks

Run all six before declaring "done":

# 1. HA Core healthy?
ha core status

# 2. All addons started?
ha addons list | jq -r '.addons[] | select(.state != "started") | "\(.name): \(.state)"'
# (should return nothing)

# 3. MQTT broker up?
docker exec mosquitto mosquitto_sub -t 'homeassistant/#' -C 1 -W 5

# 4. Zigbee coordinator attached?
docker exec zigbee2mqtt curl -s localhost:8099/api/v1/health

# 5. Entities restored? (entity count should match pre-upgrade count)
ha rest -X POST /api/states

# 6. DB healthy?
ha rest -X GET /api/config/version | jq .core_version

Check #5 is the one people skip. A "successful" upgrade with missing entities is a silent failure you won't catch until three days later when an automation fails to trigger. The entity count is your single best upgrade health indicator.

9. Cheat Sheet: Ten One-Liners

Commands I run constantly during upgrades:

# 1. Full Supervisor info
ha supervisor info | jq .

# 2. Force restart HA Core
supervisorctl restart homeassistant

# 3. HA Core logs (last 100 lines)
docker logs --tail 100 homeassistant

# 4. Any addon's logs
docker logs --tail 100 mosquitto

# 5. Enter HA Core container interactively
docker exec -it homeassistant bash

# 6. Manual Supervisor backup
ha supervisor backup create --name manual --password x

# 7. Restore a snapshot by ID
ha supervisor backup restore <ID>

# 8. Force addon restart
ha addon restart <slug>

# 9. Check HA Core migration status
sqlite3 /config/homeassistant.db ".schema migrations"

# 10. Get HA Core version from inside the container
docker exec homeassistant homeassistant --version

FAQ

"I upgraded and my box won't boot. Where do I start?"

Pull the SD, mount it on another machine, read /var/log/journal for the last boot attempt. If the boot config is intact, the issue is in /usr. If the config partition is corrupt, restore from NandUSB-Save-Manager. Both corrupt? Reflash and restore.

"My Mosquitto addon shows 'Failed' after an upgrade. How do I diagnose?"

docker logs mosquitto --tail 200
docker exec mosquitto ss -tlnp

Ninety percent of the time it's a port conflict — the old container still holds the socket. supervisorctl restart mosquitto clears it. Otherwise check log_pub_errors false and mosquitto.conf for a bind-address mismatch.

"Can I run two HA OS versions side-by-side for a canary test?"

Not on the same box. HA OS is an appliance image — one instance per boot partition. My staging strategy is a second HA VM on Proxmox, upgraded first, tested for 48 hours, then the production box.

"Does this runbook apply to HA Docker?"

Mostly. HA Docker has no Supervisor, so chapter 1's four-step chain collapses to two (image pull, Core boot). Rollback is easier — docker pull homeassistant:2025.10.6 and restart. But no built-in backup menu, no addon snapshots, no NandUSB-Save-Manager. Your backup strategy is entirely custom.

"Should I disable auto-upgrade entirely?"

For 24/7 production, yes. Auto-upgrades are how I lost Mosquitto on a Tuesday at 2:47 AM. Set auto-upgrade off for HA OS, HA Core, and every addon; check for updates once a month.