Home Assistant Data Migration: A Three-Layer Guide and Five Real Incidents
Why "Migration" Is a Different Animal from "Backup"
Backup and restore is boring, and that is a good thing. You press a button, you get a file, and the world is safe. Migration is not boring. Migration is when the disk, the operating system, the database engine, or the hardware underneath your Home Assistant changes, and suddenly "restore the backup" is no longer enough.
I have been running Home Assistant for a decade, and I have moved it from a Raspberry Pi 3 to a Pi 5, from HA OS onto a Proxmox VM, into Docker containers, and off SQLite and onto MariaDB. Every single one of those moves looked easy on paper and bit me somewhere I did not expect. This article is the version of that story I wish somebody had told me in 2018.
The core mistake people make is treating migration as one operation. It is three, and the three travel at different speeds and fail in different ways.
The Three Layers You Are Actually Moving
Before you copy anything, understand what you are holding.
Layer one: the config layer. This is configuration.yaml, secrets.yaml, the whole .storage/ directory (device registry, entity registry, auth, area registry, helpers, scenes), esphome/, blueprints/, and automations.yaml if you split your YAML. This layer is small, moves fast, and is the easiest to get right.
Layer two: the history layer. This is the recorder database, usually /config/home-assistant_v2.db (SQLite). This is the layer that kills people. It can be tens of gigabytes, it is tied to a specific database engine, and it is silently useless if you get it half right.
Layer three: the custom components and apps layer. This is custom_components/, python_scripts/, appdaemon/, and your Node-RED flow export. This layer is small but it is the one people forget, and forgetting it is how incident #1 happened.
The rule I now live by: migrate the three layers separately, verify each one before moving to the next, and never assume layer two (history) is a requirement. History is a nice-to-have. State is not.
Incident #1: Every Automation Died Because I Left custom_components Behind
I moved an HA OS install from one Pi to another. I did the smart thing: full backup, copy configuration.yaml, copy .storage, restore. HA booted clean. Then I noticed nothing was working.
The logs told the story.
Logger: homeassistant.loader
Source: components/hassio/__init__.py:...
Integration 'xiaomi_miot' not found.
Integration 'tuya_local' not found.
Every custom integration I had ever installed simply did not load, because custom_components/ was not part of the .storage directory and I had not copied it. It was sitting in a separate volume I assumed the backup had caught. It had not.
The result was nasty: all 40+ automations that called custom services loaded without error at boot, because HA does not resolve those services until runtime. When a trigger fired, the service call failed and the automation silently did nothing. It looked like broken trigger logic. It was a missing directory.
The fix is trivial and permanent: before any migration, list the directory.
If you see anything in there, it is layer three and it travels with you. Do not assume it is inside your backup.
Incident #2: 18 GB of History and a Chunk Size That Was Too Small
When I finally decided to move my recorder off SQLite and onto MariaDB, my database had grown to 18 GB after five years of sensors logging every few seconds. I had done my homework: I read that sqlite3-to-mysql was the tool, so I ran it.
I ran it with a small chunk size. It stalled for three days.
sqlite3-to-mysql \
--sqlite-file /config/home-assistant_v2.db \
--mysql-database homeassistant \
--mysql-user ha \
--mysql-password "$DB_PASS" \
--chunk 1000 \
--skip-foreign-key
A chunk of 1,000 rows against a states table with ~100 million rows means ~100,000 round trips. SQLite and MariaDB were both fine; the number of round trips was the problem. That run was never going to finish.
The fix is to tune the chunk upward. I killed the import, dropped the MariaDB tables, and re-ran with a larger chunk. The whole 18 GB moved in hours instead of days.
sqlite3-to-mysql \
--sqlite-file /config/home-assistant_v2.db \
--mysql-database homeassistant \
--mysql-user ha \
--mysql-password "$DB_PASS" \
--chunk 100000 \
--skip-foreign-key
My rule of thumb: a chunk around 50,000 to 100,000 rows is the sweet spot on a normal machine. Too small and you die of latency; too large and you die of memory. Tune it, do not trust the default.
Incident #3: "Multiple Data Disks Detected" — The Old Disk Would Not Let Go
This one is a pure HA OS story. I wanted to move my data disk to a bigger SSD using ha os datadisk move, and I had plugged the new disk in while the old one was still attached, thinking I could migrate live.
The command refused to even start.
HA OS only supports one active data disk at a time. move is not a live copy between two mounted disks; it is "take this one disk, make it the data disk." With a second disk present, HA OS will not guess which one you mean.
The fix was to physically remove the old data disk (or the new one, depending on direction) so exactly one candidate disk was present, then re-run.
The lesson: ha os datadisk move is a hardware operation, not a software one. Think about which disk is plugged in before you type it.
Incident #4: A Perfect Boot but a Broken History Timeline
This was the most insidious of the five, because nothing looked wrong. The boot was clean, integrations loaded, entities appeared. Then the history timeline was full of holes and the energy dashboard showed flat or zero values for whole weeks.
The import had not been complete: statistics and statistics_runs were nearly empty even though states had millions of rows. HA rolls states up into long-term statistics and needs those two tables consistent to do it. A partial import left gaps in the timeline that I almost missed until I zoomed in.
This is why row-count verification is not optional. Compare the SQLite source to the MariaDB target for the full six-table set.
mysql -u ha -p homeassistant <<'SQL'
SELECT 'event_data' AS tbl, COUNT(*) AS rows FROM event_data
UNION ALL SELECT 'events', COUNT(*) FROM events
UNION ALL SELECT 'states', COUNT(*) FROM states
UNION ALL SELECT 'states_meta', COUNT(*) FROM states_meta
UNION ALL SELECT 'statistics', COUNT(*) FROM statistics
UNION ALL SELECT 'statistics_runs',COUNT(*) FROM statistics_runs;
SQL
The six tables that must all arrive intact are event_data, events, states, states_meta, statistics, and statistics_runs. If any count is off, do not point HA at the new database. Fix the import first.
Incident #5: The Entity IDs Changed and My Templates Stopped Resolving
On a hardware move I re-created the device registries from scratch instead of copying .storage. The result was that Home Assistant re-discovered my Zigbee devices and assigned them new entity IDs, so sensor.living_room_temp became sensor.living_room_temp_2.
My templates did not break loudly. They broke softly: every {{ states('sensor.living_room_temp') }} returned unknown, every automation condition comparing a temperature failed silently, and dashboard cards went blank one by one.
Entity IDs are not magic. They are stored in .storage/core.entity_registry. Copy that file and do not let HA re-register your devices, and your IDs survive the move. The moment you let HA rediscover hardware without the registry, the IDs change and every string in every template points at a ghost.
So the checklist item is: verify your entity IDs are identical before you let any automation run.
# before vs after - the ID must match exactly
curl -s http://ha.local:8123/api/states/sensor.living_room_temp \
| grep -o '"entity_id":"[^"]*"'
The Cross-Form Playbook and Pre-Migration Checklist
Every migration path I have taken shares one skeleton. Pick your path, then run the checklist.
The paths:
- HA OS to HA OS (e.g. Pi 3 to Pi 5): run a full backup, restore it on the new install, then copy
.storage,custom_components/, and the recorder DB. - HA OS to a VM: same backup, restore inside the VM's HA OS. The tricky part is the recorder DB; if you keep SQLite, copy the file, otherwise go MariaDB.
- HA OS to a Docker container: the backup format is not directly usable. Unpack it manually into a mounted config volume and bring the container up against it.
docker run -d \
--name homeassistant \
--restart=unless-stopped \
-v /opt/ha/config:/config \
-v /etc/localtime:/etc/localtime:ro \
--network=host \
ghcr.io/home-assistant/home-assistant:stable
- Data disk move:
ha os datadisk move, with exactly one candidate disk attached, as incident #3 taught me.
The checklist I run before any of these:
1. Full backup (Settings > System > Backups) and download it.
2. Record HA core version and OS version (ha core info --raw-json).
3. List custom_components/ - this is layer three, do not assume it is in the backup.
4. Note the recorder DB path and size (du -sh /config/home-assistant_v2.db).
5. Export Node-RED flows and appdaemon/ separately if present.
6. Confirm exactly one data disk is attached if you plan a datadisk move.
7. Decide in advance: does history matter, or can HA rebuild a fresh DB?
SQLite to MariaDB Done Right
When history matters, MariaDB is the target I use. The move is three steps: dump, import with a tuned chunk, verify. Then point Home Assistant at it.
Two details most people get wrong. First, do not let HA write to the new database until the import is verified, or the schema_changes table and the running instance can fight the import. Second, verify the six tables before you switch db_url, per incident #4.
And keep the old SQLite file around until you have run at least a full week on the new engine. A history migration you can roll back is a migration you can afford to do twice.
Post-Migration Smoke Test
The migration is not done when HA boots. It is done when I have run the smoke test. I do not enable automations until these pass:
1. Boot log has no "Integration not found" and no fatal errors.
2. All custom_components from the checklist are present (ls /config/custom_components).
3. Every entity ID from the pre-migration list still resolves (states API returns 200).
4. The history timeline renders with no gaps for the pre-migration week.
5. The energy dashboard shows non-zero values.
6. Manually fire one automation that calls a custom service; it completes.
7. The recorder is writing to the intended database (check the log for the db_url line).
If any of these fail, stop and roll back to the backup. Do not "just tweak it live." In a migration, the only state you can trust is the one you verified.
FAQ
Do I need to move the recorder database at all?
No. If history does not matter, restore config and custom components and let HA build a fresh DB. Only move it when history or statistics matter.
Why does SQLite to MariaDB break the timeline and statistics?
Because the six-table set must be complete. A missing statistics or statistics_runs table leaves the timeline and energy dashboard full of gaps.
Can I keep the same entity IDs after moving to new hardware?
Yes, if you copy .storage and do not re-register devices. IDs live in .storage/core.entity_registry.
What is the fastest safe path for a very large history database?
Keep the recorder on the same host, or import in one batch with a large chunk size and verify row counts per table first.
What breaks first if I forget custom_components?
Every non-built-in integration loads nothing, so automations that call those services fail at runtime and HA logs "Integration not found".