# 0035. The Drupal re-import is incremental and field-owned

- **Status:** Accepted
- **Date:** 2026-09-07

## Context

The catalogue came over from the previous Drupal site in October 2025, through the SQL scripts in
`data/migration/`. Since then both sides moved: the editors here rewrote names, short names, slugs and
addresses, assigned SVG floor maps and `svgNodeId`s, set publication flags, and paid for AI translations of
219 units, 15 sites, 55 articles and 805 photos — while the client went on working in Drupal, raising prices
and adding and deleting units.

Going live means re-importing. The 2025 scripts cannot do it: every one of them starts with
`DELETE FROM … WHERE legacyId IS NOT NULL` and re-inserts, with the mapping between the two systems written
out as hundreds of hard-coded ids. Running them would erase a year of editorial work and re-download every
image.

Measured against the source on 2026-09-07, the actual delta was small and lopsided:

| | |
|---|---|
| units | 494 in the source, 468 here → 48 new, 22 gone (6 of them re-created under new ids) |
| **prices** | **438 of 446 differ** — raised in Drupal in mid-2026 |
| dimensions | 3 of 2 230 property values differ |
| Hungarian descriptions | 1 of 446 differs |
| articles | 12 new, 0 changed |

So the re-import is a small update over a large amount of curated data — the opposite of what the delete-and-
reinsert scripts assumed.

Two errors in the original mapping also came to light. `location.legacyId` pointed at a `helyszinalaprajz`
node — the floor-plan page — which is the wrong key: several nodes describe one site, two of our sites have
no node at all, and the Szentendre sites had the wrong node, so location `SZTE` (Kalászi út) claimed node
177, the Dózsa György út container yard, and inherited its photos. And the 15 addresses the 2025 migration
created were orphaned: every site points at a hand-made address instead.

## Decision

**Every field has an owner, and the importer writes only what it owns.**

| Owner | Fields |
|---|---|
| **The source** | `price1M/3M/6M/12M`, the property values, `description` (hu), `locationId`, `storageTypeId`, and whether a record exists at all |
| **Us, never overwritten** | `slug`, `name`, `nameInContract`, `shortName`, `svgNodeId`, `areaId`, the publication flags, `translations`, and everything on `location` and `address` |
| **The workbook** | whatever the client corrects — see [0036](0036-the-excel-round-trip-is-xlsx.md) |

The list lives in `ImportDiffService::STORAGE_FIELDS` and `ARTICLE_FIELDS`, and the same structure drives
the report, every `--dry-run` and the write itself, so the three cannot disagree.

Four more rules:

- **The importer never deletes.** A unit the source no longer has is unpublished and listed in the report.
  It may carry photos, property values, rentals and payments.
- **Everything is keyed on a legacy id and is idempotent.** A second run is a no-op.
- **The reliable location key is the `helyszin` taxonomy term** (`field_helyszin_tid` on a unit), not the
  floor-plan node. It is exactly one term per site. `m260907_100000_location_legacy_keys` repoints
  `location.legacyId` to it, empties the now-meaningless `location.legacyPicture`, and deletes the orphaned
  addresses. The floor-plan nodes stay, in `data/import/maps.php`, only for their photos.
- **The order is one-way.** Once the corrected workbook has been imported, the Drupal import must not run
  again: it would overwrite the client's corrections with the source's values.

Whatever the source cannot express stays in `data/import/maps.php` — the term-to-site map, the type map, the
floor-plan-node-to-site map — reviewed by hand once, keyed by short name so it survives a database that
numbers its rows differently.

## Alternatives

- **Re-run the 2025 scripts on an empty catalogue and redo the curation.** A year of editorial work and the
  AI translation bill, again.
- **Diff by hand and write one-off `UPDATE`s.** Works once, for the prices. Useless tomorrow, when the
  client hands over a corrected dump.
- **Let the source own everything and treat our database as a projection.** Simple, and it deletes the
  curation. The whole point of the migration is that this system is where the data lives now.
- **Delete the units that vanished from the source.** Rejected: 6 of the 22 are BP VAR units the client
  deleted and re-created under new ids, and deletion would take their photos and property values with them.
- **For the images: rsync the whole file tree up front, and never touch the network.** Safer, and it makes a
  350 MB transfer a prerequisite for a 41 MB delta. **Or: always crawl, no local tree** — one download per
  *use* rather than per file, and nothing left once the old site is gone. The importer does both: the local
  tree first, the old site as a fallback, **and the fallback writes into the local tree**, so it is a crawl
  that builds the rsync incrementally.

## Consequences

- **`data/migration/*.sql` is history.** It stays in the repository as the record of the first migration and
  is never run again; `commands/LegacyController.php` likewise.
- **A run reports rather than decides.** New, changed, gone, unroutable, and the data-quality problems only
  the client can answer for — three duplicated BP VAR units, ten unpublished leftovers, seven titles padded
  with whitespace. That report goes to the client before anything is imported.
- **A new site needs a hand.** The importer creates the site and its address unpublished; the SVG floor map,
  the description, the properties and the `svgNodeId`s are the editors' work.
- **A new storage type is created unpublished**, because a published one changes the site's navigation.
- **The property values are rebuilt from the source**, which is only safe because nothing has ever edited
  them here — all 14 040 rows still carry the migration's timestamp and no `updatedAt`. If that ever stops
  being true, this rule has to change.
- **The boolean conversion stays lossy in the same way the 2025 migration was.** A taxonomy value of
  `Igényelhető` ("available on request") becomes `true`, like `Igen`, because the target column is a boolean.
  Kept identical deliberately: changing it would rewrite thousands of rows for no new information.
- `location.legacySlug` is left alone. It is not source-derived at all but a hand-written short alias, and
  `ContentsController::actionDisplay()` serves the old site's 301 redirects from it.
- **The image import depends on the old site staying up**, unless the file tree has been rsynced. That is a
  deliberate, reversible dependency: `--localOnly=1` refuses the fallback, and every image the importer has
  ever fetched is still in `data/import/files/`.
- **Everything one import writes to `web/contents/` lands in a single dated directory**, because
  `File::getPath()` buckets by `createdAt`. So the upload to the server is one `tar` of one folder, and
  `import/contents-manifest` is the check that it arrived rather than the way it gets there.