Midterm is here, so this is a proper writeup of what’s landed so far, how the pieces fit together, and what the second half looks like. If you just want the code: everything described here merged in #4106.

Quick context for new readers: Nominatim identifies every place with a single class/type pair derived from OSM tags. An object tagged both tourism=hotel and amenity=restaurant becomes two rows in the database. Admin boundaries need admin_level special-casing everywhere. And there’s no way to express “wheelchair accessible cafe” at all. This project adds a proper category system to fix that at the database level.

Here’s the state of things at the halfway mark.

the data model: ltree[]

Categories are stored as an ltree[] column on place and placex, indexed with GiST. Each category is a dot-separated hierarchical path:

{osm.amenity.restaurant, osm.tourism.hotel}

I benchmarked this against TEXT[] with GIN during community bonding on a full planet import (~3 days to set up, worth it). The TEXT[] approach is what Photon effectively does at the OpenSearch level: pre-expand every prefix at index time (osm.amenity.restaurant also stores osm.amenity) and match with array overlap. It works, but you pay storage for every prefix of every category on every row, and the expansion logic lives in application code.

ltree understands hierarchy natively:

-- all amenities: restaurants, cafes, bars, everything below
WHERE categories @> 'osm.amenity'::ltree

-- exact match
WHERE 'osm.amenity.restaurant'::ltree = ANY(categories)

-- multi-value alternation (waterway checks)
WHERE categories ~ 'osm.waterway.river|stream|canal|drain|ditch'::lquery

Less storage, less code, and the query planner gets a real index to work with.

One constraint worth knowing: ltree labels are restricted to alphanumerics and underscores on PostgreSQL < 16 (pg16 relaxed this to allow hyphens, but Nominatim supports back to pg14). OSM tag values contain literally anything: Cyrillic, semicolons, emoji. The sanitizer replaces - with _ and falls back to yes for anything else illegal, so a value in Russian script becomes osm.landuse.yes rather than a corrupted label. The real value always survives in class/type and extratags. This matches Photon’s fallback behaviour, so category vocabularies stay compatible in the Nominatim -> Photon direction.

category generation: n tags -> 1 row

This is the biggest structural change in the PR. The old import pipeline called write_place() once per main tag, producing one row each:

OSM object: tourism=hotel, amenity=restaurant
  -> write_place(tourism, hotel)      -> INSERT #1
  -> write_place(amenity, restaurant) -> INSERT #2
  place: 2 rows, same osm_id

The new model does a single pass in process_tags(): collect every main tag as an osm.<key>.<value> category, pick one winner for the legacy class/type columns, strip the sibling main keys into extratags, and do exactly one INSERT:

OSM object: tourism=hotel, amenity=restaurant
  -> categories = {osm.amenity.restaurant, osm.tourism.hotel}
  -> winner: amenity (alphabetical)
  -> INSERT: class=amenity, type=restaurant,
             categories={osm.amenity.restaurant, osm.tourism.hotel},
             extratags={tourism: hotel}
  place: 1 row

My first implementation actually kept the old per-tag row generation and merged rows afterwards with a row_precedes() winner function. It worked, but it’s a pointless round trip: build N rows, immediately collapse them to 1. The rewrite generates the single row from the start.

About that “alphabetical” winner: it’s a deliberate placeholder, not a considered ranking. class/type still needs a stable value for compatibility (stability matters: the same tag set must always produce the same winner, or updates break). Once the API queries categories directly, the winner choice stops affecting search entirely. The class and type columns stay as passive information because too many API consumers rely on them in responses; categories replace their role in filtering, not presentation.

A concrete win from the merge model: duplicate rows are gone. placex shrinks. More on what that did to performance below.

ranking had to become category-aware

This was the part I underestimated. Nominatim assigns search_rank and address_rank per row based on class/type. With one class/type per row that’s a simple lookup. With a merged row that might carry both osm.boundary.administrative and osm.place.city, the ranking code has to decide which identity wins.

compute_place_rank now iterates all osm.* categories and picks the entry with the lowest positive address rank (ties broken by search rank, address rank 0 sorts last). All the SQL trigger code that previously matched on class/type had the same problem and got the same treatment:

-- before
WHERE class = 'boundary' AND type = 'administrative'

-- after
WHERE categories @> 'osm.boundary.administrative'::ltree

migration: lazy backfill

Existing installs get the column via nominatim admin --migrate. The interesting design decision is what to do about the 22 million existing placex rows with empty categories. I have covered the migration strategy in more detail in a blog if you’d like you can check: https://medium.com/@rupamgolui/adding-a-column-to-22-million-rows-without-melting-the-database-40430c4d3bd2

In summery we implemented a placex_update trigger that lazily fills categories from class/type whenever it touches a row that doesn’t have them yet. The only rows that need categories upfront are the ones other rows look up during processing, i.e. linking targets: places and waterways with rank_address <= 25. That’s roughly 20% of the table.

The backfill numbers on full planet (22,221,508 rows), because the optimization order matters:

approach	total time
indexes first, triggers enabled	~63 min
triggers disabled, indexes built after backfill	~47 min
temp-table two-step	1h 40m

The temp table experiment (materialize computed categories first, UPDATE placex second) was meant to isolate where time goes. The planner got the INSERT badly wrong and it came out twice as slow. Abandoned. The winning recipe is boring: disable the update trigger, backfill, build indexes, re-enable, and run ANALYZE at the end so the planner has fresh statistics immediately instead of waiting for autovacuum.

Final production migration: ~42 minutes on the planet. On a busy production box expect more like 1–2 hours, which is acceptable for a one-time migration.

performance: no regression, actually a gain

The thing everyone should care about with a change to placex: did search get slower?

Geocoder tester on the migrated planet, run from the server itself:

branch	result	time
master	7925 failed, 11107 passed, 3264 skipped	30m 33s
PR branch	7923 failed, 11109 passed, 3264 skipped	29m 19s

Six tests flipped status between branches, four to passing, and those turned out to be flakiness from row ordering rather than real behaviour changes. Independently, a fresh planet import with the PR code ran the test suite about 10% faster than master, most likely because the merge model collapses duplicate rows and placex is simply smaller.

what the second half looks like

Everything so far is foundation. Categories exist on every row, ranking understands them, migration works at planet scale. But nothing user-facing queries them yet. That changes now.

Next, the main event: replacing the place_classtype_* tables. Today, POI and near searches go through materialized tables, one per (class, type) pair used in special phrases, e.g. place_classtype_amenity_restaurant, each maintained by INSERT/DELETE trigger logic. With categories indexed on search_name, the same lookup becomes a single query:

-- before: dedicated table per combination
SELECT place_id FROM place_classtype_amenity_restaurant
 WHERE ST_DWithin(centroid, ...)

-- after: one indexed column
SELECT place_id FROM search_name
 WHERE categories @> 'osm.amenity.restaurant'::ltree
   AND ST_DWithin(centroid, ...)

If this holds up in benchmarks, an entire family of tables, their triggers, and their import-time maintenance disappears. poi_search.py and near_search.py get rewritten to use the new path.

Stretch, if the core lands early: API-level filtering with Photon-compatible semantics, so this becomes a real request:

/search?q=restaurant+berlin&include=cuisine.italian&exclude=food.fast_food

and a YAML-driven CategoryGenerator for categories richer than raw OSM tags (cuisine.italian, access.wheelchair.yes, osm.boundary.administrative.{admin_level}).

The design principle for all of it stays the same as the first half: categories replace class/type in filtering, never in presentation, so existing API consumers keep working untouched.

If you run Nominatim and have a use case where category filtering would matter, the second half is exactly when that input shapes what gets built. Comments open.

Thank you for your time :)

~ Agasta