This is not about the Linux Foundations Overture Maps attempt to hoodwink the Open Geospatial Consortium into standardising Overtures GERS (Global Entity Reference System) ¹. There is so little technical content available in that proposal that it is difficult to write even a short paragraph about it. But it is motivated by Overtures attempt to engage in the great American tradition of selling snake oil by suggesting that GERS solves real problems ². None of the following is new, and all this has been discussed many times in the OSM community.

It isn’t as if having a persistent id for an entity isn’t useful, for example if you are running a restaurant review site³ you definitely want a way to reference and track the state of the object you have reviews for in your geodata source, be it be based on OSM or some other data. The issue is simply:

your persistent id is not my persistent id.

Or perhaps better, your persistence is not my persistence. Lets illustrate that with the restaurant review example again: assume you’ve generated an id in some fashion and map that to an OSM object (more on that below), when do you consider the current restaurant different from the original, and when do you consider it different enough that you will want a new id?

Is it

• when the name changes?
• when the location changes? What about if it has just moved to the next block?
• when the cuisine changes?
• when the chef changes?
• when the owner changes?

And so on.

The answers to these questions depend on your use case and your business logic and a global one size fits all is very unlikely to be of any help at all. Now you might say, but there are objects, places, buildings and geographical entities that have less tendency to change, at least on a typical humans time scale and yes ids could be useful for these, but they are by their very nature easily referenced by their location⁴.

In summary a global persistent id decoupled from your use case doesn’t provide much utility and if you would utilize such a system it is likely that you would add a level of indirection to shield your internal business logic from being dependent on that of your id provider.

So if you are in the end creating your own ids anyway, how could you map them to OpenStreetMap entities?

Our holistic data model doesn’t make it very straight forward to continuously follow an entities modelling over its life cycle. To use the restaurant example again, it might start off as restaurant tagging on a building outline, morph to a Node⁵ with a location in that outline, be changed to an entrance on the outline or change to a room in the indoor mapping scheme and back again. The OSM data model doesn’t provide direct linkage between any of these, though mappers can reuse geometric elements in mapping, in the end you can’t rely on that happening in a consistent fashion. But if you can detect that such a change has happened you can use the tags (attributes) of the original element to find its current representation⁶.

Referencing a specific OSM object can be as simple as noting its type, its id and its version⁷. If the version has changed you can check the current attributes (tagging) and if they differ in a way that you consider relevant take appropriate action, which should likely always include a search for a nearby object with compatible tagging. If you find one, your id should then be mapped to the replacement. The same if the original element has been deleted. None of this is particularly difficult and you could easily use Overpass or a similar service to automatically find replacement mapping of the object you are interested in.

Enters OSMs data model curve ball: while the above works fine if your referenced object is a Node, geometry changes to linear and area features⁸ do not necessarily increase the objects version. This is not inherent in the data model, it is more convention than anything else, but it is a convention that is near universally observed. This means that I can move a building half way around the globe and its version will not change and for objects modelled by relations it can be even wilder.

The fix for this is simple: you need to not just store the elements_type_, its id and its version, you need to store an associated timestamp, in the simplest case the time when you retrieved the element to create the mapping⁹. To then check if the object has been further modified if the version hasn’t changed, retrieve the element and recursively compare your timestamp to that of the child elements¹⁰, if any of them are more recent than your timestamp, yes the object has been changed. No need to use historic data or anything exotic, the current OSM data is enough. You could even envision providing an API for this.

In summary: you don’t need Overtures GERS and you can link to OSM objects in a reasonably reliable fashion¹¹.

This is diary post 101, I should really just shut up

1. see https://github.com/opengeospatial/requests/issues/3 ↩

2. essentially the proposal just defines the output format of Overtures id generation process, without providing a specification or mechanism to generate them in a compatible format. More suspicious minds than mine might come to the conclusion that Overture actually wants to start a, probably commercial, id generation service and that this is simply the initial step to entrench it before announcing. It is interesting that all of their examples are centred around conflation/de-duplication and that can only work for a third party dataset if you use a compatible method to generate the ids, aka Overtures secret sauce. Unluckily given that the Linux Foundation gave Overture a corporate structure with zero public transparency and accountability we do not know if this is something their financials could require or not. ↩

3. See https://community.openstreetmap.org/t/a-crowd-sourced-review-service-for-openstreetmap/ for a recent discussion on the matter. ↩

4. naturally the nitpicking example here is if the Gulf of America is the same thing as the Gulf of Mexico. ↩

5. For an overview of OSMs data model see osm.wiki/Elements ↩

6. Overpass permanent ids directly use such a mechanism to link to OSM elements: osm.wiki/Overpass_API/Permanent_ID. ↩

7. Overture bridge tables do exactly that for GERS to OSM data mapping. ↩

8. Linear features are modelled as Ways that contain references to Nodes that contain the actual location, moving a child Node changes its coordinates and version, but as it is only referenced by the Way object this, as a convention, doesn’t change the Way version. Areas can be modelled in 3 different way, but relevant for this discussion are really only simple polygons that are closed Ways (that is start and end Node are the same) and multi-polygons that are modelled using Relations with individual Ways either directly closed or building closed rings. ↩

9. How well this works depends on how stale the data is when you create the mapping, a better approach would likely be to use the timestamp of the most recent child element. ↩

10. For Ways this implies simply inspecting the child Nodes. For Relations this depends on the specific type, for the most relevant multi-polygons, the members are Ways and you would need to check against the Way and child Node timestamps. ↩

11. I didn’t address the topic of ids for streets and similar objects at all, this would seem to have even less utility than for the discussed objects and you would likely be better off by simply using linear referencing. I might expand on this is a separate posting. ↩