[This is a work in progress, IANAL and this isn’t legal advice]

Lots of data is Switzerland is produced by the cantonal GIS offices (while it might seem to originate from swisstopo it often doesn’t), for example the hiking path/trail data is all cantonal and we can only utilise such data, even when using the data distributed by swisstopo, if the cantonal terms are compatible with our license.

In the following I’m using open in a hand wavy, “close enough” fashion here, and not applying the strict definition as per the open defintion. Class A refers to the federal ordinance definition for generally accessible geo data¹.

To set the scene: Switzerland does not have sui generis database rights regulation or anything similar, nor does it adhere to a sweat of the brow copyright doctrine. Between non-government entities any (minimal) protection available is based on contract and fair competition law. The big exception is geo data where the federal government has written in to law rights that are essentially a “data copyright light” and many of the cantons have followed suit.² There is no relevant case law that I know of and how any of this would work out in an actual dispute is, well, open.

… See full entry

Android 13 was released in August 2022, not yesterday, but on the other hand not so long ago.

Why is this relevant?

With Android 14 google started updating the root certificates¹ with updates to its “play” services², prior to that they were only updated with full system updates and while now you can count on such updates for multiple years that used to be very different.

This is a problem for apps running on older devices that need to access resources on the Internet with encrypted connections (that is with https) as not only can such a resource change its certificate provider and potentially by doing that change the relevant certificate authority, certificates can expire or otherwise be invalidated. If that happens the resource is essentially unusable without an update to the certificate authorities.

This is not a new problem, particularly for Vespucci³ as we support devices going back to Android 5 and without the app bringing its own copy of the relevant root certificate for Let’s Encrypt⁴ along, you wouldn’t have been able to access openstreetmap.org for years on old phones.

So it shouldn’t have been a surprise when on last Saturday an issue was opened complaining that the Polish governments geoportal was erroring⁵, but what issues users report is not always straight forward, and this was likely the first “important” source that ran in to the problem.

Now there is a quick fix and that is that the user installs the relevant certificates on their device themselves, this requires that the relevant app trusts user installed certificates and I’ve enabled that on V21.2.4 that is being distributed now. What the situation is with this configuration with other apps is unclear.

… See full entry

This is not about the Linux Foundations Overture Maps attempt to hoodwink the Open Geospatial Consortium into standardising Overtures GERS (Global Entity Reference System) ¹. There is so little technical content available in that proposal that it is difficult to write even a short paragraph about it. But it is motivated by Overtures attempt to engage in the great American tradition of selling snake oil by suggesting that GERS solves real problems ². None of the following is new, and all this has been discussed many times in the OSM community.

It isn’t as if having a persistent id for an entity isn’t useful, for example if you are running a restaurant review site³ you definitely want a way to reference and track the state of the object you have reviews for in your geodata source, be it be based on OSM or some other data. The issue is simply:

your persistent id is not my persistent id.

Or perhaps better, your persistence is not my persistence. Lets illustrate that with the restaurant review example again: assume you’ve generated an id in some fashion and map that to an OSM object (more on that below), when do you consider the current restaurant different from the original, and when do you consider it different enough that you will want a new id?

Is it

• when the name changes?
• when the location changes? What about if it has just moved to the next block?
• when the cuisine changes?
• when the chef changes?
• when the owner changes?

And so on.

The answers to these questions depend on your use case and your business logic and a global one size fits all is very unlikely to be of any help at all. Now you might say, but there are objects, places, buildings and geographical entities that have less tendency to change, at least on a typical humans time scale and yes ids could be useful for these, but they are by their very nature easily referenced by their location⁴.

… See full entry

OpenStreetMap old timers know about the infamous 2009 TIGER import of road data in the US that continues giving to this day. Our story has none of the, maybe deliberate, shenanigans (see the TIGER improvement project) that went on back then in the US, but there are clearly some similarities in the lessons that should be learnt.

Back in July of 2011 the Swiss community undertook a big effort to import municipality boundaries from swisstopo (the marketing name of the federal Swiss GIS department) (see osm.wiki/Switzerland/swissBOUNDARIES3D). Being an OSM n00b at the time with just a bit over a year editing experience I didn’t really do anything useful for the import proper, but I did organise the explicit permission needed from swisstopo as this was many years before their data would become available for use on open terms for us in September 2021. With a couple of technical hiccups along the way that are not really documented, we finally managed to complete the work by early August.

Fast forward to today: I’ve been going on for a few years now that we really need a quality assurance process so that we can discover and track differences between swisstopos data and what is in OSM. We knew and fully expected that there would be differences, because:

• at the time of the import we simplified the boundaries quite significantly because of the resource constraints of the computer hardware available to us,
• over the last 14 years we independently followed the mergers and other changes of the municipalities (see osm.wiki/Switzerland/2026_Municipality_Mergers), and didn’t expect this to improve the accuracy of the boundaries, and we knew that now and then there would be associated minor geometry changes that we wouldn’t be able to track,
• and then just general decay due to glueing and accidental modifications.

Thanks to work by our community member habi inspired by earlier work by Branko Kokanović from Serbia, we have now have daily QA data and boy, we were wrong.

… See full entry

Some of the readers may have experienced the sinking feeling when they just attempted to upload dozens if not 100s of changes and a network glitch resulted in the editor app failing the upload.

What’s the problem when that happens?

It is literally that your editing app doesn’t know if any new objects it created locally have been successfully created on the server or not. If they have been created, retrying the upload will create duplicates, and if you don’t retry you potentially loose your work.

This post is related to work that I’m doing for version 21 of Vespucci that automates all of this, but the interesting thing is that you can easily handle many of the cases manually without editor support.

As you may know OSM changesets are not atomic, but individual uploads are (yes, you can upload multiple times in to the same changeset). With other words if your upload succeeded, but your editing app lost the response from the server, all your changes will have been successfully made, and, the other way around, if your upload fails it will fail completely.

The other point to note is that changesets are opened in a separate operation and are only automatically closed after a timeout of an hour if they are not explicitly closed after the upload by your editing app.

A normal upload operation will roughly contain the following steps

• open the changeset.
• upload the data and process the response. Depending on your editing app it will update the downloaded data in place, or throw it away and re-download from the server.
• close the changeset.

This means that there are simple cases that you can handle manually, assuming you are not uploading more than once per changeset (more on that later):

… See full entry

Way back in 2015 I wrote this diary post Is OSM business unfriendly? on how core OSM limiting itself to data collection had fostered a bustling ecosystem.

Organisations from small to large, commercial and non-commercial have built their businesses on building OSM know-how, normalizing the data, aggregating it with other sources, and providing services on top of it. It is one of the major factors that has made OSM not just a hobby project, but a notable player on the global stage.

Yes, the Linux Foundations OMF is disruptive, but the disruption is mainly in that it will remove a major part of the raison d’être for these organisation in the wider OSMspace.

The surprising bit at SOTM-EU over the weekend was just how hilariously unaware both Linux Foundation members and representatives were that their main effect will be stomping out a whole raft of SMEs, and the sudden realisation by the victims that US big tech and their non-profit front are not their friends.

What does this mean for core OSM?

I’ve argued that we should be moving the boundary of what we do outwards so that we can at least provide more of what was provided by the layer of service providers. Not all of the market for geo data and services outside of the US tech bubble is going to enthusiastically embrace that their choice of suppliers has been reduced to a duopoly. But I will concede that doing nothing and letting the market forces play out is the more OSMish reaction.

PS: I would point out that all of the above has already chewed though multiple times in public. Maybe if the Linux Foundation doesn’t want to be put on the spot and look very very out of touch they should read what other people are saying about them.

I haven’t done a contributor statistics diary post in a long time, but I suspect that the latest update of the graphs on the wiki is likely to lead to some questions.

As we can see in

the number of new contributors to OSM has dropped quite a bit in 2022. As the overall number of contributors has always been dominated by the new contributors it isn’t a surprise that the overall number is down too.

The interesting question is if we can pinpoint any “source” of this reduction. As a 1st investigative step I gave the numbers for editing apps used by new editors a look. I’ve been producing the numbers for a long time, but have never published them previously, if we look at 2021 and 2022 we see:

… See full entry

Four years ago on May the 25th 2018 the, then new, EU-wide General Data Protection Regulations (GDPR) went in to force. It is therefore quite fitting that nearly to the day 4 years later, the OSMF has given up any pretense of ever fulfilling the legal requirements arising from the regulation and, perhaps more important, undertaking the ethically and morally indicated steps to protect the privacy of our contributors.

It isn’t as if the OSMF was caught by surprise in May 2018. The topic was by far the best researched, documented and planned change in its history. And it isn’t as if the quality of the work or the conclusions were in doubt, completely independent and unaware of what had been prepared by the LWG three years earlier, at last years SOTM there was a talk by Robert Riemann that came essentially to the same conclusions.

But, four years later, with the sole exception of new users having to accept the OSMF terms of service on sign up (added by yours truly), none of the required changes to the APIs and data access have even been started on OSMF properties.

To make the situation worse, OSM isn’t simply static, new services, third party and OSMF operated are being added, existing operations are changed and rarely are the data protection impacts and consequences considered.

Ironically some organizations in OSM-space have taken required steps, for example Geofabrik, OSMcha and Pascal Neis’s HDYC, that the OSMF has not.

Why have we ended in this fraught with legal and financial danger place? On the one hand there are no brownie points to be won with championing the changes to the OSM website, the API and data distribution. Since Frederik has left the board no director has been willing to show any support for the matter.

… See full entry

As every quarter I’ve updated the contributor statistics in the OSM wiki. The most notable effect is the drop off in new contributors in the 2nd half of 2021, which can at least partly be attributed to the demise of maps.me as a source of new mappers.

This is more obvious when you consider just the new users from this source

Note that the numbers include new users using organic maps, but even that doesn’t change the trend.

See my diary posts from 2020, 2018 and 2017 for more on the topic.

Just realized that I didn’t notice that on Tuesday (the 5th) I sent out the 10’000ths welcome message to a contributor.

I’ve been doing this for SOSM since December 2013, nearly 8 years ago, averaging a bit over 100 messages per month.

I’ve discussed our expectations (or rather the absence of them) before see osm.org/user/SimonPoole/diary/40071

PS: The happy “winner” was osm.org/user/JFischli

As a rule of thumb, the primary name would be the most obvious name of the feature, the one that end users expect data consumers to expose in a label or other interface element.

From the OSM wiki Key:name

I’m currently tidying up a couple of loose ends in preparation for the release of version 16 of Vespucci. One of those dangling bits was supporting the new format of the name-suggestion-index (NSI).

The NSI was conceived back in 2013 by Aaron Lidman as a list of canonical name spellings for chains of stores, restaurants and other similar facilities, most notably for use in iD. Vespucci has supported use of the NSI nearly since day one, see a diary post from 2014 (this works quite differently in current Vespucci versions). Since then it has morphed to a, supposedly, authoritative source of tagging for a very wide range of objects. Some would say that the current version has expanded its reach far beyond what is actually useful (flagpoles ffs), but that is not the topic of this post.

Prior to the release of iD 2.20 the NSI had been in a long period of stasis with updates being made to the data, but these were not actually being deployed and given the larger format changes I had all but forgotten about it and the data in Vespucci was really old. However over the last couple of months some colleagues pointed out some weird behavior in Vespucci and iD when using NSI generated presets for tagging. In particular name tags are being added to things like excrement bag dispensers and automated postal package depots.

… See full entry

Today we’ve started rolling out the 1st beta release of 16.0. If you intend to install this, or at a later date the release version, please read the release notes http://vespucci.io/help/en/16.0.0%20Release%20notes/ before installing.

Due to the preparation for targeting Android 11 there have been some changes in where files are stored that you need to know about.

The first beta release of Vespucci 15.2 has just become available on googles play store, alternatively you can download it from releases in Vespucci repository.

While it would always be nice if as many users as possible could test this as fast as possible, this year due to events outside of our control (bintray demise, mapillary sunsetting its current API at extremely short notice, not to mention the usual multiple man weeks of work due to googles requirements), we need to get this release out of the way asap, any help with that would be appreciated.

Dedicated address mode and address handling improvements

When we introduced the “simple mode” in Vespucci 12 in 2019 there was no way to directly add address nodes without switching to the old element creation mode via long press (you could however add a node and then predict the address in the property editor). To make this a bit easier 15.2 introduces a dedicated “Address” mode that switches the “plus”-button to allow adding

• an address node (including automatically adding an entrance tag if the node is on a building outline)
• an address interpolation
• a map note

Further improvements include support for addr:block, addr:conscriptionnumber and addr:block_number in the address tags preference, and the use of information from geocontext project for region specific default address tags.

Support mode specific launcher shortcuts

After adding a short cut for the “help” system in 15.1, we’ve added further short cuts for specific modes: Indoor, Tag-only and Address.

In launchers that support the functionality short cuts can be accessed by a long press on the app icon, and dragged to create short cut links for direct access.

Address mode demo

Support merging closed ways/polygons

Basic support for merging two closed ways/polygons has been added to the multi-select mode. If their are no common nodes, or if the function dedects inner rings this will create a multi-polygon, otherwise a single closed way.

… See full entry

I’ve just updated the contributor statistics for the whole year 2020.

As maps.me has hit the news again, this time in a more negative way, I thought it might interesting to give the influence of the app on new contributors another look.

… See full entry

Vespucci 15.1 has been available for a couple of days now from googles play store marking the third larger release (14.1, 15.0 and 15.1) this year and our traditional release for the holiday season.

Details on what has changed can be found further down, but first let me point out a few important points from the past year and things to be aware about that will be happening in 2021.

The OSMF and iD

2020 marked the OSMF becoming a participant in the OSM editor market place, you might say what does that concern me? Well, even if not all the developers are competing for funding (like iD, Rapid, StreetComplete and many others), all are competing for relevance. Or put differently: mapper market share. How exactly this will play out going forward isn’t clear yet, but it already has consequences in development, for example I’ve put all tablet related improvements on hold. The required effort just doesn’t makes sense with the OSMF expanding iDs use outside of the classic desktop on to tablets..

More https/SSL woes

A couple of weeks back Let’s Encrypt dropped a bomb. Let’s Encrypt is by far the most popular provider of free encryption keys for https use, and are widely used throughout open source and data, and the OpenStreetMap ecosystems. For example Let’s Encrypt certificates are used on all OSM infrastructure that has so fully migrated to https that use without is no longer possible.

Android 7.1 as the oldest version that will no longer work with Let’s Encyrpt is fairly young, and lots of users are “still”* using it. An estimated 33% of the 2.5 billion Android devices will be effected. So we can’t simply ignore this happening.

… See full entry

I pointed out some of my concerns with the change to the OSMF articles of association and changes to who is to become eligible to a member on the the OSMF mailing list a couple of weeks back, see https://lists.openstreetmap.org/pipermail/osmf-talk/2020-October/007231.html

Those comments were just on the general concepts that were being proposed not the actual text, which we have now received as part of the formal general meeting announcement.

Unluckily my concerns not only persist, they have increased with respect to the proposed change to the OSMFs articles that will allow the board to create committees that include non-board members.

Currently a large part of OSMFs business is carried out by working groups, in particular the OWG running the infrastructure, the CWG communicating on behalf of the OSMF and the LWG handling licence and other intellectual property matters.

The working groups go back to a OSMF board meeting in November 2008, https://docs.google.com/document/d/1OcoOqas1NnOLyhiyFZk12K_BpjhWeXvVU7Nl4MDSGeU/edit?usp=sharing and it is clear that these were established to have topic specific groups in which the board could work on OSMF business with additional participants without having to have the full board present. With other words, working groups have always been board committees with additional members, that this practice wasn’t quite inline with the foundations articles was and has been just a bit of fuzziness among a lot of other issues that the OSMF has had.

… See full entry

After a large number of delays, mainly to due to late breaking issues with undocumented behaviour changes and other issues due to the androidx migration, I’ve tagged the 15.0.0 release. Installable APKs should be available from google play and the Vespucci github repository in the next couple of days (not from f-droid which is lagging a quarter of a year currently and is unlikely to fix its build problems any time soon).

More information on the future of the app will be announced later.

End of support for pre Android 4.0 devices

Version 15 no longer supports devices with Android versions older than 4.0 / API 14. We’ve now migrated the Vespucci code base to use the “androidx” libraries to provide backwards compatibility over multiple Android versions and there is no feasible way to continue to use the old support libraries that would be necessary for older Android versions.

Import note: even though the app will work on Android 4.0 devices, devices with Android version prior to 4.1 do not have TLS 1.2 support that is required for accessing the OpenStreetMap API since August 2nd 2020. If you are experiencing authorization issues on Android versions between 4.1 and 4.4 see Can’t (re-)authenticate - TLS 1.0 / 1.1 issues.

Support for array like semantics for multi-selects

In some circumstances, tags with multiple objects have array-like semantics (that is a fixed number of elements) contrary to being variable length lists. A typical example are lane related tags, which contain the number of elements in the lanes (or lanes:forward and lanes:backward) tag. There is no practical difference for tags that need to have a value for every element, but in the case of tags that have free form, potentially empty values, for example destination:lanes, there was previously no way of indicating this in the preset.

… See full entry

Those that follow the Vespucci twitter account @vespucci_editor will already know, on Sunday the main had TLS 1. and 1.1 support OSM API turned off. This is of a bit wider relevance as older Android devices do not support TLS 1.2 out of the box or at all, and obviously this may effect other apps as well and even such simple things as retrieving map tiles may no longer work when tile caches are updated.

What is older in this context? Well there is no support in any form for devices older than 4.1, introduced in 2012 and no out of the box support, before 4.4.something (likely 4.4.3). 4.4 was released just 6 years ago, so is not that old. Vespucci will enable TLS 1.2 on devices in the range 4.1 - 4.4.? and will work just fine with the OSM API, with two caveats:

• it is possible that there is no support for TLS 1.2 even though there should be, sorry nothing we can do about that.
• while everything for which the network connections are directly initiated by the app should work, components that open their own connections may fail.

Off the top of my head there are only two situations in which the later applies, one not so important is opening issues on github, the other more relevant is authorizing the app via OAuth to use the OSM API. If that happens on your device you should follow the instructions on the Vespucci documentation site and switch to using login and password. Yes, the irony of this is not lost on me.

The deployment of upgraded operating system versions without TLS 1.0 and 1.1 can be followed on Dropping TLSv1 and TLSv1.1 support. If you have any questions or issues not covered above please don’t hesitate to ask.

Seems as if I never wrote a diary entry on 14.1 that has been available for a couple of months now. The release notes can be found as always on the Vespucci documentation site.

Obviously version 15 is quite far along at this point and major parts of the work have already been completed. In particular the migration to the AndroidX libraries and re-targeting for Android API 29 have been finished and, as a consequence, support for Android versions before 4.0 has now been completely dropped.

I’m not planning on adding major new functionality in this release, after major internal changes a shake-down period tends to be required due to device specific issues that are only exposed when the app gets actual use. Changing too much just compounds that. There are however a number of small changes (for example indicating the number of pending changes for upload on the main menu bar) and performance improvements, that will make the user experience more pleasant.

Work on the release can be followed in the Vespucci github repository.

… See full entry

While keeping a quarterly minor or major release schedule is a bit of an uphill battle, I believe the work on the next major version of Vespucci has progressed far enough that the first beta will be made available later this month (snapshot development builds are available now).

The fall releases always tend to suffer from porting to the next mandatory Android API/SDK version and this one is not different. I assume that this typically consumes a man-month of development work, this time around it doesn’t seem to be quite so bad however there are still a few issues I haven’t resolved that need to be fixed before release.

The bad news is that this will be the last release that will support an Android 2.3-3.0 compatible build. The reason is that to be able to continue to follow googles minimum target API/SDK rules we will have to switch to a new packaging of the Android libraries that provide backward support for older versions of Android (this implies that the estimated one man month for following google changes for next year is very optimistic). As all the package names have changed it is simply not feasible to use an old support library as we have done up to now for the legacy build with minimal addition version specific code. Undoubtedly it is a side effect of the packaging change that google is quite happy with.

My intent is to revive the legacy build for 4.0-4.3 devices when google drops support for them, which is likely not far away.

… See full entry