Previously I wrote about checking National Parks in Britain and Ireland for breakage, by counting the number of valid polygons in a rendering database.

A similar approach can also be used to check line features as well as polygons. It’s possible to count the number of contiguous sections of a given polygon. For example, here’s the Lyke Wake Walk, which is in one piece:

psql -d gis -c "SELECT count(*) FROM planet_osm_line WHERE (osm_id = '-1996318');"
 count 
-------
     2
(1 row)

and here is the Sabrina Way, which isn’t:

psql -d gis -c "SELECT count(*) FROM planet_osm_line WHERE (osm_id = '-537430');"
 count 
-------
    12
(1 row)

If something is changed in OSM that introduces an extra piece, or fixes a gap, that number will change. This script can be used to count the change in the number, like this:

Change in relation components for Sabrina_Way 537430
3c3
<     12
---
>     13

… See full entry

Taginfo is a way of finding out what tags and values are used within OpenStreetMap. There’s a worldwide instance here, and other instances exist, including various Geofabrik ones such as Antarctica here.

As well as describing keys and values used within OpenStreetMap, the worldwide instance of Taginfo also holds details of what other projects use particular OSM tags. The Taginfo/Projects page explains the update process.

Each project submits a .json file containing details of key and value usage, such as this one for an OSM Mug.

The one for the map style that I look after is here. Currently I’ve just submitted a list of keys, not values, but I have included a description for each one, as can be seen here for “designation”.

The first challenge was to create the initial list. There are references to tags in three main places in my map style - the lua style file, the carto .mml file and the individual carto .mss files such as this one. A bit of investigation (creating a unique sorted list of strings within style.lua on one hand and .mml and .mss files on the other found that although some keys do pass through style.lua unchanged, almost no keys that are later used in carto files aren’t mentioned in style.lua at all. That means that I only needed to worry about keys in style.lua. I created an initial .json in the correct format and then wrote a shell script to generate the “tags:” section of the .json file. I then filled in the descriptions where the shell script had just created placeholders, and checked it in.

Next, I wanted to be able to easily update the taginfo.json file after changes to the map style. Because I’ve also checked the previous generated file into github I can easily create a new one and compare the two:

sh -x ./taginfo_lua_genkeylist.sh > q.txt
diff taginfo_lua_nodesc.txt q.txt

for the most recent change the output of that is:

… See full entry

The OpenStreetMap Community Index is designed to show people what community groups there are within OpenStreetMap, in any part of the world. If I look near me I see a local pub meeting (sadly on hold during the pandemic), some country-level resources, and then international resources.

On a couple of occasions I’ve seen things that are missing or need updating, and this diary entry is designed to complement the existing documentation to describe what needs to be done to fix that.

Creating a local copy

In my case I already have a fork of https://github.com/osmlab/osm-community-index at https://github.com/SomeoneElseOSM/osm-community-index. There’s a “fork” button in the top right of Github’s UI that allows you to do this.

Next, I’ll create a local branch to work on. The default branch at https://github.com/SomeoneElseOSM/osm-community-index is “main” and the UI will say “This branch is even with osmlab:main.” because (in your case) you’ve only just created it. In github’s branch selector just type the name of the new branch to create. In this case I created “add_forum_at_world_level”, because I noticed that there wasn’t a link to the OSM Forum at the international level, although there are many links to country-level forums from the community index.

Next, “git clone” your repository wherever it is you want to work. I’m working on a completely new virtual machine to do this, so I have to take a slight detour first - see “setting up ssh access to github” below. Once that’s done:

mkdir ~/src
cd ~/src
git clone git@github.com:SomeoneElseOSM/osm-community-index.git
git checkout add_forum_at_world_level

Note that I’ve used a version of “git clone” that is designed to work with ssh. Github is (rightly) moving away from password-based verification and will send you a warning email if you try and use a password during the transfer period.

… See full entry

As you will no doubt have heard, browser support for flash went away at the end of last year. As you can read here, Potlatch 3 has been released and downloads for Windows and Mac are available. Linux users are asked to “Use Wine to run the Windows installer”. You may be wondering “why?” - there’s a section at the end of this diary entry that covers that. First, the “how”:

What do I need to do?

There are lots of different Linux distributions out there. Most have some sort of package manager and in most of those “wine” will be available as a package. Here is what I needed to do on a couple of desktop Ubuntu systems (both using KDE) - one running 18.04 LTS, the other 20.04 LTS.

Firstly, install Wine itself:

sudo apt install wine64

Next, you’ll probably need to install some fonts that software that normally runs on Windows needs. You can do this via a helpful script Winetricks. First, install cabextract, which is needed by the script:

sudo apt install cabextract

Next, decide where to install the “winetricks” script. I’ll put it in an existing user “bin” directory, so:

cd ~/bin
wget https://raw.githubusercontent.com/Winetricks/winetricks/master/src/winetricks
chmod +x winetricks

Next, install the fonts:

winetricks corefonts

Finally download Potlatch 3 (choose the “Windows application” option, which is a zip file) from the download page, unzip it, and run it:

wine64 Potlatch.exe

Follow the instructions that are presented and accept the defaults to “just install” it.

Running Potlatch 3

When Potlatch runs, you’ll initially see the whole world zoomed out. Rather than zooming in manually you can then “edit with remote control” from the openstreetmap.org site to go to wherever you are currently looking at there, and you can also search for any sort of OpenStreetMap URL to jump straight to it. “GPS / My tracks / load” loads GPS traces.

… See full entry

Occasionally, I (like other people I’m sure) have noticed that things “disappear” from maps they should appear on. In the case of “national parks” and “areas of outstanding natural beauty” in the UK, they’re mapped as multipolygons. This might be simple, like Nidderdale or more complicated, like Ynys Môn. What usually happens is that someone accidentally “breaks” the multipolygon, for example by creating a gap in an outer edge, so that rendering software sees it as invalid and ignores it.

What I’ve typically done in the past is, whenever I’ve noticed a problem tried to find the problem using something like the OSM Relation Analyzer or JOSM’s validator. The “area” view at Geofabrik’s OSM Inspector is also useful.

That works, but “happening to notice a problem” as a first step is not ideal. This map does show national parks / AONBs and not a lot else at zoom 7, but those tiles only typically get updated every 4 days, so it can lag behind problems.

… See full entry

I’ve added a new page to the switch2osm guide: “Updating your database as people edit OpenStreetMap”.

It’s a bit overdue (it’s based on something I wrote about 4 years ago, and the software it uses predates that by a long way), but I think it is worth making available on the main site.

I’ve tested it both on Debian 11 and on Ubuntu 20.04 LTS, and have tried to make clear what you need to do differently in each case.

For completeness see also Ircama’s excellent guide which is well worth reading as it actually provides a lot more detail about the process, and also suggests how “osmium” may be used instead of “osmosis” to process OSM files.

I’ve added a new page to the switch2osm guide: “Manually building a tile server (Debian 11)”. The Debian developers have worked to include the required software inside Debian 11, so there’s actually less setup work needed than in previous versions.

Debian 11 has not yet been released; it’s still “testing”, so there may be minor changes along the path to release. Also, there’s currently a temporary problem with the Natural Earth download site that requires a workaround, but that’s documented in the guide. When that is resolved I’ll update the notes.

I’ve not yet linked the new page into the menus because I want to give people a chance to test it and identify any problems first. Once that’s happened, I’ll add it to the “Serving Tiles” menu.

Unfortunately, in England at the moment a number of pubs aren’t able to reopen because (especially locally to me) they’re too small and have lots of small rooms or they’re music venues for which “social distancing” somewhat defeats the object of the event in the first place.

I’ve chosen to show these on this map like this:

This is deliberately different from the rendering of “normal” closed pubs:

… See full entry

I’ve recently added a new feature to https://map.atownsend.org.uk , prompted by the dreaded lurgi that is affecting everyone at the moment - the ability to show pubs with either outdoor seating or a beer garden.

Outside seating is shown like this:

with a black line at the bottom right of the icon.

Here’s a beer garden:

… See full entry

I’ve added a couple of short but interesting scripts to github. They fetch and sort posts from an OSM mailing list from some time in the past. For example, here are the topics on the talk list at the height of the licence change furore:

listtopics.sh talk 2012 May

 35  OSM : It's a shame !!!
 24  Edit review: ele=0
 22  Worst of OSM
 21  TomTom is thumping us
 17  Import of buildings in Chicago
 17  Edit review: intermittent waters
 16  OSM cycle map - ?excessive focus on long-distance routes
 15  Edit review: "building"="levels=N"
 14  Cycleways and Access tags: Left, Right, Forward, Backward?
 10  OpenSeaMap
 10  Cycle lanes & cycle tracks - my findings and a proposal
  7  Changing capitalization (Lima)
  6  OWL down
  6  OSM data density - top regions
  6  Old versions of OSM?
  6  Copy-and-paste remapping
  5  TTTracklog and TomTom core 9.4
  4  OSMCoastline / OpenStreetMapData.com
  4  Mapnik rendering issue with *_link roads
  4  (dis)Honesty and Copyright

and here’s the tagging list today:

… See full entry

I’ve added a new page to the switch2osm guide: “Manually building a tile server (20.04 LTS)”. At the same time, the old “with packages” instructions has been retired from the list as that depended on Ubuntu 14.04, which has now dropped into Extended Maintenance.

Ubuntu 20.04 (“Focal Fossa”) has now been released. There are actually very few changes from the 18.04 version - mostly just updated versions of software (including postgres 12). Following these instructions shouldn’t take more than a couple of hours for small areas - the longest period of the setup is waiting for the shapefiles used by the style to download.

As before, the page is designed to be “the least you need to do” to get a rendering server working to create tiles like the ones that you see at openstreetmap.org. If it still sounds a bit complicated, don’t forget that you can try things out with Docker very quickly - and that should work anywhere that Docker is supported.

For some time I’ve been experimenting with how other information can be added to established icons. For example, here’s how I’ve been representing various tourist information items:

In those examples there are only two pieces of information - that it’s something to do with tourist information, and whether it’s a board, guidepost, plaque, etc.

With pubs, I already show several sorts of information together. For example this one:

… See full entry

As you may be aware, the “switch2osm” site recently moved from Wordpress to Github. This means that it’s easy to request changes to the site over at the github repository. Any changes made there will automatically go live on the main site after a minute or so.

“Requesting changes” might mean commenting on existing issues or creating new ones (which will need someone else to make the code or text changes you’ve requested), or it might mean suggesting the changes yourself via a “pull request” - a list of specific changes you’d like to see made. Here is an example of a pull request - you can see what it’s a change from and to, and on this page you can see any comments and when it was merged in and by whom.

Obviously, if people can contribute via pull requests rather than just creating issues it’s better for everyone - it means that you’re not relying on someone else reinterpreting the changes that you’d like making, and it’s quicker for the maintainers too.

Normally if you wanted to make a major change to something on this site you’d “fork” a copy of the repository into your own github area, create a “branch” from that that relates to the change that you’d like to make, “clone” that locally into an environment on a local machine that allows you to test the site locally, make your changes, test your changes in a local web browser, submit your changes back to your copy of the site, and then create a pull request explaining the change you’d like to make.

What you may not know is that for simple textual changes you can actually do the whole process online. That’s not appropriate for anything that might break links, but for simple text changes (like the example above) it’ll work just fine, and this is what the rest of the diary entry describes.

… See full entry

Imagine I’ve just installed OsmAnd on an Android phone and want to change the rendering slightly.

For completeness, the OsmAnd version is 3.5.5, I downloaded it from Google Play, where it’s called “OsmAnd - Offline Travel Maps & Navigation”. The Android phone is a Blackberry DTEK50 (actually a badged Alcatel design) running Android 6.

I ran OsmAnd and skipped the initial download (because I’ve already got some .obf files to install), but you’ll probably want to download some local data.

I want to copy files to and from the device, so I’ll plug it into a PC. How you copy files depends on the OS on your PC. Next, plug the device in via USB and select “copy files via MTP” from the prompt that appears on the device (this actually varies by Android version, and what options are available to copy files to and from a device depend on that and the phone itself). A window showing an MTP connection to the device contents will probably appear - check you see something in there. If you don’t, unplug it and plug it in again.

I’m running Ubuntu Linux on the PC and want to copy files to and fro from the command line, so I’ll install something to allow me to do that, check that I can read something from the device, and unmount it again:

cd
mkdir mtp
sudo apt install jmtpfs
jmtpfs ./mtp
ls ./mtp
fusermount -u ./mtp

For me the “ls” above shows ‘Internal storage’. Before copying anything used by OsmAnd in the steps below I’ll make sure that it isn’t running on the phone. How to do this varies by phone - on mine it’s “menu” and then “close”.

Next, we’ll look at what maps are currently installed. How to do this varies by Android version, but this is how things appear on this Android 6 device:

… See full entry

Following recent discussions on OSM mailing lists about tag homogenisation it struck me that there probably wasn’t a good summary anywhere of the tools that people use to change the tags in OSM data into something that they can use. You might wonder why on earth we need to do this, given that OSM has natural language names for everything, but unfortunately many words used in OSM might have different meanings around the world, such as “city” and “highway”. However, this isn’t a diary entry about that - it’s a brief summary of what I’m aware of and have used for converting the “super detailed” tagging in OSM into something more appropriate for e.g. something rendering OSM data.

osmosis

The documentation for this is here. I wrote a diary entry about how I use this method to change the “name” language of Welsh-speaking places in Wales to Welsh, and Scots Gaelic-speaking places in Scotland to Scots Gaelic.

… See full entry

There are a lot of peaks mapped in OpenStreetMap, ranging from proper mountains to mere pimples. They’re usually rendered the same regardless of how high they are:

However, with a bit of preprocessing and a bit of help it’s possible to render higher ones at a higher zoom level:

… See full entry

I was prompted to write this because of a few discussions over the last couple of months - people talking about the default tags on various OSM elements, reactions to Amazon mappers’ additions of “last mile” service roads and discussions about how to tag England and Wales’ public rights of way network.

This is very much a personal view and one that’s very “England and Wales” centric - there are lots of different approaches to rural tagging in general and footpath tagging in particular, and many places around the world (enlightened places like Scotland and Scandinavia) have different default access rights to the curious set we have ended up with in England and Wales.

Let’s start with an example - what’s the reasoning for the tags on this way in OSM?

… See full entry

(following on from this diary entry)

Supposing you would like to overlay a set of boundaries over the top of any existing map style - what would you need to do?

Using this as a reference, create a database to hold the new boundary data.

If a historical data file is what you want, download the data for the desired date and rename it to a file name that the “update_boundaries” script will recognise:

-rw-rw-r--  1 renderaccount renderaccount   459723464 Nov  2 01:46 ukraine_2018-11-01T21:14:02Z.osm.pbf

You can also make other changes to this data if you wish using tools like osmium, osmfilter, osmosis, etc.

Modify this script to process your data.

Run the script to load the data that you have already downloaded:

sudo update_boundaries.sh current

Create a version of “mapnik.xml” (or whatever style file you are using) that looks in the database that you’ve loaded your boundary data into. In this case I just changed “dbname” references from “gis” to “gis3”.

Ensure that “/usr/local/etc/renderd.conf” knows about the new tile layer. The default one I use just has “ajt” in it; I created a copy of the last section, called it “ajt3” and changed URI and XML values.

[ajt3]
URI=/hot3/
TILEDIR=/var/lib/mod_tile
XML=/home/renderaccount/src/openstreetmap-carto/mapnik3.xml
HOST=localhost
TILESIZE=256
MAXZOOM=20

Restart renderd and apache2, and your new tiles will appear. Then in a simple Leaflet map add an overlay layer for your new tiles. This is an example that shows Leaflet overlays but there are many, many others, including on the main Leaflet site itself.

Here’s what it looks like:

… See full entry

When I developed the map style that you can see here from OSM-carto one thing that I didn’t carry forward was the representation of borders. Partly this was because it’s not easy to separate maritime ones from non-maritime ones, the the purple maritime country borders frankly don’t look very nice:

Partly it was also due to “clutter” - I didn’t want an arbitary boundary to get in the way of a river or stream or other feature.

Sometimes however, it would be useful to see what borders are where, so why not do it as an overlay? This turned out to be rather easier than anticipated.

… See full entry

I’ve been wanting to write an explainer for this for a while (even before this diary comment). The problem was that I wasn’t aware of a Docker example that (a) was reasonably up to date and (b) was designed around tile serving (rather than style design for example).

However, this docker example was recently mentioned by its author on IRC. It’s based on the building a tile server for 18.04 instructions and also handles importing data and running the tile server afterwards.

First things first, I installed Centos 7 on a virtual machine. I got a “minimal” iso image for that from here. I installed it (which involved turning the network on inside the installer and selecting the disk to install on). I also set a root password and added a non-root user (I’ll use the example “renderaccount” below), and noted the IP address that it obtained. I did a “yum update” to install any available software updates.

There are a few sets of instructions for “installing Docker on Centos” - DigitalOcean have one, and there’s one here which also contains lots of other useful information. Following the instructions there:

su -
yum install yum-utils device-mapper-persistent-data lvm2 wget
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
yum install docker-ce
systemctl start docker

If you then do this:

systemctl status docker

In the output you should see “active (running)”. Do this:

systemctl enable docker

in order to have docker start every time and then exit from root and do this from you’re non-root user:

sudo usermod -aG docker $USER

So that you can control docker from that non-root user. Log out and back in and do this:

docker container run hello-world

It should say “Hello from Docker!” to show that everything is working OK. If instead you see “permission denied” it probably means you haven’t logged out and back in again.

… See full entry