In diesem Blogpost zeige ich ein Python Skript mit dem osm2pgsql Importe automatisiert werden können. https://jakobmiksch.eu/post/osm2pgsql-replication-script/

In this blog post I show a Python script how to automate osm2pgsql imports: https://jakobmiksch.eu/post/osm2pgsql-replication-script/

Recently I found Streamlit which is a pretty cool Python library that makes it easy to create web apps for visualising data.

I converted changeset dump from planet.osm.org to Parquet file format and uploaded it to AWS S3 storage. Then created this streamlit app in their free cloud: https://ttomasz-tt-osm-changeset-analyzer-main-apdkpy.streamlit.app/ which displays some basic statistics.

The app leverages the power of DuckDB, a database engine that can query these files over internet on demand. Parquet files, which are a popular format in modern cloud data lakes, have several advantages over traditional file formats. They are column-oriented, compressed, and support range requests, which means that you can download only the portion of the file you need, instead of having to go through the entire file, making processing larger datasets much faster.

DuckDB works similarly to SQLite in that it doesn’t have a dedicated server. You run the queries locally [0]. This makes the setup super simple you either install the binary or configure connection in IDE like DBeaver and you can run SQL queries.

Running these simple SQL queries over remote Parquet files takes about a minute or two. Trying to do the same with a custom script on raw changesets.xml.bz2 file would run longer not to mention that the effort to prepare the code would be much much larger.

It would be great if OSM hosted more “consumption ready” data instead of relying on users to do their own coding and parsing.

Let me know if you have some ideas for charts/tables that could be added to the demo.

[0] - well in this case they are running on streamlit cloud’s server but you can run the queries locally on the same parquet files easily

O actual cruceiro da Alfarrapa é contemporáneo do situado no cruce de Refexón co Tabolado. O seu autor foi don Juan Míguez Domínguez. Anterior a eles é o do barrio da Torre. Os tres compóñense de pedestal, fuste octogonal e cruz. Non teñen imaxes.

O anterior cruceiro que aquí estaba caeu por mor dun carro que chocou con el, segundo unha versión. O párroco, don Esteban Viso, decidíu poñer os restos no muro da súa viña, ben visibles, e restauralo. Nunca pasou tal. Levantouse un novo cruceiro, por obra e desexo de don Juan Míguez (mesmo autor que o do Tabolado). Os restos están ben visibles no muro da antiga viña de don Esteban, no Pireiro, enfronte dun cruce de camiños, así que poden visitarse e fotografar con toda facilidade.

For this area beside the Blue Ridge Wilderness, I started out by adding what I knew of the Dangerous Park Trail and the Pueblo Park Interpretive Trail. There were a few miles of Dangerous Park already on the map, but they didn’t get all the way to the park. Unfortunately, the trail was diverging from what the Forest Service claims at the point I left it, so the little bit to the northern terminus includes guesswork. There’s some trail visible there.

I then worked on stuff in the wilderness and primitive area near the state line. It looks like someone has added in the trails from FS information (including attribution) in this area. I had a couple of adjustments based on my GPS, but the trail routes look good. I’m not sure if these are downloaded tracks or copied from the FSTopo. I’m seeing some changes between the two. The tracks that can be downloaded are more recent. I added signs to the mix. Guideposts and an information board. And parking.

I wanted to add the trails that connect to Dangerous Park, so I took the time to figure out downloading FS trail data again. There’s only about 4 different ways. Do they all connect to the same database or is it possible does one have to choose the right one to get the most recent data? All kinds of regulations are encoded into the tags on these trails. There’s also an indication of the state of the trail in “trail class”. Class 1 and 2 are generally represented here. Class 1 is minimally maintained and tread is intermittent and indistinct. Class 2 expects tread to be continuous, but still rough. Class 3 is continuous and obvious tread. These are trail_visibility statements! Always good to have that included.

So I got those trails added and while I was at it, I adjusted a few roads onto their route and added names and numbers. Lots of roads were called Saddle Mountain that are actually something else including a main one that is the Frisco Divide Road.

… See full entry

MapComplete has - for some thematic maps - the ability to leave a review on an entity with Mangrove.Reviews. Up till now, I had no idea how much this feature was used. However, due to technical reasons I had another look to the reviews module and discovered the ‘download all’-option on mangrove.reviews

Mangrove Clients

The analysis was made with data from 20 january 2023, downloaded around 17:00 UTC time.

This data contained 660 reviews. As the website making the review is recorded, we can make a breakdown of the top websites:

• https://mangrove.reviews is unsuprisingly the most popular website to make reviews on, with 318 reviews made
• MapComplete is the second (and the biggest ‘external’ website), with 192 reviews (of which 13 are made with the development version)
• toggenburg.swiss is third, with 35 reviews

A variety of smaller websites follows, each with a few reviews made. At first glance, most of them seem to be swiss or german. Furthermore, there are 5 reviews made by localhost:1234 and 7 by localhost:5000. The former is probably me, testing the creation of reviews while developing.

The full table is listed below.

… See full entry

O cruceiro máis antigo de Louredo en pé. Sería o modelo no que se fixaron canteiros do pobo, como o señor Xaquín ou Juan Míguez, para tallar os outros dous?

Cruceiro e detalle do pé

O seu autor foi don Juan Míguez Domínguez. Este cruceiro, como o do Tabolado e o da Alfarrapa, segue un mesmo estilo sinxelo, de fuste octogonal e sen imaxes.

O único con inscricións, ilexibles inscripcións na súa base. Este pé de cruceiro ten varias (queda pendente revisar por cantas caras, pero alomenos son dúas (ollada onde se percorre o pé do cruceiro dende o frontal, pasando ó lateral esquerdo, traseira e lateral dereito: https://bit.ly/3Che3rK). Esta vese se te pos fronte a el e te agachas polo lado esquerdo. O solpor axuda a detectar os riscos, inda que eu non me atrevo a dicir que poñen. Xeralmente, data de confeción e donante.

Este cruceiro moveuse hai bastantes anos. Orixinalmente, ocupaba un recuncho, en curva, onda a casa “da raiana”, pretiño da casa do “grilo”.

O barrio da Torre queda en ladeira, limitando cos lugares dos Diestros e O Regueiro, na parte alta, co Cazapedo, por un lado, e O Outeiriño, polo outro.

Diante deste cruceiro pasa a procesión da festa do san Xoán, a da natividade do bautista. Pretiño del soe pararse a imaxe para a primeira oración (a segunda é na praza do Cazapedo) e a partir del enfílase cara o Cazapedo, comenzando a veces a tocar a banda que ameniza o paso.

Actualmente, o cruceiro de Louredo máis antigo que segue en pé (o máis antigo sería o que se conserva en parte, esnaquizado, no muro da viña do Pireiro, o único con imaxinería).

Clica para máis información

Situados na beira dun muro dunha viña, no lugar do Pireiro, estes anacos dun vello cruceiro son o testemuño máis antigo dun cruceiro en Louredo. Este situábase na Alfarrapa, preto da Telleira, e é un modelo único no pobo. Dispuña das imaxes do Crucificado, no alto, por riba dunha caveira, ós pés da cruz, onde estaban dúas imaxes máis: san Xoán e a Virxe. Digo que é único porque os cruceiros que viñeron despois son todos moi parecidos entre eles, sen imaxe algunha. Se o da Torre resulta ser o máis antigo en pé, podería ser o modelo que se seguí para os da Alfarrapa e Tabolado, ambos da autoría de don Juan Míguez Domínguez.

Este vello cruceiro caeu por mor dun carro que chocou con el, segundo unha versión, ou foi derribado por algúns republicanos en datas cercanas ó alzamento nacional, segundo outras fontes orais. O párroco, don Esteban, decidíu por os restos no muro da súa viña, ben visibles, e restauralo. Nunca pasou tal. Levantouse un novo cruceiro. Os restos están ben visibles no muro da antiga viña de don Esteban, enfronte dun cruce de camiños, así que poden visitarse e fotografar con toda facilidade.

Máis en:

1. Notas sobre os cruceiros de Louredo
2. Notas sobre restos do cruceiro no Pireiro

In case you wanted to use my note viewer with another openstreetmap-website-based project - now you can. Although most likely you didn’t because there aren’t many of them and they don’t use notes actively. Additional projects that note-viewer is already configured to work with are:

• OpenHistoricalMap (note-viewer link) - without map tiles unfortunately, because they don’t have raster tiles
• OpenGeofiction (note-viewer link)

It’s not a surprise that notes aren’t heavily used in these projects. Often you place a note where the map diverges from reality and you can’t edit the map at the moment. But neither OpenHistoricalMap nor OpenGeofiction represent something that is currently real. Right now OpenGeofiction has less than 100 notes, and it’s not obvious if that project even needs notes. Maybe they might use notes to coordinate editing of their collaborative territories?

Now you can also edit the loaded and selected notes by commenting, closing or reopening them. This might be useful to deal with several notes at once. Actually it was one of the original plans for note-viewer. The situations where it’s helpful include someone modifying a lot of notes without a good reason. For example, users sometimes close existing notes without making any map modifications or providing reasons why the notes are irrelevant. They may do this because open notes look wrong (red with x marks) and closed notes look right (green with ticks). Closing a note may look like confirming it. Probably that’s why users sometimes close even their own notes without making any edits. Now it’s possible to quickly deal with such note modifications by searching for a given username, followed by filtering for user’s close actions, selecting all filtered notes and reopening them.

… See full entry

I’ve started a new project working with watmildon. While we were working together on applying the USGS Sq___ name changes to OSM we noticed was that there were often features in OSM that were out of sync with official name changes that happened years ago.

That got us thinking about walking through the USGS GNIS data set to find places where names had changed and OSM could be updated. After all, there are many features in OSM that have gnis:feature_id (and similar) tags that can be directly matched back to the GNIS data set.

After kicking the idea around for a while, we recently started writing some code. I’ve been working on a matching engine in C# that matches records from GNIS to OSM by Feature ID. The code also looks for likely matches where the feature name, primary tags, and geometry are close to the information from GNIS. So far the results are pretty good, but we’re still working on improving the matching.

Meanwhile, watmildon did some large scale statistical analysis on a local PBF file to look at the scale and scope of the problem. The results were very interesting!

Of the 2.3 million features in GNIS, there are only 1 million corresponding features with GNIS IDs in OSM. Some portion of these are surely existing features that just don’t have the gnis:feature_id (or similar) tags. But given our manual review of results from the matching code, there are a lot of GNIS features that are not present in OSM at all.

That’s not too much of a surprise. Some of the most common types of missing features are Streams, Valleys, Lakes, Springs, and Ridges – all things that not widely mapped in the US.

… See full entry

its what i do

تاسیس 1400/11/03 سهامی خاص سرمایه اولیه ** تجربه کاری 1.اصلاح هندسی میدان مادر 2اصلاح هندسی میدان نگین 3ایجاد گذر گاه عابر پیاده 4 فرهنگ سازی ترافیک بر مبحث شهر ساری نوین 5…..

As usual, I can’t quite remember how it started, but this week, I was trying to find a list of all the National Monuments of Ireland. The National Monuments Service publishes lists by county which only contain the ones under state care. These lists have made their way to Wikipedia which is a great start. They have their numbers recorded there which I have transferred to OSM under ref:IE:nm for the counties of Kilkenny, Laois and Offaly so far. (the key is recorded on the wiki, of course.)

I have sent an email to the National Monuments Service asking for a list, but so far, no reply. I probably have a reputation there by now…

However, I noticed while visiting and taking some pictures of Freshford Church that it has a black plaque (the type of which I had seen before, but never read in much detail) which stated that it was a National Monument.

… See full entry

I decided to continue trying to use JOSM for this area. I added details around the Narraguinnep Fort Historical Site, which was not simple. To add a point, I sit there in add mode and only click once so it doesn’t become a line? Hopefully that is so because that’s what I did. Then tracking down appropriate tags ended up meaning doing the same thing in iD, so not exactly a good use of time.

I continued on to details of the road around the Benchmark lookout. The track type changes halfway along. It’s nearly the boundary of the USGS map quads, so easy to miss, but they actually marked it. The road stops being improved dirt and becomes high clearance right in the middle. I did manage to figure out from JOSM how to mark that. In fact, now things are getting marked with tracktype. Smoothness was always presented, but maybe not as clear.

I decided to continue on with roads. The Forest Service marks various around the area as primary (trapezoid with an extra line markers on the map, maintained to passenger car standards) and secondary (horizontal numbers in a rectangle, should be to passenger car standards) and as 4x4 (vertical numbers in a rectangle, get the truck or even ATV). So how should one apply them? And why are they all marked as county roads, sometimes with segments with alternating numbers? None of it makes sense. I added some and lengthened some and adjusted some as I could see so their routes are all matching reality a bit better.

… See full entry

O cruceiro do Tabolado

O cruceiro do Tabolado, ou de Refexón, é o único que está nun cruce de camiños. Varal octogonal, sen inscripcións, un pedestal, sen imaxes. No cruce Louredo-Refexón (cara Remuíño, Arnoia)-Tabolado (Louredo, en liña cos barrios dos Veciños e O Pazo). O seu autor foi don Juan Míguez Domínguez. Do mesmo estilo que o da Alfarrapa e A Torre.

Confluencia de camiños

Postos ante o cruceiro (https://cutt.ly/Rb9UofQ), o camiño que sobe cara o pobo e barrio de Louredo (https://cutt.ly/2b9UYVi) queda ás costas (https://cutt.ly/Vb9UJHU), mentres que o do Tabolado (https://cutt.ly/Eb9UFHp) ábrese á man esquerda. Polo primeiro chegas ó peto de ánimas (https://cutt.ly/Rb9U8NJ) e tamén ós restos da escola pía (https://cutt.ly/Yb9Iw9Z). Polo segundo, ás viñas do Pireiro (https://cutt.ly/Bb9UQ6f), Tras do lagar e A Ladeira, ademais da escola do Tabolado (https://cutt.ly/Cb9IQS6), hoxe centro social. Hai un terceiro camiño, o da dereita, Refexón (nos mapas, Refoxón), que baixa cara Remuíño (https://cutt.ly/ob9IRwM) e que empata ca estrada cara Zaparín. Inda que se percorre en coche, hai un camiño que é o vello, quizás hoxe de monte e non practicable (https://cutt.ly/Bb9IUb7), polo que pasaron moitos louredeses no seu camiño ós muíños (https://cutt.ly/Rb9IFTL) do Inquiau (https://cutt.ly/ZYZMwXA).

O nome de cruceiro

Cruceiro do Tabolado, nun cruce de camiños. Moitos din que o cruceiro colócase en lugares así, pero non éa única opción á hora de entender de onde vén a palabra “cruceiro”. Tamén se fala de que poden ser construción en pedra que recordar a cruz procesional. E hainos que macan os límites de poboación (cruz de término).

… See full entry

This is really just down the road from the last bit. I decided to finally try out JOSM for editing. Everyone’s doing it? It was initially harder to do the simple things. Frustratingly, it wouldn’t let me start a new line rather than adding to an old one as I started adding the trail at the end of the stub of road at the Rio Lado trailhead. This trail was supposed to be a circle on the end of about 2 miles of trail, but I found about 2 miles of trail and, unrelated to the location of the circle, some other random trails. There were even equestrians on one of them. Back to the mapping, I found that joining these various lines was difficult and I even managed to upload one without any tags at all. I went back and fixed things with iD, which isn’t appropriate. It does say that there’s a steep learning curve.

Then I moved on to the Calico National Recreation Trail. This is a motorcycle trail, but it actually does see plenty of hikers and mountain bikers, too. I was aiming at a bunch of peak bagging along its spine, but apparently was too rusty in my packing of my overnight backpack. I tagged Elliot Peak and returned. Then I took a different route up to the mountain spine to tag a few more peaks in an overnight. Sockrider first and the namesake Calico last.

… See full entry

This isn’t directly related to OSM, but many times in “software” or “development” OSM channels someone has asked “how do I do X … on Windows 10?”.

The reason that’s often a question is that, for historical reasons, much of the OSM software used to do things has been developed on Linux. That’s great for people working on Linux themselves, and the Apple Macintosh users can normally jury-rig something to run that software too.

Windows users can do the same too, by installing “Windows Subsystem for Linux”. Within Windows, search for “Windows Features”:

In Windows Features, tick the box for WSL and click OK:

… See full entry

(Un cuento de san Xoán de Louredo, Cortegada, Ourense)

El cruceiro de O Tabolado, también llamado de Refexón, es uno de los puntos de interés histórico del pueblo de Louredo. En una nota posterior me ocuparé de algunos datos, aquí os comparto un cuento relacionado con él, un fragmento de la tradición oral de los vecinos.

Allí, en el cruceiro de O Tabolado, se encontró un vecino con el demonio y este le recomendó un acto de fe… de fe cristiana.

Ya lo contaban los padres de mis padres, señalando, incluso, el lugar donde esta historia sucedió. En los tiempos en que los louredeses bajaban a Remuíñoa moler el grano del maíz, sucedió que un hombre, cargado con su saco, se encontró a otro, con un pequeño fuego que lo calentaba. Eran tiempos en los que se cargaba con el fruto de la tierra y el trabajo de los labradores, llevándolo desde las casas hasta los molinos de la zona del Inquiau, al paso del río Arnoia por el ayuntamiento del mismo nombre. El camino, en descenso, era una posible vía romana, y había sido recorrido por el padre Sarmiento en el s. XVIII.

Le llamamos Refexón (Refoxón en los mapas) y comienza en un cruce de senderos, donde un sencillo crucero bendice a cuantos le dedican una callada oración. El caso es que el vecino de Louredo iba con el saco a la espalda y pasó ante el cruceiro y el hombre de la lumbre como quien pasa delante de un gato. Unos pasos más allá escucha que el hombre le dice: oye, ya que no me saludas, por lo menos saluda a este que tengo al lado (refiriéndose al cruceiro).

No cuenta la historia si el hombre siguió tal como había entrado, pero sí que, una vez regresado de moler, buscó los restos del fuego que calentaba a aquel misterioso hombre… Y no los encontró. Por lo visto era el demonio que, bajo forma humana, merodeaba el pueblo aquella noche. Descansando del camino y buscando calor ante el relente de la noche, había aprovechado para lanzarle al despreocupado vecino un pío recordatorio.

… See full entry

Querying elements from osm and doing practical things with them is really a useful and fun way of code “hacking” for me. In this case, I wanted to show farmers markets with a reactive map like leaflet like I’ve done before in my journal entries. In this case, I wanted to make it easy to browse the times, seasons, and contact info for farmers markets in the Anchorage, Alaska area. I think with some refinement this map could be useful to sellers and buyers to determine which farmers markets they can fit into their schedules.

… See full entry

GIS data for listed buildings, scheduled monuments (e.g. interesting tumuli) and more can be downloaded from https://historicengland.org.uk/listing/the-list/open-data-hub/, and here are some observations based on my experiments with geoJSON and shapefiles.

The data is under the Open Government Licence so OK for editing OSM.

The download can be kept to a manageable size by enabling “Filter as map moves” in the filter menu (funnel symbol) and then “Toggle filters” when selecting the download.

Vespucci can display a geoJSON layer, show object attributes, and even create preliminary OSM objects from them. But it didn’t work here. I discovered that the downloaded geoJSON uses EPSG:3857 (projected coordinates) whereas the geoJSON standard is EPSG:4326 (latitude and longitude).

An easy way to do the conversion was to go to https://mapshaper.org, “upload” the file (actually it’s processed on the client side), open the console and enter:

-proj from=EPSG:3857 crs=EPSG:4326

and then export.

The resulting geoJSON works well in Vespucci. You can either “Add GeoJSON layer” from the layer menu, or select it in a file manager and open with Vespucci. I could do the whole process above on a modest android tablet.

OSM tags worth thinking about include:

• heritage = 2
• heritage:operator = Historic England
• ref:GB:nhle =
• listed_status =
• wikidata =

Note that I’m not suggesting adding anything to the map that can’t be located with a ground survey, and certainly not blindly copying polygons (aside from considerations of verifiable ground truth, these may include a protective buffer around the feature).

… See full entry