Extracting OpenStreetMap Data for Ultra Mileage

Although we supply a number of PBF extract files from the OpenStreetMap global ‘planet’ file, it is possible to create your own extracts for use with the Ultra Pre-Processor to create custom road packs.

Our own extracts are created using the Osmosis command line tool. There are other feature extraction tools such as the Overpass API, but Osmosis works offline with an existing source file, and hence works better with larger files.

Osmosis is a Java tool that can be downloaded as a ZIP. Installation instructions for Windows, OS X, and Linux can be found on OpenStreetMap’s Osmosis page. The command line examples below were written using Linux (Ubuntu) running a Bash shell, with line breaks for clarity.

Osmosis works on a pipeline model, which allows it to perform a number of filter operations at the same time. Although this can result in long command lines, this can greatly reduce the total processing time.

Here is a simple filter example modified from the OpenStreetMap wiki, that extracts all speed cameras from a PBF file:

This reads the input data from “input-map.pbf” and writes the speed cameras to a new file called “radar.pbf”. osmosis also supports the XML-based “.osm” file format, which may be more useful for an example like this one.

This is the command line we use to extract road data from a general PBF file:

This is similar to the above example, except we limit the ways to only those with a highway tag defined. All highway tags are written to the output file. The “–used-node” parameter tells Osmosis to only write the nodes which are referenced in the output ways. So highway nodes are written out, but building nodes are not.

Reading of the input data can be speeded up by using the “–read-pbf-fast” parameter. This multi-threads the input process with the requested number of workers. Input is typically disk limited, but you may find significant speed-ups if you are using an SSD (solid state drive). We also filter the meta data using the “omitmetadata” parameter. Meta data consists of information such as edit dates and authors. This information is not used by Ultra Mileage, and removing it reduces the size of the output files by about 15%.

Similarly, we can filter only those features which are within a set of polygons:

This filters the input file, writing only the features within the “region.poly” polygons, to the “region.pbf” output file. “–bp” is the “bounding polygon” option. This is a text file that lists the coordinates of the polygons. This format is described on the OpenStreetMap Osmosis wiki pages, and can also be created using JOSM.

The “–tee” parameter can be used to create multiple output regions:

The “–tee” specifies a “T” junction in the pipeline, splitting it into the specified number of pipelines (2 in this case). The input data is then filtered according to the two “.poly” files and then written to their respective output files.

This is how we create the road packs and it significantly reduces the runtime because Osmosis only has to go through the input file once.

Osmosis has a large number of parameters and these are described on the Osmosis Wiki’s Detailed Usage page. Options include the ability to merge files, create changesets, transform tags, sort features, and read/write to PostGIS.

An alternative to Osmosis, is Osmconvert. Osmconvert is a simpler, faster command line utility. It can also extract regions using “.poly” files, but can only extract one region at a time.

Leave a Reply