Difference between revisions of "Installing a Map Tile Server on Linux"

From Nearline Storage
Jump to navigation Jump to search
(Created page with "These instructions are specific to the Fedora 25 Linux distribution. They can be adapted to other distributions by adjusting package names and file paths as necessary, otherw...")
 
m
 
(3 intermediate revisions by the same user not shown)
Line 2: Line 2:
  
 
The installation process is as follows:
 
The installation process is as follows:
#Install a collection of prerequisite software packages using your distribution's package manager.
+
 
#Configure the postgresql database software and create a database called "gis".
+
__TOC__
#Install and compile the mod_tile software.
 
#Install a mapnik map tile rendering stylesheet.
 
#Download the map data for the area you are working in and import it into the "gis" database.
 
#Test your installation
 
#Configure your system to start the httpd and renderd services at boot
 
#Bonus: Create an OpenLayers map web page so that you can play with your map tile server
 
#View service statistics (how many map tiles have been generated during the current session, and at what zoom levels) at http://localhost/mod_tile
 
  
 
===Install a collection of prerequisite software packages using your distribution's package manager===
 
===Install a collection of prerequisite software packages using your distribution's package manager===
Line 153: Line 146:
 
Your demo system probably doesn't have enough space to hold the map data for the whole world.  Fortunately, there are state-by-state or region-by-region subset extracts that can be downloaded and imported into the "gis" database to cover the area covered in your demonstration.  You can get these extracts (*.pbf files) from http://download.geofabrik.de and load them up with
 
Your demo system probably doesn't have enough space to hold the map data for the whole world.  Fortunately, there are state-by-state or region-by-region subset extracts that can be downloaded and imported into the "gis" database to cover the area covered in your demonstration.  You can get these extracts (*.pbf files) from http://download.geofabrik.de and load them up with
 
  osm2pgsql -d gis -C 16000 --slim --number-processes 8 filename
 
  osm2pgsql -d gis -C 16000 --slim --number-processes 8 filename
Each time you do this the data in the "gis" table is cleared out and replaced by the new data you are loading.  If you need to combine multiple extract files together to get the map coverage you need, you can append multiple files together with the osmosis tool before loading them into the database:
+
Each time you do this the data in the "gis" table is cleared out and replaced by the new data you are loading.  If you need to combine multiple extract files together to get the map coverage you need, you can append multiple files together with the [http://wiki.openstreetmap.org/wiki/Osmosis/Installation osmosis] tool before loading them into the database:
 
  osmosis --rb one.osm.pbf --rb two.osm.pbf --rb three.osm.pbf --merge --merge --wb combined.osm.pbf
 
  osmosis --rb one.osm.pbf --rb two.osm.pbf --rb three.osm.pbf --merge --merge --wb combined.osm.pbf
 
Once that is complete you can load the combined.osm.pdf into the database using the osm2pgqsl command shown above.
 
Once that is complete you can load the combined.osm.pdf into the database using the osm2pgqsl command shown above.
Line 197: Line 190:
 
  sudo systemctl enable httpd
 
  sudo systemctl enable httpd
  
===Bonus: Create an OpenLayers map web page so that you can play with your map tile server outside of IOC===
+
===Bonus: Create an OpenLayers map web page so that you can play with your map tile server===
 +
 
 +
The index.html shown below will display a full page map in your browser using your tile server.  This page uses the OpenLayers Javascript library to display the map.  I place this file in my /var/www/html directory making it the home page for my local web server, at the http://localhost URL
 +
 
 +
<pre>
 +
<!doctype html>
 +
<html lang="en">
 +
<head>
 +
<link rel="stylesheet" href="http://openlayers.org/en/v3.5.0/css/ol.css" type="text/css">
 +
<style>
 +
html, body {
 +
height: 100%;
 +
padding: 0;
 +
margin: 0;
 +
}
 +
.map {
 +
height: 100%;
 +
width: 100%;
 +
}
 +
</style>
 +
<script src="http://openlayers.org/en/v3.5.0/build/ol.js" type="text/javascript"></script>
 +
<title>OpenLayers 3.5 Test Page</title>
 +
</head>
 +
<body>
 +
<div id="map" class="map"></div>
 +
<script type="text/javascript">
 +
var map = new ol.Map({
 +
target: 'map',
 +
layers: [
 +
new ol.layer.Tile({
 +
title: 'localhost as XYZ',
 +
type: 'base',
 +
visible: false,
 +
source: new ol.source.XYZ({
 +
url: "http://localhost.localdomain/osm_tiles/{z}/{x}/{y}.png",
 +
attributions: [
 +
ol.source.OSM.ATTRIBUTION
 +
]
 +
})
 +
}),
 +
new ol.layer.Tile({
 +
title: 'localhost as OSM',
 +
type: 'base',
 +
visible: true,
 +
source: new ol.source.OSM({
 +
url: "http://localhost.localdomain/osm_tiles/{z}/{x}/{y}.png",
 +
})
 +
}),
 +
new ol.layer.Tile({
 +
title: 'Mapquest satellite',
 +
type: 'base',
 +
visible: false,
 +
source: new ol.source.MapQuest({layer: 'sat'})
 +
})
  
The index.html file attached to this page will display a full page map in your browser using your tile server which you can navigate much as you would in IOC. This page uses the same OpenLayers map software as does IOC. I place this file in my /var/www/html directory making it the home page for my local web server, at the http://localhost URL
+
],
 +
view: new ol.View({
 +
center: ol.proj.transform([0, 0], 'EPSG:4326', 'EPSG:3857'),
 +
zoom: 3
 +
}),
 +
controls: [
 +
new ol.control.Zoom(),
 +
new ol.control.ZoomSlider(),
 +
new ol.control.Rotate(),
 +
new ol.control.Attribution(),
 +
new ol.control.MousePosition({
 +
projection: 'EPSG:4326'
 +
}),
 +
new ol.control.ScaleLine()
 +
]
 +
});
 +
</script>
 +
</body>
 +
</html>
 +
</pre>
 +
[[Category:Geospatial]]

Latest revision as of 14:20, 31 December 2019

These instructions are specific to the Fedora 25 Linux distribution. They can be adapted to other distributions by adjusting package names and file paths as necessary, otherwise the process should be pretty much the same. These instructions install and run the map tile services under your main userid. Here we assume that is "ibmadmin". When complete you will have a "renderd" service running that generates map tile PNG graphics files into the /var/lib/mod_tile directory from where they are served up by your local web server. To use this as the map server in an application you would configure your base map as an XYZ map tile server with the URL http://localhost/osm_tiles/${z}/${x}/${y}.png

The installation process is as follows:

Install a collection of prerequisite software packages using your distribution's package manager

sudo dnf install postgresql postgis osm2pgsql postgresql-contrib python2-mapnik mapnik-devel httpd-devel mapbox-variant-devel carto redhat-rpm-config automake libtool gcc-c++ libmemcached-devel

If you haven't previously set up a development build environment on your workstation then also do this installation:

sudo dnf groupinstall "Development Tools"

Configure the postgresql database software and create a database called "gis"

Do the initial configuration and start postgresql:

sudo /usr/bin/postgresql-setup --initdb
sudo systemctl start postgresql
sudo systemctl enable postgresql

Create the "gis" database:

sudo -u postgres -i
-bash-4.3$ createuser ibmadmin
-bash-4.3$ createdb -E UTF8 -O ibmadmin gis
-bash-4.3$ exit

Add the postgis and hstore extensions to the "gis" database

sudo -u postgres psql
postgres=# \c gis
gis=# CREATE EXTENSION postgis;
gis=# CREATE EXTENSION postgis_topology;
gis=# CREATE EXTENSION hstore;
gis=# ALTER TABLE geometry_columns OWNER TO ibmadmin;
gis=# ALTER TABLE spatial_ref_sys OWNER TO ibmadmin;
gis=# \q

Tune postgresql for better performance by adding the following settings to /var/lib/pgsql/data/postgresql.conf: (Comment out the existing shared_buffers setting and restart postgresql after making these changes.)

shared_buffers = 4GB
work_mem = 100MB
maintenance_work_mem = 4096MB
fsync = off
autovacuum = off
random_page_cost = 1.1
effective_io_concurrency = 2

Install and compile the mod_tile software

mkdir ~/src
cd ~/src
git clone https://github.com/openstreetmap/mod_tile/
cd mod_tile
./autogen.sh
./configure
make
sudo make install
sudo make install-mod_tile
sudo ldconfig

Configure your web server to run the mod_tile module by creating the following two files and then restarting the httpd service:

/etc/httpd/conf.modules.d/10-mod_tile.conf

LoadModule tile_module /usr/lib64/httpd/modules/mod_tile.so

/etc/httpd/conf.d/mod_tile.conf

LoadTileConfigFile /usr/local/etc/renderd.conf
ModTileRenderdSocketName /var/run/renderd/renderd.sock
# Timeout before giving up for a tile to be rendered
ModTileRequestTimeout 0
# Timeout before giving up for a tile to be rendered that is otherwise missing
ModTileMissingRequestTimeout 30
Header set Access-Control-Allow-Origin "*"

Install the mapnik map tile rendering stylesheet

There are a number of different style-sheets publicly available and you can also build your own. We will use the OpenStreetMap default style sheet.

cd ~/src
svn co http://svn.openstreetmap.org/applications/rendering/mapnik mapnik-style
cd ~/src/mapnik-style
sudo ./get-coastlines.sh /usr/local/share

In order for mapnik to find the correct postgis database and the coast line data, you will need to configure the mapnik style-sheet to your local settings. In your style-sheet directory (e.g. ~/src/mapnik-style) there should be a directory inc. There are a number of files you need to adapt in this directory:

cd inc
cp fontset-settings.xml.inc.template fontset-settings.xml.inc
cp datasource-settings.xml.inc.template datasource-settings.xml.inc
cp settings.xml.inc.template settings.xml.inc

Now you need to modify each of these files:

settings.xml.inc

replace

<!ENTITY symbols "%(symbols)s">

with

<!ENTITY symbols "symbols">

replace

<!ENTITY osm2pgsql_projection "&srs%(epsg)s;">

with

<!ENTITY osm2pgsql_projection "&srs900913;">

replace

<!ENTITY dwithin_node_way "&dwithin_%(epsg)s;">

with

<!ENTITY dwithin_node_way "&dwithin_900913;">

replace

<!ENTITY world_boundaries "%(world_boundaries)s">

with

<!ENTITY world_boundaries "/usr/local/share/world_boundaries">

replace

<!ENTITY prefix "%(prefix)s">

with

<!ENTITY prefix "planet_osm">

datasource-settings.xml.inc

In this file you will need to enter your database settings. You are running postgresql on the same machine as the rendering stack, so you can comment out the parameters “password”, “host” and “port” with an HTML-style comment. This will enable mapnik to use the “unix local user” as an authentication method. Change the “dbname” from “%(dbname)s” to “gis”, “estimate_extent” to “false”, and “extent” to “-20037508,-19929239,20037508,19929239” so that the file now reads:

<!--
Settings for your postgres setup.
 
Note: feel free to leave password, host, port, or use blank
-->
 
<Parameter name="type">postgis</Parameter>
<!-- <Parameter name="password">%(password)s</Parameter> -->
<!-- <Parameter name="host">%(host)s</Parameter> -->
<!-- <Parameter name="port">%(port)s</Parameter> -->
<!-- <Parameter name="user">%(user)s</Parameter> -->
<Parameter name="dbname">gis</Parameter>
<!-- this should be 'false' if you are manually providing the 'extent' -->
<Parameter name="estimate_extent">false</Parameter>
<!-- manually provided extent in epsg 900913 for whole globe -->
<!-- providing this speeds up Mapnik database queries -->
<Parameter name="extent">-20037508,-19929239,20037508,19929239</Parameter>

Configure renderd

Change the the renderd settings by editing the /usr/local/etc/renderd.conf and change the following lines as shown (remember to change the "ibmadmin" username to your user’s name if necessary):

socketname=/var/run/renderd/renderd.sock
plugins_dir=/usr/lib64/mapnik/input
font_dir=/usr/share/fonts/dejavu
XML=/home/ibmadmin/src/mapnik-style/osm.xml
HOST=localhost

Create the files required for the mod_tile system to run (remember to change the "ibmadmin" username to your user’s name if necessary):

sudo mkdir /var/run/renderd
sudo chown ibmadmin /var/run/renderd
sudo mkdir /var/lib/mod_tile
sudo chown ibmadmin /var/lib/mod_tile

Create a file named /etc/tmpfiles.d/renderd.conf which contains this single line:

d /var/run/renderd 0755 ibmadmin ibmadmin -

This will ensure that the system recreates the /var/run/renderd directory after a reboot, when temp files are cleaned up.

Download the map data for the area you are working in and import it into the "gis" database.

Your demo system probably doesn't have enough space to hold the map data for the whole world. Fortunately, there are state-by-state or region-by-region subset extracts that can be downloaded and imported into the "gis" database to cover the area covered in your demonstration. You can get these extracts (*.pbf files) from http://download.geofabrik.de and load them up with

osm2pgsql -d gis -C 16000 --slim --number-processes 8 filename

Each time you do this the data in the "gis" table is cleared out and replaced by the new data you are loading. If you need to combine multiple extract files together to get the map coverage you need, you can append multiple files together with the osmosis tool before loading them into the database:

osmosis --rb one.osm.pbf --rb two.osm.pbf --rb three.osm.pbf --merge --merge --wb combined.osm.pbf

Once that is complete you can load the combined.osm.pdf into the database using the osm2pgqsl command shown above.

Note that renderd only generates map tiles that do not already exist in the /var/lib/mod_tile directory structure. If you update the data in your database, those updates will not be visible in your map unless you force renderd to recreate those tiles. I usually just delete all the map tiles every time I make an update, forcing renderd to start over from scratch:

sudo rm -fr /var/lib/mod_tile/*
sudo systemctl restart renderd

Test your installation

Make sure that your web server is running.

Start renderd in the foreground in a terminal window so that you can see any debugging messages:

/usr/local/bin/renderd -f -c /usr/local/etc/renderd.conf

There may be some iniparse messages about syntax errors for some of the comment lines in the renderd.conf file. You may safely ignore these. You may also safely ignore messages about a missing "unifont Medium" font.

Point your browser to http://localhost/osm_tiles/0/0/0.png You should see a small map of the world in the middle of your browser window.

Ctrl-C to stop the renderd command running in the terminal window before proceeding to the next step.

Configure your system to start the httpd and renderd services at boot

Create a service definition file for the renderd service, /usr/lib/systemd/system/renderd.service:

[Unit]
Description=renderd
After=multi-user.target
Requires=multi-user.target
 
[Service]
Type=simple
User=ibmadmin
PIDFile=/var/run/renderd/renderd.pid
ExecStart=/usr/local/bin/renderd -c /usr/local/etc/renderd.conf
 
[Install]
WantedBy=multi-user.target

Then do the following set of commands

sudo systemctl daemon-reload
sudo systemctl start renderd
sudo systemctl enable renderd
sudo systemctl enable httpd

Bonus: Create an OpenLayers map web page so that you can play with your map tile server

The index.html shown below will display a full page map in your browser using your tile server. This page uses the OpenLayers Javascript library to display the map. I place this file in my /var/www/html directory making it the home page for my local web server, at the http://localhost URL

<!doctype html>
<html lang="en">
<head>
	<link rel="stylesheet" href="http://openlayers.org/en/v3.5.0/css/ol.css" type="text/css">
	<style>
		html, body {
			height: 100%;
			padding: 0;
			margin: 0;
		}
		.map {
			height: 100%;
			width: 100%;
		}
	</style>
	<script src="http://openlayers.org/en/v3.5.0/build/ol.js" type="text/javascript"></script>
	<title>OpenLayers 3.5 Test Page</title>
</head>
 <body>
	<div id="map" class="map"></div>
	<script type="text/javascript">
		var map = new ol.Map({
			target: 'map',
			layers: [
				new ol.layer.Tile({
					title: 'localhost as XYZ',
					type: 'base',
					visible: false,
					source: new ol.source.XYZ({
						url: "http://localhost.localdomain/osm_tiles/{z}/{x}/{y}.png",
						attributions: [
							ol.source.OSM.ATTRIBUTION
						]
					})
				}),
				new ol.layer.Tile({
					title: 'localhost as OSM',
					type: 'base',
					visible: true,
					source: new ol.source.OSM({
						url: "http://localhost.localdomain/osm_tiles/{z}/{x}/{y}.png",
					})
				}),
				new ol.layer.Tile({
					title: 'Mapquest satellite',
					type: 'base',
					visible: false,
					source: new ol.source.MapQuest({layer: 'sat'})
				})

			],
			view: new ol.View({
				center: ol.proj.transform([0, 0], 'EPSG:4326', 'EPSG:3857'),
				zoom: 3
			}),
			controls: [
				new ol.control.Zoom(),
				new ol.control.ZoomSlider(),
				new ol.control.Rotate(),
				new ol.control.Attribution(),
				new ol.control.MousePosition({
					projection: 'EPSG:4326'
				}),
				new ol.control.ScaleLine()
			]
		});
	</script>
</body>
</html>