Annoyed by traffic lights? - Solve it with a gps logger, Scala, GeoTools and qGis

Jeroen van Wilgenburg

I live in Utrecht, a city where it seems the municipality tries to annoy car drivers as much as possible. Public transport isn't an option, even despite the traffic jams it's slower, less reliable and don't get me started on the attitude of bus drivers. When you want to travel from one place in Utrecht to another it's often a good idea to find the nearest highway, it might be much longer, but often much quicker. Instead of complaining (which will stop now ;) ) there might be a solution. I have a theory that a trip is much faster when you avoid traffic lights and use the highway as much as possible. To prove that theory, test my new (Android 2.2) phone, fresh up my Scala knowledge and have a cool pet project I started logging my trips.

In this article I will show you how to collect the data, parse and convert it to a usable format and show it on a map. This is just a first prototype and proving my theory will probably take a lot more time which I don't have because I'm in traffic jams all the time ;).
If you don't like all the technical mumbo jumbo, just scroll to the maps, they're cool to look at.

Preparation

Since I drive a Peugeot I had some trouble with GPS reception on my phone, because of the infrared coating on the windscreen. To solve this problem I used an external Bluetooth antenna and told Android to use that antenna. Bluetooth GPS is a nice solution for that.
Other reasons to use an external antenna are that they are much more accurate, save the battery of your phone and that you can place the antenna where the reception is best (the black rectangle behind your rear view mirror is a perfect place and this is one of the reasons this rectangle exists).
Peugeot 406 Windscreen infrared coating
When you have trouble firing up the Bluetooth GPS try disabling and enabling the GPS and/or Bluetooth on your phone. I had some trouble with my car kit, but when you connect the GPS first and then the car kit you can still use both. You have to check the connection with the GPS when someone calls you, a while ago I lost my GPS connection when my colleague Mischa called me to tell me that the left lane was much faster ;)

For the logging I used GPS Logger for Android, a simple application that writes .gpx and/or .kml to the /mnt/sdcard/GPSLogger directory. You can even set the log frequency. Note that the logging is not done a the set interval, it's just a suggestion. The antenna spits out a stream of data, when your phone is busy it just skips data. So be aware of this when processing the data and don't hardcode the interval, but use the timestamp from the antenna.
There are other logging applications available, some even draw a trace. I prefer a very fast application, I don't want to wait for an application to start up when I start my car.

Parsing the data

I decided to use the .gpx data and skip .kml. I already have some experience with gpx, but kml shouldn't be a problem. The fields I'm interested in are the coordinates (x,y) and the timestamp. Course and speed are also very useful, but that's derived data for which I'm going to use GeoTools. Since GPS doesn't really know when you're stationary you get a lot of garbage at low speeds. For now it's good enough to assume that you're stationary when the speed drops below 5 kph. Another thing is that when you remove trackpoints from the log the speed field becomes invalid, I want the average speed on a leg of the track, not at the trackpoint itself.

To parse the data I used Scala. This sample chaper explains how to parse xml in Scala. The structure of a trackpoint in the gpx is as follows:

<trkpt lat="52.131231666666665" lon="5.011798333333333">
 <ele>49.4</ele>
 <course>329.36</course>
 <speed>29.8172</speed>
 <src>gps</src>
 <time>2010-12-23T07:23:21+01:00</time>
</trkpt>

I'm interested in lat (y), lon (x) and time. With these fields I can calculate the speed and create my map.

Displaying speed

I thought it would be nice to display each leg of the track in a color that represents the speed. To display the maps I use qGis, a desktop application to display GIS-data. QGis can load all kinds of formats (shapefiles, simple .csv's and there are lots of plugins available).
After trying to generate shapefiles with GeoTools for a while I decided points were good enough (instead of legs of a track). After all I'm interested in the time spent at traffic lights.
In the nightly versions of qGis 1.7 there is a new and improved delimited text plugin. The guys at Linfiniti even made an instruction video for it! With version 1.7 it is possible to import well known text. This immediately solves my problem, unfortunately it is not yet available for Mac Os yet.

So, for now I have to show a map with points:

To import the data I converted the parsed xml to .csv first. A snippet from the output:

lonX, latY, speed, dateTime
5.074956666666667, 52.09170666666667, 37.266029160336, 2011-01-04T09:07:35.000+01:00
5.07473, 52.091948333333335, 31.436951983284487, 2011-01-04T09:07:38.000+01:00
5.0746, 52.092083333333335, 26.498677863536287, 2011-01-04T09:07:40.000+01:00

Calculating the speed

To calculate the speed I used GeoTools in Scala:

 /**
 * @param start (starting point (lon, lat))
 * @param destination (end point (lon, lat))
 * @return Distance in meters
 */
def calcDistance(start: (Double, Double), destination: (Double, Double)): Double = {
  val geodeticCalculator = new GeodeticCalculator();
  geodeticCalculator.setStartingGeographicPoint(start._1, start._2);
  geodeticCalculator.setDestinationGeographicPoint(destination._1, destination._2);

  // To be computed only once for ever for a given GeodeticCalculator
  val ellipsoid = geodeticCalculator.getEllipsoid();
  val sourceUnit = ellipsoid.getAxisUnit();
  val targetUnit = SI.METER;

  val converterTo = sourceUnit.getConverterTo(targetUnit);

  // To apply after each distance calculation
  converterTo.convert(geodeticCalculator.getOrthodromicDistance());
}

Speed is calculated the following way: (distanceInMeters / deltaMillis) * 3600

Delimited text layer

The csv file can be imported into qGis with the delimited text plugin. QGis is smart enough to see that lonX and latY are the x and y coordinates. When the layer is imported you will see a map with points, all in the same color. To change the colors double click on the layer in the legend on the left. Pick symbology and graduated symbol as legend type. I increased the number of classes to 7 and hit the classify button. QGis generates some kind of gradient for you, which can be changed easily. With save and load style you can save and load these settings for later use.

OpenStreetMap raster layer (georeferencing)

To create a base layer I downloaded a map from OpenStreetMap. When you go to the site and have a nice view of a map you want to use in qGis you can click on the Export tab. This opens a dock in which you can export an image. Remember the coordinates, you need them in qGis!

In qGis open the Georeferencer plugin. Click the first button on the toolbar and open the image you saved. Zoom in to the upper left corner of the image and add dot (the three dots in the toolbar) and enter the coordinates you just saved. Do the same for the lower right corner. Click the green play button and click on the folder right of output raster. You now have a georeferenced tiff image which can be added as a raster layer in qGis.

Of course this map has only one zoom level, which looks a bit distorted (see the screenshot in the next paragraph).

I'm not sure if the plugins I mentioned are installed by default, if they're not and you have trouble please let me know, then I'll add a section to this blog. Usually plugins can be enabled/disabled und the menu plugins, manage plugins.

Displaying stationary time

To display the stationary time I used the interval the speed drops below 5 kph. This time is included as a field in the csv, which we can use in qGis.
In qGis create a delimited text layer. At symbology pick the traffic light symbol and increase its size (I used 5 points). Now add a label. 'Field containing label' should be the field that has the delay in it. The label displays on top of the symbol by default, below right is a better place. Increase the font size and color (ie. 24 and black) and add a buffer (2.0, white).
Now there will appear traffic light symbols on the map with a delay next to it. Of course we also want to display a trace of the driven route. This can be achieved with a gpx layer. Go to the menu Plugins, Gps, Gps tools.
Click on the tab Load GPX..., browse for the gpx and select the Tracks at Feature types.
At the properties the line size and color can be adjusted.

The purple lines represent two different trips. The trip I analyzed starts at the traffic light with number 17, via 18 and ending with 15. On OpenStreetMap:  Exit 17 on highway A12.

Conclusion/Future

I always thought I was waiting minutes in front of traffic lights, it's not that bad. It rarely happens that you have to wait more than one minute. Guess I have to work on my impatience ;)

I display the stationary time and this doesn't necessarily have to be at a traffic light. So a stop symbol might be more suitable ;)

The next step is 'snapping' the gps data to the OpenStreetMap roads and calculate the average speed. I'm also working on getting nicer looking maps (now the maps are distorted and only suitable for one zoom level), it probably has something to do with coordinate systems and I might give OpenLayers a shot.

Androgeoid is a very nice site with reviews of Android apps, it really helped me to pick the right tools for the job.

Sources

GPS Logger for Android
Bluetooth GPS

Androgeoid

GeoTools
OpenStreetMap
Delimted text plugin video
Herding XML in Scala

Comments (4)

  1. Friso - Reply

    February 3, 2011 at 8:07 pm

    Nice work! Would be cool if you'd put the code up on github or similar together with a little howto that could get others going with this...

  2. Bernd - Reply

    November 25, 2011 at 2:59 am

    nice article.
    About your image "problem" (if you haven't solved it yet by yourself):
    Load your gpx-Layer.
    Install the Open Layers Plugin from the repositories.
    Add the OSM- or Google map of your choice from the plugin.
    Disable everything on the map except the OSM Layer.
    File -> Save Image and then add the new raster to your project as a referenced background image (saves you a lot of reloading time of the map layer when moving or zooming)

    Keep in mind that the plugin sets your project crs to Google Mercator (900913)

  3. Jeroen van Wilgenburg - Reply

    November 25, 2011 at 11:22 am

    Thanks a lot! I hadn't solved it yet.
    At FOSS4G I heard there was a QGIS plugin to display OSM maps, but didn't realize it was called OpenLayers ;)

    I had some trouble installing it, but this blog helped me out:
    http://indicatrix.wordpress.com/2011/04/06/basemaps-in-qgis/

  4. Nathanael Boehm - Reply

    April 19, 2013 at 2:38 pm

    Hi, I'm on QGIS 1.9 and I'm playing around with GPX data and trying to do similar visualisations, but I'm trying to figure out how to display the GPS track points as a line instead of an ugly sequence of dots, similar to what ExpertGPS does with Measure, and GPSVisualiszr does with Elevation. Got any ideas or thoughts? Everything I've thought of means losing attributes, including splitting the route line by the 300 points/vertices into 299 mini lines. At the moment I'm using the new blending modes to faux mask oversized track points with the underlying track line but it only works if the background is white which means no map layers.

Add a Comment