Monday, November 18, 2013

Silly Geographic Precision

I tread water in oceans of latitude longitude coordinates.  Every vector data set I encounter is big fat sets of latitude (up-downiness) and longitude (left-rightiness) bits of info which combine (the "co" in coordinates) to pinpoint a spot on the surface of the Earth (and those combine to trace lines or areas).  The more precise the coordinate's numbers, the finer the pinpointing.  But sometimes I see data (usually the results of address geocoding) with a ridiculous number of digits past the decimal place, implying waaaaay too specific a location.  For example, today I saw the sample below as the result of an address geocode.  I'll use a large courier font to connote just how epic this is...


That's a tight geocoding.  Not only does the level of precision pinpoint a building, it pinpoints a specific atom in the building.

Here's a breakout of coordinate precision by the actual cartographic scale they purport:

Decimal Places
 Actual Distance
Say What?
6 10 centimeters Your footprint, if you were standing on the toes of one foot.
7 1.0 centimeter A watermelon seed.
8 1.0 millimeter The width of paperclip wire.
9 0.1 millimeter The width of a strand of hair.
10 10 microns A speck of pollen.
11 1.0 micron A piece of cigarette smoke.
12 0.1 micron You're doing virus-level mapping at this point.
13 10 nanometers Does it matter how big this is?
14 1.0 nanometer Your fingernail grows about this far in one second.
15 0.1 nanometer An atom. An atom! What are you mapping?

As a reference, six decimal places of precision is generally plenty-good-enough territory for cartography.  Unless you are collecting the cornerstone base survey coordinate for a mechanical engineer, let's call this good.

Not all Longitudes are the Same
A degree of Latitude is about 68.71 miles, and that's pretty* consistent as you go north or south (when you climb up or down the ladder of latitude, each rung is the same distance).  A degree of Longitude is widest at the equator (about 69.17 miles) but gets narrower and narrower until they all pinch together right down to nothing at the the poles.  The examples above are pretty much best case examples when it comes to Longitude; they get even sillier when you move away from the equator.

A Fine Mesh
Latitude and Longitude lines are the tics of an imaginary mesh that covers the world -like pixels on a screen.  Every time you add a number to the right of each decimal in a Lat Long coordinate, you subdivide the mesh of the Earth by ten each way (bumping your resolution 100 times finer each step).  Things get crazy in a hurry and it's common to encounter data with not just meaningless, but deceivingly precise coordinates.  Just because something is precise, that doesn't mean it's accurate.

P.S. I read a really good book by Dava Sobel a while back on the surprisingly epic history of Longitude.  It's called Longitude.  If you are nerdy enough to have read down this far then it's a safe bet you'll enjoy it.


  1. "If you are nerdy enough to have read down this far then it's a safe bet you'll enjoy it." Actually I'm so nerdy I read it long before reading this post. ;-)

  2. Great post though potentially 'inaccurate' to say distance between lines of latitude are the same no matter how far north or south. That works for a sphere. Not for an ellipsoid. Given Earth is the shape of an m&m, squashed so it's larger round the equator, it makes the distance of 1 degree of latitude at a pole nearly 1 mile longer than at the equator.

    1. I updated the language to be softer on distance. Ah the dance of precision.

  3. Great and useful post. I'm printing copies for the students that visit us; full web address included for further cartographic enlightenment. Thanks.

  4. As a counter-argument to the position of this blog, we did some testing of error introduced by conducting a series of Helmert 7-parameter transformations some time ago - I.e. going from say UTM WGS84 -> LL WGS84 -> LL ED50 -> projected units on ED50, and then reversing the entire process.

    It was done to test the robust nature of various software applications in conducting coordinate operations.

    It's all a bit hazy now (possibly 2007), but keeping ridiculous levels of precision (possibly 12 dec places), meant that the error introduced was very small (to something like 7 or 8 dec places). This meant that the GI Systems used to read the data, treated the pre- and post- transformed data as equivalent.

    For interest, even with using huge and seemingly unnecessary precision, there was still a contest to be won. And the winner? FME by Safe Software.

    Also - "Longitude" is a real page turner... Should be on the reading list of every Undergrad Geography Degree Course comprising GIS, as should "Datums and Map Projections" by Jonathan Iliffe from UCL.

  5. Thanks for the insights, Andrew! Yes, I suppose atomic precision is useful for running the engines in reverse to see how well the transforms went. Also, FME is great; I second that.

  6. Thank you for this. I had often told people that 4 to 6 is enough, but never had anything solid and clear like your list to back it up.

    Longitude is a great book, I brought it up often in the classes I taught.