blankblank blank


Author Archive for Leszek Pawlowicz

Mashup And Map Demographic Statistics With DataMasher

Take one of DataMasher’s 90-odd datasets (demographic, economic, energy, health, etc. data by US state), add/subtract/multiple/divide it by another dataset, and plot the results as a choropleth map (plus get the results in tabular form):

2009-10-31_204803

Mashups are posted publicly for comment and ratings. Results may be significant, or be totally devoid of meaning; YMMV. DataMasher makes no claims as to the statistical validity of any of the maps created. You can also select and plot individual datasets by themselves. There’s a short video on how the process currently works, and DataMasher promises more datasets and capabilities are coming.

Via Lifehacker.




Another Tool For Identifying Garmin Map Tiles

A few months ago, I covered GMapTool, a program that lets you identify the name and coverage area of a Garmin .img map file, whose only outward identifying info is a cryptic 8-digit filename that doesn’t reflect anything about its contents. Some people who have download the most recent version of GMapTool have reported getting virus warnings from their AV programs. I’ve checked GMapTool with MS Security Essentials (a terrific free AV program, BTW; unobtrusive, and doesn’t use a lot of system resources) and Avast antivirus, and neither one has spotted a problem – I suspect that the warnings are a false positive, though I can’t be sure.

Regardless, if you’re leery of using GMapTool because of the virus warnings, an alternative program is Mapsource Map Explorer. It doesn’t have the additional capabilities of GMapTool to split and combine .img map files, plus it only works on maps installed in MapSource (GMapTool can work on any .img file in a directory, not just those installed in MapSource). But if all you want is data characteristics for a set of installed map tiles, it does the job:

MapSourceExplorer

While the identifying map number may have less than 8 digits (e.g. “”162923” above), the .img filename for that map tile will have enough leading zeroes to pad out the name to 8 digits, i.e. “00162923.img”.




G-Raster Version 1.3 Released – Raster Maps On Garmin GPS Units

I”ve posted about my free app G-Raster before – it converts raster imagery like topo maps and aerial imagery into the format compatible with recent Garmin GPS models (the Colorado, Oregon, and Dakota series). I’ve just uploaded version 1.3, which adds some major improvements:

– The option to rescale images that are too large for Garmin units (> 100 million pixels) to a smaller size; the previous version only let you crop the image

– Automatic rescaling of images to make sure that less than 100 tiles (the maximum for Garmin units) are created

– Display of the image resolution in meters per pixel, plus the maximum Garmin zoom level at which an image can be viewed without pixelization (aliasing)

– A view of the central part of the raster imagery, updated when the image is rescaled to a smaller size, or when the JPG compression is modified

– Your choice of 6 different algorithms for use in rescaling imagery

– Adjustment of the JPG compression factor using a slider, with immediate updating of the image view so that you can see the effects.

– Major additions to the help file

graster13




Draw Freely On Google Maps, Export In Multiple Formats

ScribbleMaps is one of the slickest Google Maps apps I’ve ever seen. Draw and/or place any of the following on a Google Map view:

  • Freehand lines
  • Straight lines
  • Circles
  • Rectangles
  • Arbitrary shapes
  • Text
  • Images (needs a web link to the image)
  • Standard Google Map/Earth placemark icons (select from multiple sets):

placemarks

You also have control of the color, opacity and line thickness from the toolbar, along with the ability to search for locations:

scribblemapcontrols

Once the shapes are created, you can move them, modify them by shifting vertices, rotate them, or erase them. Once done with your map, you have multiple export options:

  • JPEG
  • Print directly from browser
  • Embeddable widget (no Google Maps API key required):

  • Facebook map
  • Regular Google Maps view
  • Google Earth plugin (though this didn’t work for me)
  • KML file (for viewing in Google Earth)

scribblemapsGE

  • GPX (for export to your GPS); only exports points and lines, the only geometric shapes that GPX files support

You can also save your map for future editing, and get a dedicated web link for it; you choose a password for it so that only you can edit it, or delete it later.

Very slick, and very cool; I’m putting ScribbleMaps into my main bookmarks list.




Historical Marker Database

The Historical Marker Database (HMdb.org) is a voluntary effort to document and catalog historical markers, mainly from the US and Canada (a few markers from foreign countries are also listed). Markers are listed by category and state; you can also search by multiple parameters (keywords, postal code, county, etc.). Marker info pages have a photo of the marker, a copy of the marker text …

histmarker

… latitude/longitude of the marker, a context photo showing the general surroundings of the marker, and a list of other nearby markers. A link will take you to a Google Maps page showing the location of the marker and other nearby markers; a link at the top of the map page will let you download a GPX file with all the marker positions for upload to your GPS unit (TomTom POI files are also available). There’s even a custom version of the page for mobile phone; enter your latitude/longitude and get a listing of up to 20 markers near your current location.

The site also accepts submissions of historical makers not yet in the database (requires free registration).




One-Click Reverse Geocoding In Google Maps

Digital Inspiration’s Find Postal Address … site is a one-trick pony, but does it well. Zoom to a location in Google Maps, click on the map, and a pop-up will give you the closest postal address to the point you’re clicking on:

postaladdressinGoogleMaps

A search box aids in zooming in quickly to a desired city or zip code.




Tools To Help With Creating Garmin KMZ Files With G-Raster

Last post, I covered G-Raster, a free program I wrote that converts georeferenced imagery (GeoTiffs, MRSID, BSB, IMG, and graphics with worldfiles) into KMZ overlays compatible with some of Garmin’s more recent handheld GPS models (the Colorado, Oregon, and Dakota series). While G-Raster does a good job by itself on most such imagery, like that found at the USGS Seamless Server, there are a few programs that may be more convenient for obtaining georeferenced raster graphics (topo maps and aerial imagery), and modifying the imagery if necessary.

USAPhotoMaps: I’ve covered this free program before; it downloads USGS topographic maps, black-and-white 1-meter aerial imagery, and urban 0.25-meter color imagery from the Terraserver site. But you can also export JPG copies of the imagery using the Big JPEG function (File => Create a BIG jpeg file), which also exports a worldfile in UTM NAD83 coordinates. G-Raster has built-in support for these Big JPEG files; you only need to select “USAPhotoMaps Big JPEG” as the file type, then select the exported image file (found in the program directory after export). You’ll need the UTM zone as well – get that by choosing UTM coordinates for display (View => Lat/Lon => UTM). The imagery is in JPEG format, and won’t be in quite as high quality as that from the USGS Seamless Server, but should be more than good enough for most uses. As a bonus, you can also create GPX waypoint files directly in USAPhotoMaps using the map imagery; this lets you create both the navigation points and map imagery at the same time.

MicroDEM: I’ve also covered MicroDEM many times on this site. A small subset of its many functions can be helpful in editing/modifying imagery for use in creating Garmin KMZ overlays:

– The maximum pixel size for creating a Garmin KMZ overlay is 100 million pixels; if your image is larger than that, G-Raster will modify it to meet that criteria. But you can get better control over the final results by modifying the image yourself. MicroDEM lets you crop an image, or rescale it to a lower resolution.

– MicroDEM also offers direct access to Terraserver imagery, and can export it in GeoTiff format. While a bit more cumbersome than USAPhotoMaps, the final results are of slightly higher quality, and also offers the capability to choose the pixel resolution for the Terraserver imagery. For example, USAPhotoMaps exports black-and-white aerial imagery in either 1-meter per pixel or 8-meter per pixel resolutions; MicroDEM offers the choice of 1,2,4,8,16,32 and 64 meter per pixel resolution. By trading off resolution, you can cover larger areas in a single KMZ overlay file.

– Finally, MicroDEM can open shapefiles, color them thematically, and export the map as a G-Raster-compatible GeoTiff. While there are tools that can convert shapefiles into Garmin vector maps that display much more quickly than raster maps, this is a far easier approach, and gives you better control over the final appearance of the imagery as viewed on your GPS.




Create Garmin-Compatible Raster KMZ Files From Georeferenced Imagery With G-Raster

Garmin recently announced support for raster imagery like aerial photos and topo maps for some of their handheld GPS units, specifically their newer product lines: the Colorado, Oregon, and Dakota series. See recent posts at GPS Tracklog and GPS Fix for more information on this new feature. It requires download and installation of beta firmware on your Garmin GPS unit, and conversion of the raster images to a compatible format. Fortunately, the compatible format is the open format KMZ image overlay, which can be created in Google Earth:

  1. Zoom to the desired location in Google Earth
  2. Load the map or aerial image as an Image Overlay
  3. Calibrate the image by manually positioning, rotating and resizing it until it corresponds to the underlying Google Earth imagery.
  4. Save it as a KMZ file.
  5. Copy it to the \Garmin\Custom Maps folder on your Garmin GPS unit.

A fairly straightforward process, but there are some drawbacks:

  • The image must be no larger than one megapixel in size (1024 x 1024; 2048 x 512; etc.)
  • You can chop an image into subsections/tiles and combine them into a single overlay file, but this requires individually calibrating each tile
  • You can have no more than 100 such tiles loaded onto your GPS unit
  • If you have an image that’s georeferenced with either embedded geodata (like a GeoTiff, ERDAS Imagine, MRSID or BSB file), or a worldfile, there’s no way to use that georeferencing data, and you have to manually convert the file to JPG format

I’ve just finished version 1.0 of G-Raster, a utility that helps solve some of these problems:

  • Loads map/aerial imagery files with embedded geodata or worldfiles
  • Automatically extracts the geographic data, and optimally reprojects the image
  • Chops images larger than 1 megapixel into smaller subtiles, and calibrates all of them
  • Converts the image into JPG format
  • Creates a KMZ file that can be copied over to your Garmin unit

You can find the link to the latest version of the program installer here (look for the G-Raster download link, then the link that says “Click here to start download”). Once installed, run the program, and it will walk you through the process step by step.

graster

Step 1: Loading The Data

G-Raster accepts four basic types of georeferenced raster imagery:

Type 1: Files with the geographic data embedded in the graphic file itself. These include:

  • GeoTiffs: TIFF images with embedded geodata
  • MRSID imagery (file extension .sid or .mrsid)
  • NOAA BSB imagery (file extension .kap)
  • ERDAS Imagine format (file extension .img

Note: These must include all geographic data, including the type of coordinate system used. A great source for free USGS topo maps and aerial imagery is the USGS Seamless Server, which offers this data in the compatible GeoTiff format. Links to that site and other useful sources of data are available from the program menu.

Type 2: JPG imagery (“Big JPEG”) generated by the free program USAPhotoMaps. You will need to specify the UTM zone for the map you want to use.

Type 3: General graphic files with geodata in worldfile format for the UTM coordinate system (e.g. JPG with .jgw, BMP with .bpw, GIF with .gfw, TIF with .tfw). You will need to specify the UTM zone for the map, and also the datum and hemisphere if appropriate (default is northern hemisphere, WGS84 datum)

Type 4: General graphic files with geodata in worldfile format in any coordinate system. But you will have to: a) Know what that coordinate system is; b) Look up its EPSG code; c) Enter the code in the specified box

The process is easiest for Type 1 files; click the “Load …” button, and choose the file you want to create a KMZ overlay for. The program will automatically process the file, and take you directly to Step 3.

For Type 2 files, you will need to select the JPG file created by USAPhotomaps, normally placed into that program’s main directory. You will also need to specify the UTM zone for that map; you can get this information from USAPhotoMaps directly by choosing View => LatLon => UTM from the menu when viewing the maps you want to use; the UTM zone will be listed at the top. Alternatively, try one of the links under the UTM/EPSG Info menu. G-Raster will highlight the input section in red to remind you that it needs that data:

redcode

The process is similar for Type 3 files, except that you’ll have more choices for types of graphics (JPG, TIFF, BMG, GIF), and in addition to the UTM zone you’ll need to make sure the correct datum and hemisphere is chosen.

The process for Type 4 files is like that for Type 3 files, except that you will need to enter the EPSG code for the data in question.

Step 2: Process the raster graphic

After you enter the required data for files of Type 2-4, the button marked “Process Raster Graphic” will be enabled. Click on this button to start the process of converting the map graphic into the correct format. When it’s complete, a message will show up in green directly below this step, and the “Create KMZ File” button will be enabled. You can now go to Step 3

Possible problems:

– If the file is larger than 100 million pixels, it will be impossible for the program to process it as is; you’ll have the option of cropping the outer edges of the picture to bring it down to an acceptable size, but this will eliminate the map image data at the edges.

– In rare cases, an image close to but just below the maximum pixel count may not be able to be chopped up into 100 tiles or less. Future versions of the program will offer you the chance to resize the image slightly to fix this problem, but for now, you’ll have to crop or resize the image yourself.

Step 3: Set Parameters For KMZ Files

You’re almost ready to create the KMZ overlay file, but there are several options you might want to change:

KMZ Filename: The final output file will be saved using the name specified here in the same directory as the graphic file used to create it. Default is “Overlay_name”, and I’m assuming you’ll want to change this. If you choose a name that corresponds to a file that already exists, you’ll get an error message when you try to create the file.

JPG Quality: The output imagery is required by Garmin to be in JPG format. You can adjust the quality of the image from highest (100) to lowest (50 here) to reduce the total filesize, which may improve speed. Garmin’s recommended JPG quality number is 80, but you can experiment with lower/higher numbers to see which works better for you. One possible complication: the maximum image filesize is 3 MB, and in very rare circumstances choosing a high quality (98-100) might result in a tile that’s too large. Wouldn’t worry about it, but it’s something to keep in mind.

DrawOrder: Garmin offers two options for viewing the raster imagery. With On Top, the raster image displays above everything else, including any loaded vector maps. With Below Vector, the raster map still sits on top of some vector imagery (like lakes and parks), but can have other vector data on top of it (like roads and contour lines).

50% Transparency In Google Earth: Makes the overlay partially transparent in Google Earth, to help with determining whether it’s lined up correctly with underlying imagery. Has no effect on the map’s appearance in the Garmin unit’s display.

Rescale Factor / Rescale Image: Rescales the image to a smaller size based on the rescale factor entered ( > 1 only). For example, a rescale factor of 2 resizes the x and y dimensions by two, making the image a factor of 4 smaller in total pixel size. The advantage of this is to reduce the total number of tiles required to cover the image, in case you want to create multiple overlays with a total number of tiles less than the maximum 100. The disadvantage of this is that you will lose some image resolution, but depending on the Garmin zoom level you view the image at, you might not even notice the diminished resolution.

Step 4: Create KMZ File

Once you’ve set the parameters at left as desired, click the “Create KMZ File” button, and in a few seconds a message will pop up telling you that the KMZ file has been created in the same directory as the original graphic file.

If you like, you can now change some of the parameters, and create another KMZ file; this way, you can experiment with different settings. Just be sure to specify a different name for every one.

Step 5: Check the file in Google Earth

IMO, this is a mandatory step – you have to make sure the map is positioned correctly before trying to use it in your Garmin GPS unit.

You should have Google Earth installed on your computer for this step; if you don’t, go to the Google Earth website, and download/install the program on your computer. You can then double-click on the KMZ file created in Step 4 to load it into Google Earth. The KMZ overlay will be listed in the “Temporary Places” section of the info pane at left, and Google Earth will automatically zoom into the area covered by the map. Clicking the checkbox next to the overlay name will turn the image on and off, helping to see if it lines up correctly. If you chose 50% transparency as an option, the map image will be slightly transparent so that you can see what’s underneath – this may help you see how well the image is placed:

transparent_overlay

Depending on your location, you may notice a shift in position between the KMZ overlay and the underlying imagery of anywhere from 0 – 100 meters, with 10-20 meters being common for the US. This is not an indication of an error in the overlay, but rather a reflection of a fundamental limitation with Google Earth. The geographic position of Google Earth imagery isn’t always 100% accurate; see this post for more info. Unfortunately, there’s no way now to know how far off the imagery is shifted, which in turn imposes a limitation on how confident you can be that your overlay will be calibrated correctly in your Garmin unit.

Step 6 – Load The Overlay For Use In Your Garmin GPS Unit

Until the Garmin firmware is out of beta, I strongly recommend that you only install KMZ files on an external memory data card, not in the internal memory of your Garmin unit. This way, if there’s a problem, you can fix it by removing the card.

1. Take the external card, and put it into a separate card reader (not the Garmin itself). It will show up on your computer as a separate drive.

2. Open the drive, and go to the \Garmin\CustomMaps folder (if it doesn’t exist, you’ll have to create it)

3. Copy the KMZ raster overlay file into this folder.

4. Install the external memory card into your computer.

5. Follow Garmin’s instructions on its use, including turning it on and off.

In my next post, I’ll offer some tips and tricks to make the most of G-Raster.

Additional support for this new Garmin feature is available at the Garmin Custom Maps forum, including a helpful post if you have problems after installing a map.