blankblank blank

Archive for the 'MapWindow' Category

Crowdsourcing And Coordinating Data Collection With Handheld GPS Units

Got an email today from someone with a data collection/coordination/assembly problem. They have multiple people out in the field with Garmin GPS units recording data, and bringing it back to a central location for collation/combination when they’re done. Since Garmin GPS units have limited capabilities for data storage – coordinates, name, comment, date and time, elevation, and that’s pretty much it – additional data associated with a point has to be recorded by hand. Once back from the field, data has to be downloaded from individual units, associated with the additional attribute data from forms, all the data combined together into a single dataset, then converted into GIS-friendly format. And the process they had come up with wasn’t really working well for them.

I think this highlights some of the major limitations of classic stand-alone handheld GPS units. They’re really designed for us in personal data collection, not combined data collection; assembling data from multiple units can take a lot of work. Plus, their limited data collection capabilities require offloading data attribute acquisition to other formats (e.g. pencil and paper), adding the addition problem of associating that data with coordinates later on. There are lots of professional solutions for these problems, like Trimble or MobileMapper GPS units, Terrasync and ArcPad software, but these can be complicated and expensive. I challenge you to find anyone with enough patience to use Terrasync for a single day without swearing at least once; I know I can’t  ;-).

For a few years, I’ve had CyberTracker on my list of potential topics to post on. CyberTracker is a terrific data acquisition and collation tool for field data with tons of great features, like custom data acquisition form design for easy data entry, moving maps, easy data collation and conversion to GIS-friendly format. Plus, the software is free, and the hardware is (relatively) affordable. But unfortunately, I think time and technology are passing Cybertracker by:

  • While the central data software runs on Windows, the field data collection software runs on old-school Palm OS and Windows Mobile. Palm OS is dead; Windows Mobile development has stopped with version 6.5, and only maintenance updates are scheduled. So the useful lifetime of any data collection system built around CyberTracker is limited. If you need a fast/cheap/short-term solution, CyberTracker is free for non-profit use, and you could probably pick up the hardware very cheaply on eBay. Long-term, I don’t see it having much of a future in its current form.
  • The data collation model is also becoming obsolete. With CyberTracker, you have to bring all the data units to a single data download/collation computer. While there are still places in the world where this might make sense, an increasingly-connected world means that data download/collation to the cloud instead of an individual computer makes more sense.

I’m really excited about the possibilities for crowdsourced geographic data collection and collation using portable devices running Android OS (yeah, iOS too). There are already several interesting apps for doing this, and I hope to cover some of those soon on my AndroGeoid website. And I suspect that the small number of apps that can currently do this will be quickly joined by far more apps, and far more capable apps, in the very near future. But that doesn’t help with my emailer’s current problem. I suspect there are many different ways you could do this, but here’s the first approach that came to me using all-free software and services.

1. Make sure that everyone on the project has a Google account (i.e. Gmail); completely free.

2. Have the project leader created a single main data spreadsheet on Google Docs, with all the desired data attributes (e.g. point name, coordinates, comments, additional data fields from the paper forms, etc.) and share a link to that spreadsheet with other project members so that they can edit it as well.

3. Project members can download data from their Garmin units using DNRGarmin, and then export the data from DNRGarmin in CSV format.

4. Load the data into the spreadsheet program of your choice, and add/edit data from data forms to make it conform to the data structure of the main Google Docs spreadsheet.

5. Copy the data cells in the spreadsheet program, and paste them into the main Google Docs spreadsheet. Note: Use Ctrl-C and Ctrl-V to copy and paste cells into Google Docs instead of using the Google Docs Edit menu to perform those operations, as the latter doesn’t seem to work for pasting data from different applications into Google Docs. You no longer have a single computer as a choke point for data entry; multiple people can add data to a spreadsheet at the same time, and Google Docs will coordinate data entry so that nothing is lost. And if you save the spreadsheet data from steps 3 and 4 as separate files, you’ll have backup copies of the original data as well.

For those who aren’t comfortable with working with spreadsheets, Google Docs lets you set up a “Form” to let anyone add data to a Google Docs spreadsheet directly; however, this increases the chances of coordinate data entry error.

6. Now that you have all the data centralized in Google Docs, you have lots of flexibility in how you can handle it:

  • Export the data in CSV format, and you can then import it into any GIS program that supports CSV data. If your GIS program doesn’t, use MapWindow to convert your CSV file into shapefile format first.
  • Unlike shapefile attribute tables, where adding/removing/re-ordering attribute data columns can be a pain, you can easily perform those operations in Google Docs and then re-export the data in CSV format.
  • Convert the data directly into a continuously-updated KML network link for display in Google Earth or Maps using Google’s Spreadsheet Mapper tool.
  • Use any of Google Docs built-in tools to analyze/plot/sort/visualize data, including their way-cool Fusion Tables.
  • And I’m sure there are more options I haven’t thought of.

Know an alternate approach? Have any additional ideas?  I welcome your links and suggestions in the Comments section below.

MapWindow GIS Updated To Version 4.7; Next-Gen Version 6 Beta Available

It’s been over a year since I last posted about a version update (4.5) for the free Windows GIS MapWindow (topic of multiple previous posts). Just noticed that version 4.7 was released just a bit more than a week ago. Improvements since 4.5 include:

  • Stability improvements, particularly when dealing with large numbers of shapes in shapefiles
  • TauDEM/Automatic Watershed Delineation Help Files
  • Progress indicator improvements in many plug-ins
  • Unit conversions available for more standard units and more accurate
  • New image properties in MapWindow (Use Histogram, Allow Hillshade, Buffersize, Image Color Scheme, Set to Grey
  • Zooming with mouse wheel
  • Updated GDAL projection library
  • Added GEOS shape functions (Contains, Crosses, Touches, Intersects, Overlaps, etc.)
  • New printing plug-in
  • New image properties (Use Histogram, Allow Hillshade, Buffersize, Image Color Scheme, Set to Grey)
  • Bug fixes and speed improvements
  • Improved documentation

And if you want to look at the future of MapWindow, there’s a Codeplex development page for MapWindow 6 where you can download the latest beta (which looks to be in a pretty usable state) and fairly extensive documentation. What’s the difference between MapWindow 4.x and 6? From the Forward:

The MapWindow 6 platform is a major step forward. It is completely .NET based, has no COM or ActiveX references, and is totally built around the idea of interfaces and components. If you are a developer you can rebuild the MapWindow 6 desktop application in about 10 minutes by simply assembling the correct components and hitting the “compile” button. If you are an end user, you will find significantly improved symbology and printing capabilities and a new modeling toolkit and environment that are just plain awesome.

First stable release of MapWindow 6 is scheduled for the first half of 2010, hopefully at the 1st International MapWindow Users And Developers Conference.

Assigning Attributes From A Polygon Shapefile To Another Shapefile

Here’s a recap of yesterday’s request from TD:

Wondering if you know of an easy way to create centroids (points) from polygons – I am using MapWindow and various other free tools. What I am ultimately trying to do is assign attribute data (parcel #) to a polygon shapefile representing buildings.  In other words, for each building polygon I want to assign it the correct parcel #.

Yesterday’s post went through the first part of this process, converting the building polygon shapefile  into a centroids (approximate building geometric centers) point shapefile, carrying over the building shapefile attributes to the corresponding centroid points. Now I need to assign attribute data from the parcel polygon shapefile to the centroids.

Continue reading ‘Assigning Attributes From A Polygon Shapefile To Another Shapefile’

Get DEM Elevations For A List Of Points

YP asks:

…would you know an easy (and free) way to query a DEM (Digital Elevation Model) to get elevations for a list of points (potentially as many as a few thousand)

Well, I know one way to do it with free software:

Continue reading ‘Get DEM Elevations For A List Of Points’

New Stable Release Of MapWindow GIS

The open source GIS MapWindow (Windows only) has just put out its latest Stable Release (SR), version 4.5. This is the first SR in about 8 months. New features include:

Continue reading ‘New Stable Release Of MapWindow GIS’

Importing Google Earth Imagery Into A GIS

This post has gotten a very strong reaction from many people, mostly negative. They say use of Google Earth imagery in the manner outlined here is inappropriate, an illegal violation of the Google Earth EULA/TOS. More specifically, words like “theft”, “stealing” and “dirty” have been used to describe this. I feel that this use of Google Earth imagery for personal and non-commercial uses falls under the guidelines of “fair use”, and I explain why here. “Fair use” is not theft. But I acknowledge that I could very well be wrong about this falling under the umbrella of “fair use” protection, in which case it would be illegal. I’d welcome opinions from those knowledgeable about the rights and limitations of “fair use”, and will pull the post if an expert on these issues says that using Google Earth imagery as described below doesn’t constitute “fair use”.

You can save the displayed view in Google Earth by using the File=> Save => Save Image command (Ctrl-Alt-S is the shortcut), which saves the contents of the view window as a JPG file. But in order to use this image in a GIS program, it has to be georeferenced, i.e. coordinates assigned to the pixels. There are free image georeferencing options out there, and I’ll cover some of those soon, but that’s not necessary in this case – there’s a simpler way to get the georeferencing data for an image exported from Google Earth.

You will need to have installed Google Earth (of course), the free GIS program MapWindow, and the Shape2Earth plugin for MapWindow. Shape2Earth’s main function is converting shapefiles to Google Earth KML files, and while the full version costs $29.95, it will work in unregistered demo mode for up to 500 shapefile vertices, and here for georeferencing Google Earth imagery.

1. Open Google Earth, and zoom in to the area of interest.

2. Double-click on the “N” in the compass at the upper right to orient north to the top of the screen. Also double-click on the “0 tilt” icon (the box with the “X” in it on the left in the tilt adjustor) to set the tilt angle to zero.

3. Open MapWindow; if you haven’t already activated the Shape2Earth plugin, do so from the plugins menu.

4. From the Shape2Earth menu in MapWindow, select “Get Image from GE”, and click on the “Capture Image” button. If you haven’t already oriented the image with north at the top and zero tilt, you will be prompted to do so.

5. Save the image with the desired name as a JPG file. This will save a black-and-white jpg image of the current view in the Google Earth window, along with a worldfile (*.jpgw) containing the georeferencing information.

6. Go back to Google Earth, and without moving or changing the view in any way, save the current view (File => Save => Save Image) with the same name and location as the image in step 5,. This overwrites the previous black-and-white image with a color one of exactly the same area.

You now have the Google Earth image with the georeferencing data contained in the worldfile (geographic coordinates, WGS84 datum), which most GIS programs can open and read successfully. If there’s a problem, try changing the extension of the worldfile from *.jpgw to *.jgw.

Limitations of this approach:

  • The view in Google Earth is orthographic, i.e. you’re looking at a sphere from above. For views that are zoomed to view a relatively small area, the area is essentially a flat plane, and the georeferenced image shouldn’t have a large deviation from the true coordinates. The larger the area covered, the greater the deviation between the true and georeferenced coordinates.
  • Coordinates in Google Earth can be off from the true coordinates even for small areas; see this post for more info. This process doesn’t correct for any such deviation.
  • Imagery in Google Earth is copyrighted, and using it for any other purpose than viewing in Google Earth falls under the restrictions of copyright law and fair use. In particular, you have to leave the Google logo and copyright notices unaltered to maintain proper attribution.

MapWindow GIS Version 4.4 Is Out

Just a brief note to say that the open source GIS MapWindow has reached version 4.4, and the new version is definitely a worthwhile upgrade if you already have the program. A whole bunch of shapefile editing capabilities have been added, there’s now an Undo function in the shapefile editor you can turn on, higher-resolution raster export, plus bug fixes and a few other features. I plan to do a video in the near future demonstrating these new capabilities, similar to the ones I’ve done before on MapWindow.

High-Resolution Sea Level Rise Effects In Google Earth

Last week, I posted on how to create an animated depiction of sea level rise due to a storm surge or global warming. Here’s a movie of the flooding for the Manhattan/NYC area:

Got a note from Frank Taylor over at the Google Earth Blog saying that it looked pretty interesting, and that he would have linked to it if it were in Google Earth. That got me to thinking about how you could show the effects of coastal sea level rise in Google Earth with high spatial resolution, both static and animated. I’ll start with the easy one: static.

Continue reading ‘High-Resolution Sea Level Rise Effects In Google Earth’