Exploring the world of free tools for GIS, GPS, Google Earth, neogeography, and more.
New from Google, a Google Earth network link that shows near-real-time US flight positions, full 3D position:
Mouse-over a plane, and see the route it’s taken up to that point in blue:
Click on a plane, and get a pop-up with flight info, on-time status:
Click on “Download flight path”, and download a KML time animation of the plane’s flight to that point; click on the play button at upper right to start the animation.
Flight info is delayed roughly 15 minutes.
Got an email today from someone with a data collection/coordination/assembly problem. They have multiple people out in the field with Garmin GPS units recording data, and bringing it back to a central location for collation/combination when they’re done. Since Garmin GPS units have limited capabilities for data storage – coordinates, name, comment, date and time, elevation, and that’s pretty much it – additional data associated with a point has to be recorded by hand. Once back from the field, data has to be downloaded from individual units, associated with the additional attribute data from forms, all the data combined together into a single dataset, then converted into GIS-friendly format. And the process they had come up with wasn’t really working well for them.
I think this highlights some of the major limitations of classic stand-alone handheld GPS units. They’re really designed for us in personal data collection, not combined data collection; assembling data from multiple units can take a lot of work. Plus, their limited data collection capabilities require offloading data attribute acquisition to other formats (e.g. pencil and paper), adding the addition problem of associating that data with coordinates later on. There are lots of professional solutions for these problems, like Trimble or MobileMapper GPS units, Terrasync and ArcPad software, but these can be complicated and expensive. I challenge you to find anyone with enough patience to use Terrasync for a single day without swearing at least once; I know I can’t ;-).
For a few years, I’ve had CyberTracker on my list of potential topics to post on. CyberTracker is a terrific data acquisition and collation tool for field data with tons of great features, like custom data acquisition form design for easy data entry, moving maps, easy data collation and conversion to GIS-friendly format. Plus, the software is free, and the hardware is (relatively) affordable. But unfortunately, I think time and technology are passing Cybertracker by:
I’m really excited about the possibilities for crowdsourced geographic data collection and collation using portable devices running Android OS (yeah, iOS too). There are already several interesting apps for doing this, and I hope to cover some of those soon on my AndroGeoid website. And I suspect that the small number of apps that can currently do this will be quickly joined by far more apps, and far more capable apps, in the very near future. But that doesn’t help with my emailer’s current problem. I suspect there are many different ways you could do this, but here’s the first approach that came to me using all-free software and services.
1. Make sure that everyone on the project has a Google account (i.e. Gmail); completely free.
2. Have the project leader created a single main data spreadsheet on Google Docs, with all the desired data attributes (e.g. point name, coordinates, comments, additional data fields from the paper forms, etc.) and share a link to that spreadsheet with other project members so that they can edit it as well.
3. Project members can download data from their Garmin units using DNRGarmin, and then export the data from DNRGarmin in CSV format.
4. Load the data into the spreadsheet program of your choice, and add/edit data from data forms to make it conform to the data structure of the main Google Docs spreadsheet.
5. Copy the data cells in the spreadsheet program, and paste them into the main Google Docs spreadsheet. Note: Use Ctrl-C and Ctrl-V to copy and paste cells into Google Docs instead of using the Google Docs Edit menu to perform those operations, as the latter doesn’t seem to work for pasting data from different applications into Google Docs. You no longer have a single computer as a choke point for data entry; multiple people can add data to a spreadsheet at the same time, and Google Docs will coordinate data entry so that nothing is lost. And if you save the spreadsheet data from steps 3 and 4 as separate files, you’ll have backup copies of the original data as well.
For those who aren’t comfortable with working with spreadsheets, Google Docs lets you set up a “Form” to let anyone add data to a Google Docs spreadsheet directly; however, this increases the chances of coordinate data entry error.
6. Now that you have all the data centralized in Google Docs, you have lots of flexibility in how you can handle it:
Know an alternate approach? Have any additional ideas? I welcome your links and suggestions in the Comments section below.
Figures that the week after I posted about US National Weather Service live network KMZ links, Google added live weather data to Google Earth. Look in the Layers pane for the Weather folder, where you’ll find layers for Clouds, Conditions And Forecasts, and Radar. The Clouds layer is worldwide and appears to be fairly up-to-date, though there’s no time stamp to confirm this. Clicking on a “Conditions” icon brings up “current” conditions and the forecast” for the entire world (even including locations in Antarctica).
But the time stamps shows the current conditions are about 3-4 hours behind the actual time.
Last week, I compared the National Weather Service radar conditions using their KMZ overlay:
With Google’s radar data for the same area and time:
Fairly close, but it looked to me as though Google’s data was about 30-60 minutes behind. And the NWS data not only showed a better-defined range of precipitation intensity, but was available in animated format, so you could see where the rain was heading. Google Earth only has radar data for North American and Western Europe. So, it’s a decent start for weather in GE, but it needs some work; for the US, the National Weather Service KMZ overlays are a much better choice.
I’ve been using the website of my local National Doppler Radar Station for years now to monitor the animated radar precipitation loop, but I just noticed that there are a pair of links labeled KML and GIS Users. The KML link takes you to a page where you can select and generate a KML/KMZ link to live radar data from any single radar station, or from custom combinations of multiple radar stations. You can select from the data type you want:
and choose either animated loops or static images to display in Google Earth:
For animated loops, I’d recommend setting the animation speed slider in the middle of the range or higher, and checking the Loop Animation box:
And it usually takes several loops before all the image overlays fully load in for display.
The generated KMZ files are network links, updated every two minutes so that they’re always current. So if you download a KMZ link from this site, open it in Google Earth, then save it in My Places, you’ll always have it available for up-to-date radar views.
The GIS Users link takes you to a general info page, and then to a directory page where you can download static GIF images with current radar data. You’ll need to know the three-letter code designation for your desired radar station, but there doesn’t seem to be an easy way to look that up; go to the main page, click on the desired radar station, then hunt around on the page to find the 3-letter code. The GIF images have corresponding .gfw worldfiles for georeferencing (latitude/longitude, NAD83), so you can load them into your GIS viewer. The GIF background color is transparent, so if your GIS viewer supports it, you can display the data as an overlay on top of another map, like this one of one-hour-total precipitation on top of a topo map:
In the process of writing yesterday’s post on Garmin Basecamp, I found an annoying flaw on how Google Earth handles GPX files. In a recent GPS talk I gave, I surprised some people when I told them that Google Earth can open some GPS-related formats like GPX, LOC and others directly; you just need to select the type of files you want to open with the drop-down in the lower-right corner:
But if the waypoints in GPX files come time-stamped, either with the time you created them in the field or in a program, Google Earth assumes that you want to use this time-related data, and brings up a time slider in the upper-left-hand corner:
The first time you open the file, the time slider will run from start to finish, with waypoints popping up and disappearing as the time indicator hits their creation time. You only see all the waypoints when the time slider has run all the way through to the end. Very annoying.
But it gets worse. If you uncheck the GPS data box in the Places pane to hide the data, then check it again, you’ll see nothing at all except the time slider:
Hitting the play button on the time slider will make the waypoints appear and disappear quickly, and at the end the only waypoint visible will be the last one created; the time slider will look like this:
To see all the waypoints, you’ll need to move the “start-time-extent” slider all the way to the left:
Same behavior if you save the data permanently to “My Places”. This is pretty retarded behavior; I hope Google adds the option to turn off time-related data display when it’s not wanted. Until then, I banged together a simple Windows-only program called GPXTimeStripper that will remove all time-related data from a GPX file (KML files, too, although this may not work in every case). Download the file at this link; it’s a zipped stand-alone executable. Run the program (won’t win any interface awards):
Click on the main button, choose the GPX or KML file you want to process, and the program will remove the time data from the file, and save it with “_TS” appended to the filename. You’ll get a pop-up box with the name and location of the new file, which should always be the same location as the input file. If the pop-up becomes annoying (which it will), check the box in the lower-left-hand corner to turn it off. Help button takes you to this page; Exit does what you’d expect.
No real checks for overwriting older files, and may not work on every file correctly (report bugs). Use at your own risk. BTW, Google, would it kill you to have KML files created in Google Earth terminate in CR-LF instead of just LF? It would simplify the job of reading them in correctly.
KMLGeocode (available here under the Google Geocoder listing) takes an address file in Excel (or XML) format, and creates a Google Earth KML file that plots geocodable addresses in the correct position. Load an address XLS file into the program (like this sample data also available on the download page):
You have to specify the column names that contain the key address fields required for geocoding, as well as the output KML filename. Once done, create the KML output file and open it in Google Earth. Addresses that could be geocoded will be plotted with orange pushpins:
Clicking on pushpin will bring up all the data fields for a record in the spreadsheet, not just the specified ones. Addresses that couldn’t be geocoded are given yellow pushpin designations, and will either show up plotted in a general location, or not plotted at all in Google Earth (though they will be listed in the Place pane under the KML file listing:
To embed the actual geographic coordinates for each of these addresses, right-click on the KML file listing (people.kml in the above example), and save it as another KML file with a different name. You can now open this different KML file in the KML Geocode Report program:
Specify the name of a text output file; this output will be a spreadsheet-readable CSV file that contains the original spreadsheet data with the longitude and latitude of geocodable addresses appended at the end as X and Y coordinates:
ID,Name,Group,Street,Street2,City,State,Zip,Phone,Email,X,Y
11, Suzanne White, Office of Information Resources, 312 8th Ave North, Suite 1600, Nashville, TN, 37243, 615-253-4799, suzanne.white@state.tn.us,-86.784031,36.164133
12, Kurt Snider, US Fish and Wildlife Service, 446 Neal St, , Cookeville, TN, 38501, 931-528-6481, kurt_snider@fws.gov,-85.497903,36.138056
14, David Tirpak, Comptroller of the Treasury, 505 Deaderick St, Suite 1700, Nashville, TN, 37243-0277, 615-401-7820, david.tirpak@state.tn.us,-86.781603,36.165238
The “UnMatched” file is supposed to contain the non-geocodable addresses in KML format (at least, according to the PDF manual for these programs), but it didn’t show up for the sample data when I tried it. When you “Process” the files, in addition to the output file above, you’ll get a pop-up telling you the overall geocoding success percentage:
Note: When installing both programs, the installation directory is listed simply as “C:\Program Files\”; however, unless you specify the new name, it is installed in the directory “C:\Program Files\BRalston”, and “BRalston” is the folder in the Start Menu where you’ll find the program shortcut icons.
The GE Coords program simplifies the job of grabbing coordinates for a point in Google Earth, and putting them into your clipboard, from where you can paste them into the application of your choice. Not terribly complicated to use (and the website has more complete instructions if you need them):
1. Install the program (you’ll find it in the “Thots Utilities” folder)
2. Fire up Google Earth, press the yellow pushpin icon on the toolbar, and drag it to your desired location:
3. After closing the placemark properties box, right-click on the pushpin and select “Copy” from the drop-down; this will copy the full KML code for the placemark into the clipboard
4. Run the GE Coords program; it parses the KML code, extracts the position, and copies the coordinates into the clipboard. You’ll hear a beep when it’s done. DM is the default format:
N 35° 01.640 W 111° 01.364
But running the program without any placemark code in the clipboard brings up the option to “Setup” the desired default coordinate format:
I first posted about HeyWhatsThat 2+ years ago; its initial functions included:
Mike Kosowsky has been adding functions to HeyWhatsThat since then, and writes to announce two more:
1. A simulator of Friday’s annular solar eclipse:
2. A web app to let you find out what you’re looking at using your iPhone or Android; point your phone’s browser to m.heywt.com.