blankblank blank


Archive for January, 2009 Page 2 of 3



Automating The Google Earth Caching Process

In a previous post, I described how to manually create a Google Earth disk cache file for an area. For small areas, this is probably the fastest approach, but for larger areas, doing it manually can get tedious. I’ve found three programs that can automate this process, but before reviewing them, it might be useful to go through common factors for all of them.

All three programs work by creating a KML file for a particular location, and then updating that file on user-specified schedule, typically every 5-10 seconds, with a new location. You need to create a Network Link in Google Earth to this KML file that updates on a regular basis as well; as the KML file is updated on a regular basis, the Network Link picks this up, and moves the Google Earth view to the new location, where imagery is downloaded and added to the cache.

To create a Network Link in Google Earth:

1. Open up Google Earth, and from the Add menu choose Network Link.

2. Click the Browse button, and select the KML file used by the cache automation programs (CashBild.kml in the example below):

Network link Google Earth

3. Select the “Periodically” option for the Time-Based Refresh, and then set the refresh time to 5-10 seconds (more on this shortly).

4. Check the “Fly to View on Refresh” box; this tells Google Earth to move to the updated location in the Network link file.

When the location is updated in the network link, Google Earth will fly to that location, download the imagery and display it. But if the speed with which it flies to the location is too slow, then the location may shift to a new one before all the data is downloaded. To make sure this doesn’t happen, go to Tools => Options, Touring tab, and set the “Fly-To Speed” to a higher value than the default of 0.180:

Google Earth fly to speed

Here I’ve set it midway, which should be fast enough for caching, but not too fast for regular fly-to operations. You can experiment with this setting to find the one that works best for you.

The caching programs I’ll be reviewing have the following options:

  • Set the bounds of the area you want to cache. Two of them require you to specify the NW corner of the area in latitude/longitude, and the east and south bounds of the area from that point in distance (meters or kilometers); the third requires the E/W longitude and N/S latitude bounds. Because of how they work, they don’t function directly over the poles, or across the 180W/180E longitude transition (essentially the International Date Line)
  • Set the spacing for successive points. This sets the distance that the view moves between one point and another. It would be convenient if the programs calculated the correct value for you, so that every bit of imagery in your designated area was covered, but they don’t. You have to set this manually, and the correct value will depend on the altitude the views are downloaded at (i.e. the image resolution).  Set it too small, and you won’t miss any imagery, but it will take longer; set it too large, and you might miss some areas. I’ve found that a reasonable rule-of-thumb is to set the spacing about 10-15% smaller than the altitude you set (see the next item), and then watch the first few Google Earth views to see if you’re getting enough overlap between successive views. If you’re getting too much overlap, increase the spacing; no overlap, decrease the spacing. You can also set the eye altitude you plan to use manually in Google Earth, and then use the Ruler tool to manually measure the field of view in both E/W and N/S. Take the smaller of the two values, decrease it by about 10-15%, and that should be a satisfactory spacing
  • Set the altitude. Two of the programs let you set the altitude/elevation over terrain (Eye alt – elev in Google Earth), while the third requires you to set that manually. The lower the altitude, the higher the resolution of the imagery downloaded to the cache, but the more time it will take and the larger the final cache size. For some areas with very high resolution imagery, the eye altitude will need to be no more than a few hundred meters to capture the highest resolution imagery. For other areas where only Landsat imagery is available, this could easily be set to a kilometer or two. If you want the cache to contain the imagery appropriate for varying heights, you’ll have to run these programs with different eye altitudes for the same area. For GE Cacher, you’ll need to
  • Set the update time.Two of the programs update the KML file with a user-defined time interval. The longer the time interval, the longer it will take, but too short an interval may not give Google Earth enough time to download all the data. The appropriate interval will depend on your Internet connection speed, and how fast the Google Earth servers can provide the data. Start with about 10 seconds, and then adjust it up or down based on whether all the data for a view is downloaded before it moves to a new view. Important: You should set the Network Link Time-Based Refresh interval to be no longer than the update time, otherwise you might miss some of the position updates. You might even try the shortest refresh interval available, one second, and see if that works for you; too often shouldn’t cause a problem, too seldom will. For the third program with a fixed update interval, GE Cacher, you should definitely set the Refresh to one second.

Once everything is set up, all the programs will have the option to start the caching, and manually pause or stop it before it’s completed if you want. As with the manual caching process, you might want to empty the current disk cache before starting, and set the memory cache as small as possible. When you’re done with a caching session, you should either uncheck or delete the Network Link to the KML file; otherwise, next time you run Google Earth, it will go to that file’s location/view and keep returning there regardless of where you want to see. If you want to save the cache data, copy the files from their location to another one for safekeeping.

Next up: reviews of the Google Earth cache automation programs.




Manually Caching Google Earth Data

The last post covered the basics of the Google Earth disk cache, and how it lets you use Google Earth even when you don’t have an Internet connection. Today I’ll talk about the simplest method to create a Google Earth disk cache file for an area – doing it manually

To create a localized Google Earth cache file manually:

1. Start up Google Earth while connected to the Internet

2. Go to Tools => Options, and select the Cache tab:

1-10-2009-1.09.26 PM

3. Click the “Clear disk cache” button to remove all data from the cache and cache index files. This maximizes the available amount of space for your cached data, so that you can cover more area with the cached data.

4. Set the Memory Cache Size to the minimum allowable value of 16MB; this will force Google Earth to put data in the disk cache more quickly. Click the “Clear memory cache” button, then exit the Options screen

5. In Google Earth, navigate to your area of interest.

6. Set your viewing altitude (the “Eye alt”) for the desired imagery resolution. Higher altitudes mean lower resolution, but the data will take up less space. Lower altitudes give you higher image resolution, but at the cost of more disk space. If there are major changes in elevation over the area, you may have to change your altitude to match them to keep your relative height; if the terrain elevation drops, but your “eye alt” stays the same, the image resolution will decrease because you are further away from the surface.

7. Wait for the “data loading” indicator at lower right to show that all the data for that area and resolution has been downloaded; it will display as a solid light-blue circle when downloading is complete for that area. You’ll also see the imagery change from blurry to sharp.

8. With the mouse or cursor keys, move the view to an adjacent area that overlaps the first area at least partially, and wait for the “data loading” indicator to show once again that all the data has been downloaded.

9. Repeat this process until you’ve covered your entire area of interest.

10. If want data at several different resolution levels, you could repeat steps 5-9, but at different viewing altitudes.

11. When you’re finished with steps 5-10, you might consider zooming all the way in to the closest level at one point, then all the way out to the planetary view level. It seems to me that this prompts Google Earth to write all of its data cached in RAM to the disk cache, but I could be wrong about this. It also seems as though there can be a time delay between viewing a location, and when it’s cached to disk, so waiting 10 minutes after you’ve finished covering the area of interest might be a good idea.

You now have a set of Google Earth data for that area saved in the disk cache. If you take your computer offline and start up Google Earth, it will tell you that it can’t log on to the server and download data, but it can still read in whatever data it has in the cache. You can copy this cached data (see this previous post for where to find it) into a different directory, and then copy it back into the cache directory when you need it, or even copy it to a different computer.

For small areas, manual caching of data won’t take too long, but for larger areas, manually scrolling to an area, waiting for the data to download, then scrolling to another area gets old very quickly. Fortunately, there are several free applications that can partially automate the process, and I’ll cover some of those in the next few posts.




Using The Google Earth Cache – Basics

When you view a location in Google Earth, it caches the imagery data (but not the elevation data), storing it locally so that as you scroll back and forth across different areas, it doesn’t have to download the same imagery over and over again from the server on the Internet. There are actually two sets of locally cached data:

Memory Cache: Data stored in system RAM. This information can be retrieved very quickly, but vanishes when you exit the application.

Disk Cache: Data stored on the hard drive, and which persists after program exit. So if you have data cached for a particular area in Google Earth, exit the program, and then return to the same area the next time you use Google Earth, GE will load the imagery information directly from the hard drive cache – a lot faster than having to download it again over the Internet.

The size of these caches can be set in the Tools => Options section, under the Cache tab:

Google Earth cache options

The maximum values are the ones above, 500MB for memory cache, and 2GB (2000MB) for disk cache; apparently, Google doesn’t want people to download the entire imagery dataset for the whole planet to their systems :). If you’ve got the disk space, and most modern systems do, you should definitely set the disk cache size at this maximum value. Optimum value for the memory cache will depend on how much RAM you have installed on your system, and how much your OS uses. If you have a system with a relatively small amount of RAM (512 MB to 1 GB), making this memory cache too large might impede performance. On most modern systems, with 2GB or more of RAM, the maximum memory cache size of 500MB shouldn’t cause any problems.

Disk cache data is stored by default in the C:\Users\your_user_name\AppData\Local\Google\GoogleEarth directory in Windows Vista.If you can’t find it there, you can determine the actual location using the registry entry HKEY-CURRENT-User/Software/Google /Google Earth Plus, under the CachePath entry. There are two files: dbCache.dat which contains the actual data, and dbCache.dat.index, which presumably is the index file for the data. You can clear out all the disk cache data using the “Clear disk cache” button, which removes all the data but keeps the files there. To delete the files completely, you need to either log out of the server (File => Server Log Out) and then click the “Delete cache file” button you see above, or exit Google Earth and delete those two files directly from the cache directory. You can also copy them to another directory to hang on to them.

Why would you want to delete or copy these files? Two reasons:

1. If you’re having crashing or graphics issues with Google Earth, Google recommends deleting the disk cache files as one possible fix.

2. You can “pre-cache” a set of data for a particular area of interest, then copy them to a different directory on your hard drive. Then, when you want to look at this area in detail in Google Earth, you can copy these files back to the original directory, and Google Earth will access this data from the hard drive, a lot faster than downloading it again from the Google Earth server. If you have a decent Internet connection all the time, this is likely to be more trouble than it’s worth. But if you have a slow connection, this can speed things up enormously. And if you’d like to use Google Earth on your laptop, in a location that has no Internet connection, using cached data is your only option; Google Earth can access and use cached data when no Internet connection is present or available. You can copy this cache data from one computer to another as well.

Note: This only caches image  terrain data from Google Earth, not data from KML/KMZ files. If the KML/KMZ files are self-contained, you can load them into Google Earth and view the data, but if they’re network links, and you’re not connected to the network, the data in the links won’t show up.

Coming up next: how to cache Google Earth data manually.




Using Apple And Online Tools With Google Earth

I haven’t posted much about Apple-related geography tools on this blog, for two very good reasons:

1. I don’t own any Apple hardware (Mac, iPhone or iTouch)

2. There’s a lot less geographically-oriented software available for the Mac than there is for Windows, or even Linux

But Stefan Geens, who runs the Ogle Earth blog, does have a Mac and and iPhone, and posts regularly on geographic applications for those platforms, usually Google Earth related. And he’s just put up several posts (one, two, three) on how he used these tools along with Google Earth to document a trip in Egypt to visit a number of archaeological sites. The series is very much worth a look, even if you don’t use Apple products:

- The trip itself, and the locations documented, are incredibly interesting, especially if you’re into Egyptian archaeology. In particular, the cubic panoramas embedded in the file, using 360cities.net, are jaw-droppingly spectacular. Stefan documents his panorama-creation process here.

- A number of the tools used are online-based, so they’re not tied to Apple platforms and can be used by anyone

- Valery Hronusov posted a few days ago on his blog about using the DropBox service (free for up to 2 GB of data) to sync a KMZ file on your desktop to your DropBox account, and then create a publicly-accessible URL link to this file that anyone can access. Stefan created such a link to his Egyptian travelogue, demonstrating how you can now have an Internet-accessible KMZ link that updates automatically whenever you change the matching file on your desktop; no FTP or other file upload process required. KMZ is a better choice than KML for most cases, since it compresses the file and also incorporates all of the supporting materials (like photographs).

- Stefan also uses the caching capability of Google Earth on the iPhone, storing GE data for the locations he was going to visit using a WiFi link for use when he couldn’t get a good connection, or when it would have been expensive to do so. This is a good technique to use with Windows laptops as well, if you’d like to use Google Earth in a location where an Internet connection is either unavailable or expensive. I’ve been meaning to do short series on this for a while, and this is as good a prompt as any, so the rest of this week’s posts will be on using Google Earth’s caching capability on a Windows system.




Another New MapChannels Feature: Geography Quiz Maps

In addition to the new TeamMaps feature, MapChannels has a new embeddable geography game called QuizMaps:

Choose World or State Capitals, and get a world/US map with letters plotted at various locations, as in the example screen capture below:

MapChannels geography quiz

Unscramble all the letters to get the correct answer; one of the letters is plotted at the same location as the answer. Panoramio photo clues from the answer location can show up on the left, but are usually as useless as the example you see above. The correct answer to the image above is left as an exercise for the reader.




Visitor-Editable Google “Team Maps” At Map Channels

The MapChannels website has a new feature called “Team Maps”, which lets you create an embeddable Google Maps view which visitors are free to add their own annotated markers to. From the website, features include:

  • (1) Create a new empty Team Map, (2) create a map from an existing KML or (3) create a map using a pre-defined event venue map data
  • Customizable data schema – allowing you to define a set of data fields for your place markers
  • Customizable info window and tooltip contents – use javascript template code to format your data schema
  • Pre-defined javascript template functions to display images, YouTube videos, website thumbnails and RSS feeds within info windows
  • Define the set of icons that can be used for map markers
  • Optionally select other Map Channels users to be an admin or editor for your map
  • Review user edits before applying any changes to your map
  • Lock the map to disable map editing by users
  • Automatically generate KML, GeoRSS and Tab-delimited text feeds for your map data
  • Import data from KML files (e.g. Google My Maps) or tab-delimited text (Spreadsheet) files
  • Embed your Team Map into a web page using a single line of HTML code. This code uses an IFrame, the URL within the IFrame can include parameters allowing you to display a specific place on the map (plus other options).

Here’s a simple active  map I created to test the service. The initial display bounds for the map are set automatically by the loaded data; for this map, I’ve added two placemarks already to set those bounds:

As a viewer of the map, you can submit a placemark for entry on this map using the “Add Place” link at the upper right; it lets you add not just a marker, but also a name, description, web link, and note for the map creator/editor:

1-7-2009-10.43.05 PM

You position the marker by dragging it on the map to the desired location. Once done, click on “Complete”, and your placemark is submitted to the map’s creator for review and approval before it shows up on the map. Feel free to add a placemark to the above map to try it out, but don’t look for it to show up – I won’t be reviewing/approving any submissions on this test map.

Current limitations (from the website):

  • Only markers (points) can be added to a Team Map. Lines and polygons are not yet supported
  • Team Maps are suitable for map collections of up to 500 markers, performance will be affected when you use a larger number of markers.
  • Please contact (MapChannels) if you are getting more than 5,000 page views per day for maps, this assists in planning server resources.



Online Tilt-Shift Photo Maker

There’s been a mini-craze online for creating tilt-shift photos, where the focus position and depth of field on a landscape/cityscape are manipulated to make the picture look like a miniature reproduction. For authentic tilt-shift photos, you need a special lens, but you can re-create the effect using Photoshop filters, or by following tutorials online. Now there’s an online website called TiltShiftmaker that lets you upload your own photos, and applies a tilt-shift effect to them:

Original photo from the island of Santorini (from Flickr):

1361244291_49b0773bc7_b

Run through TiltShiftmaker:

santorini

The process works best on images that don’t have a few objects much taller than the rest of the scene, as that can result in the base of the object being at a different focus/blur than the top.

Via Lifehacker.




Garmin Topo Maps For Afghanistan, Baghdad

Just noticed on the GPSFileDepot site that they’ve started posting 20-ft.-contour topographic maps for Afghanistan for Garmin GPS units, created by Steakhouse Studio. Here’s a MapSource screenshot of the Kandahar area:

Garmin topo map of Afghanistan

Twenty-five topo maps are currently available, covering one-by-one or one-by-two degree quadrangles; by my best estimate, this covers roughly a half of Aghanistan’s territory, but it looks like new maps are being added on a regular basis. The link page also include road maps for Afghanistan as a whole, and for metro Kabul.

Also at GPSFileDepot is a Garmin topo map for the Baghdad, Iraq area, with 20-ft. countours, roads and points of interest. Plus, there’s a permanent download page for the routable Garmin map of Baghdad mentioned in an earlier post.

Windows and Mac installers available for all these mapsets.

Addendum (5/22/2009): An upgraded set of these maps is available for US Government use only; contact the author for info.