New 30m GDEM

[Golf-HotelDelta]

http://www.astronomy.com/asy/default.aspx?c=a&id=8414

[Timmo]

wow- Interesting indeed!

Edit- Looks like demand is high- Im having trouble accessing the sites

[rhumbaflappy]

It will take about a month for the excitement to die down.

Meanwhile we can use this thread to discuss information about the data.

I know nothing about it right now. Is this raw data, or has this been adjusted to remove errors? What's the effects of buildings and vegetation? Has waterbody flattening been added? What's the binary format of the data... headers... ?

I think I have read this is 1 arc second data. That would make it good for about LOD10.

I think this data covers the entire planet, unlike the SRTM data that only covers to about 60 degrees north or south.

From the above link:

The new ASTER data expands coverage to 99 percent, from 83° north latitude and 83° south. Each elevation measurement point in the new data is 98 feet (30 meters) apart.

The ASTER data fill in many of the voids in the shuttle mission's data, such as in very steep terrains and in some deserts," said Michael Kobrick, Shuttle Radar Topography Mission project scientist at JPL. "NASA is working to combine the ASTER data with that of the Shuttle Radar Topography Mission and other sources to produce an even better global topographic map."

Dick

[rhumbaflappy]

Some more information:

Information source:
G. Bryan Bailey, Co-lead CEOS LSI Constellation


Data Posting: 30 m
DEM Output Format: GeoTIFF, signed 16 bits, and 1 m/DN
Referenced to the WGS84/EGM96 geoid
Geographic latitude and longitude
Coverage: 83? N to 83? S
Special DN Values : -9999 for void pixels
0 for sea water body
Accuracies: 20 m with 95 % confidence for vertical
data (elevation)
30 m with 95 % confidence for
horizontal data (geolocation)
Data Set: 22,895 1? x 1º tiles

Methodology:
1. Stereo-correlate entire ~ 1.5 million scene ASTER Archive; 2. Cloud mask to remove cloudy pixels;
3. Stack all DEMS & remove residual bad values and outliers; 4. Partition data into 1º x 1º tiles ---


ASTER scene
(60km x 60km)

Tile (1 arc degree square)

Automated processing

Generation of a seamless tile of DEM using all ASTER data ever acquired over the tile

So... sealevel is forced to zero elevation and -9999 represents no data. The tiles are 1*x1*. They are geotiff signed 16-bit with no projection applied ( geographic ) and apparently no header, other than the geotiff format. Conversion to a BIL format should be trivial with a program such as Global Mapper. The FSX resample should be able to use data "as is", but conversion to BIL would be needed for FS9 and earlier resampling.

I have also read that extensive cloud removal was processed automatically, whereby several passes of the same area yields a composite of the elevation.

30 meter horizontal accuracy and 20 meter vertical accuracy. I have read that values have been "spot compared" to existing known values from SRTM and NED datasets, and real-world elevation readings. I saw a table where the NED data set for the US was considered superior to the ASTER, the SRTM about the same for 30 meter US data, and worldwide, the ASTER is considered far superior to the SRTM 3 arc second data.

It seems some work was done to minimize the holes and spikes, by using a more dense data availability than SRTM, and using software to automatically process this data.

I don't know what was used to set the sealevel as zero... perhaps the same old SWBD masks for SRTM data.

Dick

[rhumbaflappy]

Some more info:

http://www.ersdac.or.jp/GDEM/E/4.html

ASTER Global DEM

The ASTER instrument was built by METI and launched onboard NASA’s Terra spacecraft in December 1999. It has an along-track stereoscopic capability using its near infrared spectral band and its nadir-viewing and backward-viewing telescopes to acquire stereo image data with a base-to-height ratio of 0.6. The spatial resolution is 15 m in the horizontal plane. One nadir-looking ASTER VNIR scene consists of 4,100 samples by 4,200 lines, corresponding to about 60 km-by-60 km ground area.

The methodology used to produce the ASTER GDEM involved automated processing of the entire 1.5-million-scene ASTER archive, including stereo-correlation to produce 1,264,118 individual scene-based ASTER DEMs, cloud masking to remove cloudy pixels, stacking all cloud-screened DEMs, removing residual bad values and outliers, averaging selected data to create final pixel values, and then correcting residual anomalies before partitioning the data into 1°-by-1° tiles. It took approximately one year to complete production of the beta version of the ASTER GDEM using a fully automated approach.

The ASTER GDEM covers land surfaces between 83°N and 83°S and is composed of 22,600 1°-by-1° tiles. Tiles that contain at least 0.01% land area are included. The ASTER GDEM is in GeoTIFF format with geographic lat/long coordinates and a 1 arc-second (30 m) grid of elevation postings. It is referenced to the WGS84/EGM96 geoid. Pre-production estimated accuracies for this global product were 20 meters at 95 % confidence for vertical data and 30 meters at 95 % confidence for horizontal data.

Initial studies to validate and characterize the ASTER GDEM confirm that pre-production accuracy estimates are generally achieved for most of the global land surface, although results do vary and true accuracies do not meet pre-production estimates for some areas. In addition, Version 1 of the ASTER GDEM does contain certain residual anomalies and artifacts that affect the accuracy of the product and may be impediments to effective utilization for certain applications. Consequently, METI and NASA acknowledge that Version 1 of the ASTER GDEM should be viewed as “experimental” or “research grade.” Nevertheless, they are confident that the ASTER GDEM represents an important contribution to the global earth observation community.

ASTER GDEM tiles may be downloaded electronically from ERSDAC by visiting http://www.gdem.aster.ersdac.or.jp/and from the LP DAAC by visiting https://wist.echo.nasa.gov/~wist/api/imswelcome/.

Although this data has been processed to remove clouds and setting the sealevel to zero, this is essentially raw data. Note it took the software a year to process the elevations to this stage!

Dick

[mendiola_loyola]

Will this data be available for download for free?

Alfredo Mendiola Loyola
Lima, Perú

[GaryGB]

Hello:

See also the thread at:

http://forums1.avsim.net/index.php?showtopic=253387

GaryGB

[rhumbaflappy]

This data is free... I'm actually downloading some now. I can confirm it is geotiff.

http://www.gdem.aster.ersdac.or.jp/search.jsp

There is a problem. The usage of the data is specified by the download agreement. You must indicate the usage of the data before downloading, and agree to credit Nasa and Japan's Ministry of Economy, Trade and Industry (METI).

It's the usage that stumbles us. The catagories are narrowly defined and do not include flight simulation, hobby, non-profit data acquisition. So we'd have to indicate what area of research we would use for the data. "Water" is a good one... or climate or weather.

Your conscience would have to guide you here. For personal use, anything would work... actually "Water" is pretty close to what I require in my own personal usage.

If you would specify the BGLs derived are to be used for climatological research using FSX, then you should be legal, as this is an acceptable use. Going beyond freeware, using this purpose, might be difficult.

Dick

[vlada stoje]

I also used the "water"

[rhumbaflappy]

One more comment.

I downloaded and reviewed the area around Guadalcanal, in the Solomon Islands... an area I am familiar with using DEMs and Landsat imagery.

The ASTER data is definitely research grade. Lots of oddities that look like cloud interference, and the determination of sea level is still a mystery, as it is NOT derived from the SWBD or Landsat imagery. In other words, the coastlines are poorly defined.

We would have been better off without the sea level flattening, as we could have used the SWBD, which is much better, or made our own from LandSat7 images ( or even ASTER images ).

But other than the occasional clouds not removed, and the strange sea level determination, the data is OK.





Dick

[rhumbaflappy]

Here's another review:

http://www.viewfinderpanoramas.org/reviews.html#aster

I agree with his perception of this data... it's very raw.

I recently did an LOD7 world mesh for CFS2 using SRTM and dems from Viewfinderpanoramas ( with Jonathan de Ferranti's permission, of course ). A great source addition.

Dick

[scott967]

Been trying to get data via the .jp site; so far, can't get past selecting tiles and attempting download (timing out). The usage rules seem very restrictive if you look at the NASA site, not so much so from that Japan site. My impression is that the limitation might be on redistribution of the data itself, rather than projects that result from the data though it's far from clear.

scott s.
.

[Golf-HotelDelta]

I have been attempting to get two tiles from .jp. The first downloaded fine, the second keeps timing out in Servlet/Download

I am awaiting an email from NASA on tiles I ordered there.

[Timmo]

I've managed to download all of New Zealand (half a GB of data) but haven't had a chance to have a look at it in depth yet.

I did immediately see a few specs of 'no data' but the biggest test, in terms of comparision with the SRTM data will be in the Fjordland area which had large areas of errors caused by cloud and water. Hopefully the different remote sensing method will have given better information in this beautiful but tricky area (tricky from a remote sensing POV- Very steep terrain, often covered in cloud and lots of water)

The one thing that did concern me was the second part of the data agreement (the first being me unable to really select a field of study from the list that was related to Flightsim/entertainment). This stated that the data wouldn't be shared outside the study group (or something similar)

Could we argue that uploading to a site that requires a login constitutes being part of a study group?

Edit- I downloaded from the NASA site, I couldnt even figure out how to actually get at the data from the Japanese site??

[rhumbaflappy]

Hi Timmo.

And going beyond the limits for usage, if the ASTER is used to fill voids in SRTM data, then which dataset are we actually using, and does this still require ASTER restrictions, as the data has been significantly altered?

SRTM is absolutely free of restrictions. ASTER apparently is not. And yet, the SRTM dataset may be reworked using ASTER data:

http://pub7.bravenet.com/forum/stati...20114&cmd=show

There are 2 official versions of SRTM data. V1 is the original DEM and v2 was improved by removal of small voids and imposition of shorelines. Much information is available on the USGS web pages and ftp site. CGIAR (see previous posts) has produced a void-filled version some folks are calling v3, but that is not an official NASA or USGS product. We're currently working on filling voids in SRTM data with ASTER-derived and other topographic data. Perhaps in a year or two we'll be able to release a better void-filled product.
-Tom Farr

Here are the restrictions:

The following selection is requested to understand how data are utlized in the context of societal benefit areas defined by the Group on Earth Observation (GEO).

The above part is a request, although forced if you want to access the data. GEO wants to know what your general area of interest is. Maybe biodiversity is closest to our usage from the list.

ASTER GDEM Policy Agreements
I agree to redistribute the ASTER GDEM *only* to individuals within my organization or project of intended use or in response to disasters in support of the GEO Disaster Theme. (Required)
When presenting or publishing ASTER GDEM data, I agree to include "ASTER GDEM is a product of METI and NASA." (Required)
Apply these order options (if applicable) to all granules currently in my shopping cart for data set:
ASTER Global Digital Elevation Model V001.

The 2nd part ( above ) is the actual restriction... You agree not to distribute the actual zip or it's contents ( the actual GDEM ) outside your organization or project group. My project group is myself.

Note that we still can publish that data.. as in making a map or image ( or BGL ), as long as we include "ASTER GDEM is a product of METI and NASA."

My take on this is we are free to make BGLs and publish them ( and distribute the BGLS ). We are not free to distribute the DEMs themselves. That makes sense as they wish to preserve the integrity of the GDems, and to stave off any criticism of the data.

This is like what CGIAR has done with SRTM data... they have made their own version not sanctioned by NASA, and have therefore altered the available datasets. I don't think NASA is very happy about it, especially as the CGIAR data seems to have had a 1/2 pixel shift.

A BGL is much different, as it is not a DEM and not usable for that purpose... it's only usable in our "flying game". It is no longer a Gdem or DEM or HGT.

I'm no lawyer, but this seems to cover the bases for unrestricted usage for both freeware and payware. If I make BGLs from the data, I would state:

This elevational BGL is derived from SRTM and ASTER GDem. ASTER GDEM is a product of METI and NASA.

Dick

[Timmo]

Right- Finally unzipped the data and started having a look at it.

One thing that is perplexing me at the moment is a series of what look like craters all through the data? Only very small changes in elevation but they do appear under certain colour ramps and shading.

Any ideas?

They aren't caused by man made features (I thought maybe returns off roads/concrete could have been the cause) and don't appear to be natural either. It is a similar effect to when water is left on a photograph and it makes water spots.

In the DEM they would be sinks, but the stream/waterway information I have and local knowledge shows them in very unlikely places from a hydrology point of view?



I've also noticed some quite obvious seam effects

[Golf-HotelDelta]

Am I correct in thinking that the GeoTiffs are compatible with FSX Resample?

It seems to have worked for the two files I downloaded from NASA

[rhumbaflappy]

The geotiffs should be work with FSX resample, but the resulting mesh is likely to be of poor quality... due to the defects in the raw data you already noted.

I seriously discourage anyone from making a distributed mesh from this data, unless you are willing to rework it up to the SRTM quality. The last thing the community needs is more poor quality mesh, when we already have decent datasets.

Dick

[nickw]

Any of you fancy trying it on Ireland?

[=Hollywood=]

I selected biodiversity...

Downloaded a large chunk of the Philippines and it is pretty raw... With the USA covered by both NED 10m and the USA and Europed covered by NEXTMap 5m data those regions really won't benefit from the ASTER data...

For places like Australia however like the Glasshouse Mountains local areas with specific features will make some of the data a Godsend... Definitely not a good source for global mesh, and that has been the case with ASTER for a long time.

The RMSE of ASTER is pretty comparible to SRTM, but SRTM also has many many voids such as in Africa... If there was a good way to automate a blended SRTM/ASTER mesh from the two datasets it'd be great...

Definitely some spikes on the data as well in some locations...