1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

New 30m GDEM

Discussion in 'Terrain Design General' started by Golf-HotelDelta, 30/6/09.

  1. Golf-HotelDelta

    Golf-HotelDelta

    Joined:
    20/12/04
    Messages:
    9,132
    Country:
    unitedkingdom
  2. Timmo

    Timmo

    Joined:
    10/1/07
    Messages:
    547
    Country:
    newzealand
    wow- Interesting indeed!

    Edit- Looks like demand is high- Im having trouble accessing the sites
    Last edited: 1/7/09
  3. rhumbaflappy

    rhumbaflappy Moderator Staff Member Resource contributor

    Joined:
    2/6/04
    Messages:
    2,349
    Country:
    us-wisconsin
    It will take about a month for the excitement to die down.

    Meanwhile we can use this thread to discuss information about the data.

    I know nothing about it right now. Is this raw data, or has this been adjusted to remove errors? What's the effects of buildings and vegetation? Has waterbody flattening been added? What's the binary format of the data... headers... ?

    I think I have read this is 1 arc second data. That would make it good for about LOD10.

    I think this data covers the entire planet, unlike the SRTM data that only covers to about 60 degrees north or south.

    From the above link:


    Dick
    Last edited: 1/7/09
  4. rhumbaflappy

    rhumbaflappy Moderator Staff Member Resource contributor

    Joined:
    2/6/04
    Messages:
    2,349
    Country:
    us-wisconsin
    Some more information:

    So... sealevel is forced to zero elevation and -9999 represents no data. The tiles are 1*x1*. They are geotiff signed 16-bit with no projection applied ( geographic ) and apparently no header, other than the geotiff format. Conversion to a BIL format should be trivial with a program such as Global Mapper. The FSX resample should be able to use data "as is", but conversion to BIL would be needed for FS9 and earlier resampling.

    I have also read that extensive cloud removal was processed automatically, whereby several passes of the same area yields a composite of the elevation.

    30 meter horizontal accuracy and 20 meter vertical accuracy. I have read that values have been "spot compared" to existing known values from SRTM and NED datasets, and real-world elevation readings. I saw a table where the NED data set for the US was considered superior to the ASTER, the SRTM about the same for 30 meter US data, and worldwide, the ASTER is considered far superior to the SRTM 3 arc second data.

    It seems some work was done to minimize the holes and spikes, by using a more dense data availability than SRTM, and using software to automatically process this data.

    I don't know what was used to set the sealevel as zero... perhaps the same old SWBD masks for SRTM data.

    Dick
  5. rhumbaflappy

    rhumbaflappy Moderator Staff Member Resource contributor

    Joined:
    2/6/04
    Messages:
    2,349
    Country:
    us-wisconsin
    Some more info:

    http://www.ersdac.or.jp/GDEM/E/4.html

    Although this data has been processed to remove clouds and setting the sealevel to zero, this is essentially raw data. Note it took the software a year to process the elevations to this stage!

    Dick
  6. mendiola_loyola

    mendiola_loyola Resource contributor

    Joined:
    18/10/06
    Messages:
    264
    Country:
    peru
    Will this data be available for download for free?

    Alfredo Mendiola Loyola
    Lima, Perú
  7. GaryGB

    GaryGB

    Joined:
    23/12/05
    Messages:
    1,388
    Country:
    us-illinois
  8. rhumbaflappy

    rhumbaflappy Moderator Staff Member Resource contributor

    Joined:
    2/6/04
    Messages:
    2,349
    Country:
    us-wisconsin
    This data is free... I'm actually downloading some now. I can confirm it is geotiff.

    http://www.gdem.aster.ersdac.or.jp/search.jsp

    There is a problem. The usage of the data is specified by the download agreement. You must indicate the usage of the data before downloading, and agree to credit Nasa and Japan's Ministry of Economy, Trade and Industry (METI).

    It's the usage that stumbles us. The catagories are narrowly defined and do not include flight simulation, hobby, non-profit data acquisition. So we'd have to indicate what area of research we would use for the data. "Water" is a good one... or climate or weather.

    Your conscience would have to guide you here. For personal use, anything would work... actually "Water" is pretty close to what I require in my own personal usage.

    If you would specify the BGLs derived are to be used for climatological research using FSX, then you should be legal, as this is an acceptable use. Going beyond freeware, using this purpose, might be difficult.

    Dick
  9. vlada stoje

    vlada stoje

    Joined:
    24/11/06
    Messages:
    30
    Country:
    czechrepublic
    I also used the "water" :)
  10. rhumbaflappy

    rhumbaflappy Moderator Staff Member Resource contributor

    Joined:
    2/6/04
    Messages:
    2,349
    Country:
    us-wisconsin
    One more comment.

    I downloaded and reviewed the area around Guadalcanal, in the Solomon Islands... an area I am familiar with using DEMs and Landsat imagery.

    The ASTER data is definitely research grade. Lots of oddities that look like cloud interference, and the determination of sea level is still a mystery, as it is NOT derived from the SWBD or Landsat imagery. In other words, the coastlines are poorly defined.

    We would have been better off without the sea level flattening, as we could have used the SWBD, which is much better, or made our own from LandSat7 images ( or even ASTER images ).

    But other than the occasional clouds not removed, and the strange sea level determination, the data is OK.

    [​IMG]

    [​IMG]

    Dick
  11. rhumbaflappy

    rhumbaflappy Moderator Staff Member Resource contributor

    Joined:
    2/6/04
    Messages:
    2,349
    Country:
    us-wisconsin
    Here's another review:

    http://www.viewfinderpanoramas.org/reviews.html#aster

    I agree with his perception of this data... it's very raw.

    I recently did an LOD7 world mesh for CFS2 using SRTM and dems from Viewfinderpanoramas ( with Jonathan de Ferranti's permission, of course ). A great source addition.

    Dick
  12. scott967

    scott967

    Joined:
    3/2/05
    Messages:
    1,266
    Country:
    us-hawaii
    Been trying to get data via the .jp site; so far, can't get past selecting tiles and attempting download (timing out). The usage rules seem very restrictive if you look at the NASA site, not so much so from that Japan site. My impression is that the limitation might be on redistribution of the data itself, rather than projects that result from the data though it's far from clear.

    scott s.
    .
  13. Golf-HotelDelta

    Golf-HotelDelta

    Joined:
    20/12/04
    Messages:
    9,132
    Country:
    unitedkingdom
    I have been attempting to get two tiles from .jp. The first downloaded fine, the second keeps timing out in Servlet/Download :(

    I am awaiting an email from NASA on tiles I ordered there.
  14. Timmo

    Timmo

    Joined:
    10/1/07
    Messages:
    547
    Country:
    newzealand
    I've managed to download all of New Zealand (half a GB of data) but haven't had a chance to have a look at it in depth yet.

    I did immediately see a few specs of 'no data' but the biggest test, in terms of comparision with the SRTM data will be in the Fjordland area which had large areas of errors caused by cloud and water. Hopefully the different remote sensing method will have given better information in this beautiful but tricky area (tricky from a remote sensing POV- Very steep terrain, often covered in cloud and lots of water)

    The one thing that did concern me was the second part of the data agreement (the first being me unable to really select a field of study from the list that was related to Flightsim/entertainment). This stated that the data wouldn't be shared outside the study group (or something similar)

    Could we argue that uploading to a site that requires a login constitutes being part of a study group?

    Edit- I downloaded from the NASA site, I couldnt even figure out how to actually get at the data from the Japanese site??
    Last edited: 2/7/09
  15. rhumbaflappy

    rhumbaflappy Moderator Staff Member Resource contributor

    Joined:
    2/6/04
    Messages:
    2,349
    Country:
    us-wisconsin
    Hi Timmo.

    And going beyond the limits for usage, if the ASTER is used to fill voids in SRTM data, then which dataset are we actually using, and does this still require ASTER restrictions, as the data has been significantly altered?

    SRTM is absolutely free of restrictions. ASTER apparently is not. And yet, the SRTM dataset may be reworked using ASTER data:

    http://pub7.bravenet.com/forum/static/show.php?usernum=537683448&frmid=8&msgid=720114&cmd=show

    Here are the restrictions:

    The above part is a request, although forced if you want to access the data. GEO wants to know what your general area of interest is. Maybe biodiversity is closest to our usage from the list.

    The 2nd part ( above ) is the actual restriction... You agree not to distribute the actual zip or it's contents ( the actual GDEM ) outside your organization or project group. My project group is myself.

    Note that we still can publish that data.. as in making a map or image ( or BGL ), as long as we include "ASTER GDEM is a product of METI and NASA."

    My take on this is we are free to make BGLs and publish them ( and distribute the BGLS ). We are not free to distribute the DEMs themselves. That makes sense as they wish to preserve the integrity of the GDems, and to stave off any criticism of the data.

    This is like what CGIAR has done with SRTM data... they have made their own version not sanctioned by NASA, and have therefore altered the available datasets. I don't think NASA is very happy about it, especially as the CGIAR data seems to have had a 1/2 pixel shift.

    A BGL is much different, as it is not a DEM and not usable for that purpose... it's only usable in our "flying game". It is no longer a Gdem or DEM or HGT.

    I'm no lawyer, but this seems to cover the bases for unrestricted usage for both freeware and payware. If I make BGLs from the data, I would state:

    Dick
    Last edited: 2/7/09
  16. Timmo

    Timmo

    Joined:
    10/1/07
    Messages:
    547
    Country:
    newzealand
    Right- Finally unzipped the data and started having a look at it.

    One thing that is perplexing me at the moment is a series of what look like craters all through the data? Only very small changes in elevation but they do appear under certain colour ramps and shading.

    Any ideas?

    They aren't caused by man made features (I thought maybe returns off roads/concrete could have been the cause) and don't appear to be natural either. It is a similar effect to when water is left on a photograph and it makes water spots.

    In the DEM they would be sinks, but the stream/waterway information I have and local knowledge shows them in very unlikely places from a hydrology point of view?

    [​IMG]

    I've also noticed some quite obvious seam effects
  17. Golf-HotelDelta

    Golf-HotelDelta

    Joined:
    20/12/04
    Messages:
    9,132
    Country:
    unitedkingdom
    Am I correct in thinking that the GeoTiffs are compatible with FSX Resample?

    It seems to have worked for the two files I downloaded from NASA

    [​IMG]
  18. rhumbaflappy

    rhumbaflappy Moderator Staff Member Resource contributor

    Joined:
    2/6/04
    Messages:
    2,349
    Country:
    us-wisconsin
    The geotiffs should be work with FSX resample, but the resulting mesh is likely to be of poor quality... due to the defects in the raw data you already noted.

    I seriously discourage anyone from making a distributed mesh from this data, unless you are willing to rework it up to the SRTM quality. The last thing the community needs is more poor quality mesh, when we already have decent datasets.

    Dick
  19. nickw

    nickw Administrator Staff Member

    Joined:
    28/5/04
    Messages:
    646
    Country:
    ireland
    Any of you fancy trying it on Ireland? ;)
  20. =Hollywood=

    =Hollywood=

    Joined:
    13/12/06
    Messages:
    458
    Country:
    unitedstates
    I selected biodiversity...

    Downloaded a large chunk of the Philippines and it is pretty raw... With the USA covered by both NED 10m and the USA and Europed covered by NEXTMap 5m data those regions really won't benefit from the ASTER data...

    For places like Australia however like the Glasshouse Mountains local areas with specific features will make some of the data a Godsend... Definitely not a good source for global mesh, and that has been the case with ASTER for a long time.

    The RMSE of ASTER is pretty comparible to SRTM, but SRTM also has many many voids such as in Africa... If there was a good way to automate a blended SRTM/ASTER mesh from the two datasets it'd be great...

    Definitely some spikes on the data as well in some locations...

Share This Page