[IDV #BER-255195]: IDV - IDV and subsetting Grids from OpenDAP/THREDDS

Valentijn-

> >This will be fixed in tomorrow's build (9/7), however the scaling issue
> still remains.  I hope to have that resolved next week, but upgrading to
> the next version of netCDF >is problematic because some of the API's
> changed.
> 
> Thanks for your reply, and for fixing the navigation issue!
> 
> While the "double scaling" issue is being looked at, we'll use the
> following formula to temporaly bypass the problem:
> 
> newValue = 13.333333333333333333333333333333 * orgValue - 3 # scale
> values back one (0.075 -> 40/3 = 13.3333) and apply the offset (-3
> degrees)

The double scaling is fixed in the latest nightly build.  The add_offset
issue still exists.  Perhaps Ethan has given you some ideas on how to
use NcML to handle this.

Don Murray
 
> Cheers, Valentijn
> -----Original Message-----
> From: Unidata IDV Support [mailto:address@hidden]
> Sent: Thursday, September 07, 2006 12:11 AM
> To: Valentijn Venus
> Cc: address@hidden
> Subject: [IDV #BER-255195]: IDV - IDV and subsetting Grids from
> OpenDAP/THREDDS
> 
> Hi Valentijn-
> 
> > Institution: ITC
> > Package Version: 2.1b1
> > Operating System: os.name:Windows XP; os.arch:x86; os.version:5.1;
> > Hardware Information: java.vendor:Sun Microsystems Inc.;
> > java.version:1.6.0-beta2; java.home:D:\\Program Files\\Java\\jre1.6.0;
> 
> > j3d.version:1.3.2 fcs (build12); j3d.vendor:Sun Microsystems, Inc.;
> > j3d.renderer:OpenGL;
> > Inquiry:
> > Dear support,
> >
> > I'm running the "nightly" buid of the IDV and If you select "I'm still
> feeling lucky" in Data Chooser / Catalog, when you are about to load the
> data, you will have the "Omni Control" as the only available option for
> displaying the data.
> > However, if you select "Grids from an OPeNDAP server" instead, you
> will get the advanced from, where you are able to select (among others)
> the field needed, the spatial subset and the aggregation, which is
> exactly what you want.
> 
> I've been playing with this with some of the recent inquiries you sent
> in.  There are some bugs in the netCDF Java library and IDV that we've
> been working on to make this work better.  However, as you are finding,
> not all OPeNDAP datasets are created equal and some work and some don't.
> Also, for those that use the netCDF layer, you don't need the .dods
> extension.
> 
> > I found that this works for the Pathfinder data. An example URL is:
> > http://data.nodc.noaa.gov/cgi-bin/nph-dods/pathfinder/Version5.0/8day/
> > 2001/2001001-2001008.s0481pfv50-sst-16b.hdf
> 
> This is one of the datasets that has uncovered some problems.  First, as
> you found, the navigation is off because the netCDF layer was not giving
> the correct units for latitude and longitude.  Secondly, there is a
> scaling bug in the library which causes the values to be scaled twice.
> This is fixed in the next version of the netCDF library.  Unfortunately,
> we can't use that yet. Lastly, there is an "add_off" attribute for these
> values which does not get used because the standard attribute name for
> netCDF is "add_offset".
> 
> > It does not work for the other OPeNDAP datasets e.g. TOMS/EP:
> > http://reason.gsfc.nasa.gov/opendap-bin/nph-dods/FTP_DATA/Giovanni/OPS
> > /TOMS/EP/TOMS-EP_L3-TOMSEPL3_2005m0409_v8.HDF
> 
> I'll pass this along to the netCDF folks.  At the very least, it should
> fail gracefully instead of throwing a NullPointerException.  I think the
> problem is that the x and y dimension variables cannot be forced to be
> latitude and longitude, i.e., how is one to know that
> XDim%3aTOMS%20Level%203 is Longitude and  YDim%3aTOMS%20Level%203 is
> latitude based on the metadata in the dataset?
> 
> > So the last case i have to "hardcode" the geographic/parameter
> subsetting and choose the lucky load option. The url the looks as
> follows:
> > http://reason.gsfc.nasa.gov/opendap-bin/nph-dods/FTP_DATA/Giovanni/OPS
> > /TOMS/EP/TOMS-EP_L3-TOMSEPL3_2005m0409_v8.HDF.dods?Erythemal[0:1:179][
> > 0:1:287]
> 
> The netCDF layer cannot handle the constraint expressions.
> 
> > My main conclusion from this experiment has been that the so-called
> right-part of the URL (everything after the ?), should be generated by
> IDV. IDV has such provision, as long as you tell it that it is dealing
> with an OPeNDAP (DODS) server. The problem sometimes is that we have
> encountered an OPeNDAP server that is not compatible with IDV (like
> TOMS/EP). IDV seems to expect some parameters into the .das or .dds
> responses of DODS servers, which can not be found.
> 
> Correct.  Is there some documentation somewhere on the conventions used
> for these datasets that would allow a mapping of variables like the x
> and y to latitude and longitude?
> 
> >
> > Question 1:
> > In such a case, can ncml "enhance" the way our THREDDS server
> interpretes the HDF/netCDF DODS server (e.g by reinterpreting the
> variables/units or by adding metadata) necessary for IDV to show the
> spatial subsetting GUI?
> 
> I'll defer that to the netCDF group.
> 
> > Question 2:
> > Hopefully this is a bug, but spatial subsetting of the Pathfinder
> Gridded SST seems to confuse the IDV  (see the attached bundle). Maybe a
> projection/navigation issue?
> 
> This will be fixed in tomorrow's build (9/7), however the scaling issue
> still remains.  I hope to have that resolved next week, but upgrading to
> the next version of netCDF is problematic because some of the API's
> changed.
> 
> Don
> 
> Ticket Details
> ===================
> Ticket ID: BER-255195
> Department: Support IDV
> Priority: Normal
> Status: Open
> 
> 
> 


Ticket Details
===================
Ticket ID: BER-255195
Department: Support IDV
Priority: High
Status: Closed