[Support #JXE-761208]: IDV with NetCDF including land (and IDV 2.7beta2)

Hi Nikitas-

> Thanks, we will try reinstalling 2.7 tomorrow and see.

The fix should be in today's build. 
 
> With regard to your question about the grid, our grid is curvilinear, and
> curvilinear grids can be twisted in every direction. Lat-longs for the land
> cells of twisted curvilinear grids can only be dummy values if anything.

Still not sure I understand this, but okay. ;-)

> Yet, what if we cheated? Can we just assign lat=1, long=1, and depth=1 (with
> the same sigma transform) to the land cells, and NaN for the mapped quantity
> values at these land cells? Would that work in IDV? Would IDV render a field
> with holes (NANs in the values of the 3D field, not the location)?

Then it wouldn't truly be a grid since the values would not be
monotonically increasing/decreasing.

I have an idea about a possible solution to this, but I won't
be able to implement/test it for a couple of weeks.  We have
a workshop all next week and are trying to get the 2.7 release
finished up.

Don Murray
> -----Original Message-----
> From: Unidata IDV Support [mailto:address@hidden]
> Sent: Wednesday, June 03, 2009 1:13 PM
> To: address@hidden
> Cc: address@hidden; address@hidden;
> address@hidden
> Subject: [Support #JXE-761208]: IDV with NetCDF including land (and IDV
> 2.7beta2)
> 
> Hi Liang-
> 
> > Hi,DonI uploaded two files,one is NYHOPS_noland.cdf, the other is
> NYHOPSFrom
> > Table.cdf. With IDV 2.7,both can't be read in. But with IDV 2.6, the
> volume
> > rendering is working perfect for the first cdf file,that's the file with
> no
> > land  included. The second cdf file can't do the volume rendering work
> with
> > IDV 2.6, in which land is included.
> >
> > For IDV2.7,the error information is: couldn't get data 2 for the first cdf
> > file.
> > The error information is:Couldn't get data
> > Gridded3DSet: samples values may not be missing. For the second cdf file.
> 
> There was a bug in the netCDF-Java stack that caused the problems with these
> files for 2.7.  We actually came across that yesterday with another file and
> the fix was checked into svn yesterday.  I'm not sure when that will
> make it into our nightly build.
> 
> The 3D rendering with the missing lat/lons is tricky.  The netCDF library
> replaces the xpos,ypos with the lat/lon values at each point.  For us
> to handle the sigma coordinates, we make a mapping from lat,lon,sigma <->
> lat,lon,altitude.  We do this so people can slice on sigma levels.
> However, we can't do this when the lat/lons are missing because the
> transform is not invertible.   I'm still unclear why you have missing
> lat/lons at any x,y if the grid can be mapped onto the earth.
> 
> I don't really have a solution to that at this point, but will keep
> pondering it.
> 
> Don
> 
> Don Murray
> > address@hidden> wrote:
> >
> > > Liang,
> > > Please answer Don's questions below for me, with some detail, and upload
> on
> > > his site:
> > > 1) an example that does not work with 2.7 but works with 2.6, and
> > > 2) the original (table.txt) version of the grid that does not render due
> to
> > > land.
> > > Please CC me in the response.
> > > -Nickitas
> > >
> > > -----Original Message-----
> > > From: Unidata IDV Support [mailto:address@hidden]
> > > Sent: Tuesday, June 02, 2009 4:57 PM
> > > To: address@hidden
> > > Cc: address@hidden
> > > Subject: [Support #JXE-761208]: IDV with NetCDF including land (and IDV
> > > 2.7beta2)
> > >
> > >
> > > Hi Nickitas-
> > >
> > > > We took efforts to make our NetCDF output for our NY/NJ Harbor ocean
> > > model
> > > > CF-compliant, and now IDV renders it in 3D very nicely. We've seen
> > > probes,
> > > > transects, volume rendering. Very cool!
> > >
> > > Great!
> > >
> > > > BUT,
> > > >
> > > >
> > > >
> > > > Only if we create sub-arrays that do not include land (NaN lat-lon and
> > > out
> > > > of range packed values). This is a coastal ocean model, and as such,
> land
> > > is
> > > > part of the full output array. Because the grid is not a nice
> rectangle
> > > but
> > > > quite distorted in the X-Y, lat-long coordinate values on land would
> be
> > > > nonsensical. But if we try to load the full output NetCDF CF-compliant
> > > file
> > > > with the native land/ocean array, IDV does not render in 3D.
> > >
> > > Is there a particular error that is generated?
> > >
> > > My guess is
> > > > that IDV is trying to render the land parts of the array that are NaN
> and
> > > > fails,  because it does not know how to distinguish between contiguous
> > > areas
> > > > of NaNs (such as land) versus internal NaN such as bad data, and does
> not
> > > > know what to do with data gaps in general.
> > > >
> > > >
> > > >
> > > > Any clues? Have you come up with any smart ways of dealing with this
> > > > problem? By the way, IDV 2.7 beta 2 does not work at all for us, only
> > > 2.6,
> > > > which is interesting in its own.
> > >
> > > We definitely want to fix what's broke in 2.7b2, so could you point
> > > me to a sample data file or upload one to:
> > >
> > > http://motherlode.ucar.edu/repository/alias/idvupload
> > >
> > > Once we get that fixed, I can look further into the other issue.  Having
> > > NaN's for lat/lon is tricky because the netCDF layer translates that
> into
> > > a lat/lon projection.  If there is a dimensionless vertical coordinate,
> > > that complicates it further since we have to have a transformation
> between
> > > lat,lon,sigma <-> lat,lon,alt for the 3D displays.
> > >
> > > Don Murray
> > >
> > > Ticket Details
> > > ===================
> > > Ticket ID: JXE-761208
> > > Department: Support IDV
> > > Priority: Normal
> > > Status: Open
> > >
> > >
> > >
> >
> >
> 
> 
> Ticket Details
> ===================
> Ticket ID: JXE-761208
> Department: Support IDV
> Priority: Normal
> Status: Open
> 
> 
> No virus found in this incoming message.
> Checked by AVG - www.avg.com
> Version: 8.5.339 / Virus Database: 270.12.50/2150 - Release Date: 06/03/09
> 05:53:00
> 
> 


Ticket Details
===================
Ticket ID: JXE-761208
Department: Support IDV
Priority: Normal
Status: Open