[Support #JXE-761208]: IDV with NetCDF including land (and IDV 2.7beta2)

Hi Nickitas-

> This will be great Don. If it's ok with you, I will share our conversation
> with our ocean colleagues. We are all looking forward to have an IDV version
> that can treat (neglect) land in 3d visuals.

You might want to wait until I figure out if this works or not. ;-)

If these are truly unsrutctured grids, I think the ultimate approach
would be to define a CF convention for unstructured grids and work
toward implementing that.  I know Rich Signell has been involved in
some of that work.

Don
> -----Original Message-----
> From: Unidata IDV Support [mailto:address@hidden]
> Sent: Friday, June 05, 2009 3:43 PM
> To: address@hidden
> Cc: address@hidden; address@hidden;
> address@hidden
> Subject: [Support #JXE-761208]: IDV with NetCDF including land (and IDV
> 2.7beta2)
> 
> Hi Nikitas-
> 
> > Thanks, we will try reinstalling 2.7 tomorrow and see.
> 
> The fix should be in today's build.
> 
> > With regard to your question about the grid, our grid is curvilinear, and
> > curvilinear grids can be twisted in every direction. Lat-longs for the
> land
> > cells of twisted curvilinear grids can only be dummy values if anything.
> 
> Still not sure I understand this, but okay. ;-)
> 
> > Yet, what if we cheated? Can we just assign lat=1, long=1, and depth=1
> (with
> > the same sigma transform) to the land cells, and NaN for the mapped
> quantity
> > values at these land cells? Would that work in IDV? Would IDV render a
> field
> > with holes (NANs in the values of the 3D field, not the location)?
> 
> Then it wouldn't truly be a grid since the values would not be
> monotonically increasing/decreasing.
> 
> I have an idea about a possible solution to this, but I won't
> be able to implement/test it for a couple of weeks.  We have
> a workshop all next week and are trying to get the 2.7 release
> finished up.
> 
> Don Murray
> > -----Original Message-----
> > From: Unidata IDV Support [mailto:address@hidden]
> > Sent: Wednesday, June 03, 2009 1:13 PM
> > To: address@hidden
> > Cc: address@hidden; address@hidden;
> > address@hidden
> > Subject: [Support #JXE-761208]: IDV with NetCDF including land (and IDV
> > 2.7beta2)
> >
> > Hi Liang-
> >
> > > Hi,DonI uploaded two files,one is NYHOPS_noland.cdf, the other is
> > NYHOPSFrom
> > > Table.cdf. With IDV 2.7,both can't be read in. But with IDV 2.6, the
> > volume
> > > rendering is working perfect for the first cdf file,that's the file with
> > no
> > > land  included. The second cdf file can't do the volume rendering work
> > with
> > > IDV 2.6, in which land is included.
> > >
> > > For IDV2.7,the error information is: couldn't get data 2 for the first
> cdf
> > > file.
> > > The error information is:Couldn't get data
> > > Gridded3DSet: samples values may not be missing. For the second cdf
> file.
> >
> > There was a bug in the netCDF-Java stack that caused the problems with
> these
> > files for 2.7.  We actually came across that yesterday with another file
> and
> > the fix was checked into svn yesterday.  I'm not sure when that will
> > make it into our nightly build.
> >
> > The 3D rendering with the missing lat/lons is tricky.  The netCDF library
> > replaces the xpos,ypos with the lat/lon values at each point.  For us
> > to handle the sigma coordinates, we make a mapping from lat,lon,sigma <->
> > lat,lon,altitude.  We do this so people can slice on sigma levels.
> > However, we can't do this when the lat/lons are missing because the
> > transform is not invertible.   I'm still unclear why you have missing
> > lat/lons at any x,y if the grid can be mapped onto the earth.
> >
> > I don't really have a solution to that at this point, but will keep
> > pondering it.
> >
> > Don
> >
> > Don Murray
> > > address@hidden> wrote:
> > >
> > > > Liang,
> > > > Please answer Don's questions below for me, with some detail, and
> upload
> > on
> > > > his site:
> > > > 1) an example that does not work with 2.7 but works with 2.6, and
> > > > 2) the original (table.txt) version of the grid that does not render
> due
> > to
> > > > land.
> > > > Please CC me in the response.
> > > > -Nickitas
> > > >
> > > > -----Original Message-----
> > > > From: Unidata IDV Support [mailto:address@hidden]
> > > > Sent: Tuesday, June 02, 2009 4:57 PM
> > > > To: address@hidden
> > > > Cc: address@hidden
> > > > Subject: [Support #JXE-761208]: IDV with NetCDF including land (and
> IDV
> > > > 2.7beta2)
> > > >
> > > >
> > > > Hi Nickitas-
> > > >
> > > > > We took efforts to make our NetCDF output for our NY/NJ Harbor ocean
> > > > model
> > > > > CF-compliant, and now IDV renders it in 3D very nicely. We've seen
> > > > probes,
> > > > > transects, volume rendering. Very cool!
> > > >
> > > > Great!
> > > >
> > > > > BUT,
> > > > >
> > > > >
> > > > >
> > > > > Only if we create sub-arrays that do not include land (NaN lat-lon
> and
> > > > out
> > > > > of range packed values). This is a coastal ocean model, and as such,
> > land
> > > > is
> > > > > part of the full output array. Because the grid is not a nice
> > rectangle
> > > > but
> > > > > quite distorted in the X-Y, lat-long coordinate values on land would
> > be
> > > > > nonsensical. But if we try to load the full output NetCDF
> CF-compliant
> > > > file
> > > > > with the native land/ocean array, IDV does not render in 3D.
> > > >
> > > > Is there a particular error that is generated?
> > > >
> > > > My guess is
> > > > > that IDV is trying to render the land parts of the array that are
> NaN
> > and
> > > > > fails,  because it does not know how to distinguish between
> contiguous
> > > > areas
> > > > > of NaNs (such as land) versus internal NaN such as bad data, and
> does
> > not
> > > > > know what to do with data gaps in general.
> > > > >
> > > > >
> > > > >
> > > > > Any clues? Have you come up with any smart ways of dealing with this
> > > > > problem? By the way, IDV 2.7 beta 2 does not work at all for us,
> only
> > > > 2.6,
> > > > > which is interesting in its own.
> > > >
> > > > We definitely want to fix what's broke in 2.7b2, so could you point
> > > > me to a sample data file or upload one to:
> > > >
> > > > http://motherlode.ucar.edu/repository/alias/idvupload
> > > >
> > > > Once we get that fixed, I can look further into the other issue.
> Having
> > > > NaN's for lat/lon is tricky because the netCDF layer translates that
> > into
> > > > a lat/lon projection.  If there is a dimensionless vertical
> coordinate,
> > > > that complicates it further since we have to have a transformation
> > between
> > > > lat,lon,sigma <-> lat,lon,alt for the 3D displays.
> > > >
> > > > Don Murray
> > > >
> > > > Ticket Details
> > > > ===================
> > > > Ticket ID: JXE-761208
> > > > Department: Support IDV
> > > > Priority: Normal
> > > > Status: Open
> > > >
> > > >
> > > >
> > >
> > >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: JXE-761208
> > Department: Support IDV
> > Priority: Normal
> > Status: Open
> >
> >
> > No virus found in this incoming message.
> > Checked by AVG - www.avg.com
> > Version: 8.5.339 / Virus Database: 270.12.50/2150 - Release Date: 06/03/09
> > 05:53:00
> >
> >
> 
> 
> Ticket Details
> ===================
> Ticket ID: JXE-761208
> Department: Support IDV
> Priority: Normal
> Status: Open
> 
> 
> No virus found in this incoming message.
> Checked by AVG - www.avg.com
> Version: 8.5.339 / Virus Database: 270.12.53/2156 - Release Date: 06/05/09
> 06:24:00
> 
> 


Ticket Details
===================
Ticket ID: JXE-761208
Department: Support IDV
Priority: Normal
Status: Open