[netCDFJava #XIR-445342]: Problem with netCDF-Java CoordinateAxis returning wrong values

> Hi John....
> 
> Thanks for the explanation.  I understand things better now, and I
> conclude that the "only" issue remaining is that the IDV does not, for
> the "NN" file, create whatever is required so that the main 2D display
> can auto-set the projection to cover just the region enclosed by the
> data.    It also does not display correctly in the "world" projection
> (see attached, using the nightly build).  I can remap this into a
> South Pole conformal and it looks great, though...

We will have to look at the IDV code, probably cant get to it until after the 
workshop.

Long term, im starting to develop a new feature type class for satellite data, 
which could have different behavior than the grids. So if you have ideas how 
things could be handled better, id be interested.

> 
> McV has another (possibly related) issue in that the 2D display used
> for the scatter plots does not understand longitudes < -180
> apparently.  We will have to deal with that one, I guess.

If you pass the lon value through a normalizer like LatLonPointImpl.lonNormal() 
you can adjust the range. If you dont need connectedness, maybe that is 
sufficent.



John


> 
> 
> 
> On Mon, Jul 15, 2013 at 5:08 PM, Unidata netCDF Java Support
> <address@hidden> wrote:
> > hi Tom:
> >
> > The issue is a bit subtle. Its not a matter of putting the longitudes into 
> > some standard range of [x, x+360]. Whats going on is that the CDM is 
> > creating "projection coordinates" (not very well named) which are an 
> > intermediate coordinate. These are required to be continuous and invertible 
> > (no grid lines can crosss), therefore they cannot cross any discontinuous 
> > seam. we do this by examining the longitudes and adding 360 if there is a 
> > discontinuity. Changing the longitudes in this way has been happening since 
> > version 4.1 i think; in 4.3 I added some new code to do a better job of it 
> > in the 2D case, and ive been using this rather extreme example to improve 
> > that algorithm.
> >
> > The IDV can get the original coordinates if it wants. It also can recover 
> > them by passing the values it gets from CooordainetAxis2D through a [x, 
> > x+360] normalizer. Im guessing that the seam is being handled in visad or 
> > something like that. but it may not be getting it right.
> >
> > Im not really sure what the IDV is doing, My guess is that it is already 
> > renormalizing longitudes, because there appears to be no difference between 
> > CDM 4.2 and 4.3 in the IDV. If you look at the IDV rendering using the 
> > "world projection (?)" it appears incorrect to me (sean can you send a 
> > screen capture?), compared to what ToolsUI GridRenderer does (attached). 
> > The South Pole projection looks very nice, OTOH.
> >
> > John
> >
> >
> >> So then the IDV reading is handling the files differently?  It should
> >> be consistent.  Is that possible?
> >>
> >> I also observed that this is a change from the previous version of the
> >> IDV where the longitudes in the file were not changed, and I am
> >> concerned a little about that.
> >>
> >> We will also have to explore the possibilities of making the VisAD
> >> library understand +/- 360 instead of just 0:360 and -180:+180....
> >>
> >> tom
> >>
> >> On Mon, Jul 15, 2013 at 11:29 AM, Unidata netCDF Java Support
> >> <address@hidden> wrote:
> >> > Hi all:
> >> >
> >> > it looks to me the CDM handles the 2 identically. The difference may be 
> >> > that the NN lons happen to span >= 360 (?)
> >> >
> >> > John
> >> >
> >> >> Good Monday, Sean...
> >> >>
> >> >> I have tested the two files on the IDV nightly from today, and as I
> >> >> suspected the behavior is different:
> >> >>
> >> >> When I display the data from the "NSS.GHRR.NF... ", a "projection" is
> >> >> created and the "auto-set projection" works as expected.
> >> >>
> >> >> When I display the data from the "NSS.GHRR.NN..." file, no
> >> >> "projection" is created.
> >> >>
> >> >> In both cases, I note that the Field Selector "Region" tab does show
> >> >> the proper outline of the domain.
> >> >>
> >> >> Finally, as far as I can tell, the two files have identical
> >> >> structures, just the size and coverage are different.
> >> >>
> >> >> The effect of this is that within McV, the display used for the
> >> >> Scatter Plot does not show the plan view of the data for the "NN"
> >> >> file.  It does show the data for the "NF" file, thanks to the first
> >> >> round of changes John made.
> >> >>
> >> >> Both files are still on our ftp site
> >> >> (ftp://ftp.ssec.wisc.edu/pub/ssec/tomw/) but if you can't get them (I
> >> >> think they "expire" today or tomorrow), just let me know.
> >> >>
> >> >> Thanks...from hot and sunny MSN...
> >> >>
> >> >> tom
> >> >>
> >> >>
> >> >> On Thu, Jul 11, 2013 at 10:46 PM, Tom Whittaker <address@hidden> wrote:
> >> >> > Hi Sean....
> >> >> >
> >> >> > Thanks for checking this out.   I won't be back in the office until
> >> >> > Monday and I will write you back then.  I want to reconfirm my
> >> >> > observations between the file you've been testing with and the
> >> >> > previous one (the original one that we had problems with).    I
> >> >> > appreciate you checking into this....I will be in touch.
> >> >> >
> >> >> > tom
> >> >> >
> >> >> >
> >> >> > On Thu, Jul 11, 2013 at 3:15 PM, Unidata netCDF Java Support
> >> >> > <address@hidden> wrote:
> >> >> >> Hi Tom,
> >> >> >>
> >> >> >> I've loaded the file into the IDV using 3.1u1, 4.0u1, and the latest 
> >> >> >> version
> >> >> >> from master, and I can't see a difference between any of them. If I 
> >> >> >> view the
> >> >> >> file using the South Pole Projection, then it looks great. The 
> >> >> >> DataProbe also
> >> >> >> seems to be working the same in all three as well.
> >> >> >>
> >> >> >> You mentioned in another email that the IDV isn't able to set a 
> >> >> >> projection
> >> >> >> from the display. The reason is that there isn't any projection 
> >> >> >> information
> >> >> >> in the file, likely because lat / lon are the only coordinate 
> >> >> >> variables in
> >> >> >> the file.
> >> >> >>
> >> >> >> I guess overall I am confused as to what the issue is. I mean, I see 
> >> >> >> that the
> >> >> >> lat / lon returned from the CoordinateAxis is different from those 
> >> >> >> in the file,
> >> >> >> but as John said, these are "enhanced" since you are going through
> >> >> >> NetcdfDataset. At any rate, I can't tell if the IDV is having any 
> >> >> >> issues with
> >> >> >> this, given that everything seems to behave the same, from a IDV
> >> >> >> users perspective, between 3.1u1, 4.0u1, and master.
> >> >> >>
> >> >> >> Could you give us more info on the application in which the values 
> >> >> >> returned
> >> >> >> from CoordinateAxis is messing things up?
> >> >> >>
> >> >> >> Thanks!
> >> >> >>
> >> >> >> Sean
> >> >> >>
> >> >> >>> Hi John....
> >> >> >>>
> >> >> >>> I've started to evaluate the changes....and unfortunately, there are
> >> >> >>> some ripple effects -- some things just are not working.
> >> >> >>>
> >> >> >>> In thinking about this, shouldn't it be the responsibility of the
> >> >> >>> drawing routine to sort out the "seams"?  I am concerned (more so
> >> >> >>> now...) that the netCDF library is changing values handed back to 
> >> >> >>> the
> >> >> >>> caller when none of the metadata says do that.  My gut is telling me
> >> >> >>> that is not the "right" thing to do...the values in the file (as
> >> >> >>> modified only by missing code, the
> >> >> >>> offset and the scaling, if applicable) should be returned to the
> >> >> >>> caller, not something that "magically" is done.
> >> >> >>>
> >> >> >>> While it might be a nice "service" to provide for some applications,
> >> >> >>> perhaps the proper way is to introduce some attribute that says
> >> >> >>> "please_fix_my_seams"; otherwise leave the values sacrosanct....
> >> >> >>>
> >> >> >>> What do other people think?
> >> >> >>>
> >> >> >>> tom
> >> >> >>>
> >> >> >>> On Wed, Jul 3, 2013 at 3:30 PM, John Caron <address@hidden> wrote:
> >> >> >>>
> >> >> >>> > what we are doing is converting from lat/lon with valid range +- 
> >> >> >>> > 180 to
> >> >> >>> > coordinate values which "removes the seam" from the coordinates 
> >> >> >>> > (by adding
> >> >> >>> > +- 360) , so that the drawing routines dont have to worry about 
> >> >> >>> > the
> >> >> >>> > coordinates jumping. so the valid_range of the original coords 
> >> >> >>> > doesnt matter
> >> >> >>> > here.
> >> >> >>> >
> >> >> >>> > in fact the problem is that the longitude on row 6614 jumps 110 
> >> >> >>> > degrees,
> >> >> >>> > when i had a max jump of 100 degrees. i have adjusted the 
> >> >> >>> > algorithm again,
> >> >> >>> > now that row has (just the last 40 values):
> >> >> >>>
> >> >> >>> --
> >> >> >>> Tom Whittaker
> >> >> >>> University of Wisconsin-Madison
> >> >> >>> Space Science & Engineering Center (SSEC)
> >> >> >>> Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> >> >> >>> 1225 W. Dayton Street
> >> >> >>> Madison, WI  53706  USA
> >> >> >>> ph: +1 608 262 2759
> >> >> >>>
> >> >> >>>
> >> >> >>
> >> >> >>
> >> >> >> Ticket Details
> >> >> >> ===================
> >> >> >> Ticket ID: XIR-445342
> >> >> >> Department: Support netCDF Java
> >> >> >> Priority: Urgent
> >> >> >> Status: Open
> >> >> >>
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Tom Whittaker
> >> >> > University of Wisconsin-Madison
> >> >> > Space Science & Engineering Center (SSEC)
> >> >> > Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> >> >> > 1225 W. Dayton Street
> >> >> > Madison, WI  53706  USA
> >> >> > ph: +1 608 262 2759
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Tom Whittaker
> >> >> University of Wisconsin-Madison
> >> >> Space Science & Engineering Center (SSEC)
> >> >> Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> >> >> 1225 W. Dayton Street
> >> >> Madison, WI  53706  USA
> >> >> ph: +1 608 262 2759
> >> >>
> >> >>
> >> >
> >> >
> >> > Ticket Details
> >> > ===================
> >> > Ticket ID: XIR-445342
> >> > Department: Support netCDF Java
> >> > Priority: Urgent
> >> > Status: Open
> >> >
> >>
> >>
> >>
> >> --
> >> Tom Whittaker
> >> University of Wisconsin-Madison
> >> Space Science & Engineering Center (SSEC)
> >> Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> >> 1225 W. Dayton Street
> >> Madison, WI  53706  USA
> >> ph: +1 608 262 2759
> >>
> >>
> >
> > Ticket Details
> > ===================
> > Ticket ID: XIR-445342
> > Department: Support netCDF Java
> > Priority: Urgent
> > Status: Open
> 
> 
> 
> --
> Tom Whittaker
> University of Wisconsin-Madison
> Space Science & Engineering Center (SSEC)
> Cooperative Institute for Meteorological Satellite Studies (CIMSS)
> 1225 W. Dayton Street
> Madison, WI  53706  USA
> ph: +1 608 262 2759
> 
> 

Ticket Details
===================
Ticket ID: XIR-445342
Department: Support netCDF Java
Priority: Urgent
Status: Open