[IDV #JXK-429403]: In a netCDF file, the 2D field plots correctly, but the 3D field may not plot correctly dependent on sampling mode. [1465]

No problem! A little tricky to find, but it's all good (and fun!) :-)

Sean

> Hi Sean -
> 
> I passed your response off to the user and she said that this was her
> problem.  She says thanks for your help.
> 
> Thanks!
> Bob
> 
> On 5/10/2013 2:50 PM, Unidata IDV Support wrote:
> > Ah, yes. 2D coordinate variables are supported, but not time dependent 
> > coordinate
> > variables. My mistake. However, now I have really found the issue.
> >
> > When trying to use a cdl file to create mulitdimensional arrays, you have 
> > to use
> > row major ordering - this user used column major ordering (i.e. data must 
> > be listed in
> > the order it is going to go into memory).
> >
> > For example, the TAir variable has dimensions (t, p, lat, lon) [1,2,5,10]. 
> > So, when writing out the data in
> > ascii, it needs to go like this:
> >
> > row 1: value(time 1, pres 1, lat 1, lon 1), value(time 1, pres 1, lat 1, 
> > lon2), ..., value(time 1, pres 1, lat 1, lon 10)
> > row 2: value(time 1, pres 1, lat 2, lon 1), value(time 1, pres 1, lat 2, 
> > lon2), ..., value(time 1, pres 1, lat 2, lon 10)
> > ....
> > ....
> > row 5: value(time 1, pres 1, lat5, lon 1), value(time 1, pres 1, lat5, 
> > lon2), ..., value(time 1, pres 1, lat 5, lon 10)
> >
> > then, on row 6, we jump to the next pressure value:
> > row 6: value(time 1, pres 2, lat1, lon 1), value(time 1, pres 2, lat1, 
> > lon2), ..., value(time 1, pres 2, lat 1, lon 10)
> >
> > Same goes for the lat / lon variables and their data values.
> >
> > Confusing? Yep, but that's how C rolls.
> >
> > Cheers,
> >
> > Sean
> >
> >> Hi Sean -
> >>
> >> I passed this off to our user, and she still believes that something
> >> isn't working correctly with her original file.  I passed off your
> >> corrected CDL file along with your response, and here is what she had to
> >> say:
> >>
> >>> That is disturbing.  I have seen a few examples of hdf and netCDF
> >>> files at the center with 2D lat/lon variables.   The values for mask
> >>> and TAir did not have to be the same.  The point was that for TAir, I
> >>> was not seeing the values I expected to see.  When I adjust the
> >>> lat/lon variables to 1D as Sean instructed, I do see the values I
> >>> expect to see.
> >>> I think there is still a problem...
> >>>
> >>> According to the CF1.6 section 5.2, there can be 2D lat/lon coordinate
> >>> variables...
> >>>
> >>> http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.6/cf-conventions.html#idp5553648
> >>>
> >>> Joleen
> >>>
> >> - Bob
> >>
> >> On 5/10/2013 11:48 AM, Unidata IDV Support wrote:
> >>> No problem :-)
> >>>
> >>> Sean
> >>>
> >>>> Hi Sean -
> >>>>
> >>>> Thanks!  The user wasn't 100% sure the CDL file was correct... using
> >>>> your fixed CDL file, everything is working fine in McIDAS-V with both
> >>>> sampling modes.
> >>>>
> >>>> Thanks again!
> >>>> Bob
> >>>>
> >>>> On 5/10/2013 11:02 AM, Unidata IDV Support wrote:
> >>>>> Hi Bob,
> >>>>>
> >>>>> It looks like there were a few issues int he CDL file that were 
> >>>>> complicating
> >>>>> things. For example, the values used in the variable mask were not 
> >>>>> exactly
> >>>>> the same as the values used on the 300 hPa level of TAir. A bigger issue
> >>>>> is that the lat and lon variables, which were intended to be coordinate
> >>>>> variables, were 2D, and from what I can tell regarding CF, that is not
> >>>>> allowed.
> >>>>>
> >>>>> I adjusted the CDL file, and things seem to be working correctly with 
> >>>>> the
> >>>>> new netCDF file that is generated. Please take a look and let me know if
> >>>>> everything seems ok on your end.
> >>>>>
> >>>>> Cheers!
> >>>>>
> >>>>> Sean
> >>>>>
> >>>>>> Hi Sean -
> >>>>>>
> >>>>>> Thanks for looking into this one.  I've attached the original file the
> >>>>>> user provided (test.cdl) and I just used ncgen to create a netCDF file
> >>>>>> from it.
> >>>>>>
> >>>>>> Thanks -
> >>>>>> Bob
> >>>>>>
> >>>>>> On 5/10/2013 9:14 AM, Unidata IDV Support wrote:
> >>>>>>> Greetings Bob,
> >>>>>>>
> >>>>>>> Would you mind sending the cdl file? I'll generate the netCDF file 
> >>>>>>> locally and
> >>>>>>> try to see what is going on in the 3D displays.
> >>>>>>>
> >>>>>>> Cheers!
> >>>>>>>
> >>>>>>> Sean
> >>>>>>>
> >>>>>>>> Hello -
> >>>>>>>>
> >>>>>>>> We have a user who has a netCDF file whose 3D data may not plot 
> >>>>>>>> correctly based on the sampling mode.  It appears that Weighted 
> >>>>>>>> Average works better than Nearest Neighbor, but both have their 
> >>>>>>>> bugs.  If you want a sample netCDF file as well as the *.cdl file 
> >>>>>>>> that created it, please let us know.  Any ideas of what may be going 
> >>>>>>>> on?
> >>>>>>>>
> >>>>>>>> Thanks!
> >>>>>>>> Bob Carp
> >>>>>>>>
> >>>>>>>> ----==== Inquiry ====----
> >>>>>>>> 1465
> >>>>>>>>
> >>>>>>>> ----==== Summary ====----
> >>>>>>>> In a netCDF file, the 2D field plots correctly, but the 3D field may 
> >>>>>>>> not plot correctly dependent on sampling mode.
> >>>>>>>>
> >>>>>>>> ----==== Request ====----
> >>>>>>>> 2013-05-07 - Bob Carp
> >>>>>>>> Joleen has a netCDF file that she created from a *.cdl file using 
> >>>>>>>> ncgen.  This file contains 2 arbitrary pressure levels, and random 
> >>>>>>>> temperature values that plot out at different lat/lon points (almost 
> >>>>>>>> in the structure of a 10 x 5 grid).  After adding this file, you can 
> >>>>>>>> display a Value Plot display of the 2D field fine with Weighted 
> >>>>>>>> Average or Nearest Neighbor sampling modes.  The problem comes when 
> >>>>>>>> you try displaying the 3D data:
> >>>>>>>>
> >>>>>>>> - When using Weighted Average, select the 3D field, a pressure 
> >>>>>>>> level, the Value Plot display, and Create Display.  The data values 
> >>>>>>>> displayed do not match what is in the *.cdl file.  Some lines of 
> >>>>>>>> numbers are repeated, and some values don't plot at all.  In the 
> >>>>>>>> Layer Controls (Displays tab), if you change the level, and then 
> >>>>>>>> change back to your original level, the data displays much better 
> >>>>>>>> (matches up with what is in the *.cdl file with the exception of a 
> >>>>>>>> little bit of interpolation).
> >>>>>>>>
> >>>>>>>> - When using Nearest Neighbor, select the 3D field, a pressure 
> >>>>>>>> level, the Value Plot display, and Create Display.  The data values 
> >>>>>>>> displayed do not match what is in the *.cdl file, and messing around 
> >>>>>>>> in the Layer Controls (Displays tab) does not correct the values.
> >>>>>>>>
> >>>>>>>> This was tested with both McV and IDV nightlies dated 05/07.
> >>>>>>>>
> >>>>>>>> 2013-05-06 - Bob Carp
> >>>>>>>> Joleen's initial email to the help desk:
> >>>>>>>> ----------------------------------------
> >>>>>>>> Bob was looking at this with me.  When I display 2D values, the data 
> >>>>>>>> displays monotonically, the same as it is written in the file.
> >>>>>>>>
> >>>>>>>> However, the 3D grids seem to use the wrong neighbor when displaying 
> >>>>>>>> (nearest neighbor is default for my setup).
> >>>>>>>>
> >>>>>>>> Check the input data object with
> >>>>>>>> a=selectData()
> >>>>>>>> print a
> >>>>>>>>
> >>>>>>>> 2013-05-06 - Bob Carp
> >>>>>>>> I added test.cdl and test.nc to /home/mcuser/inquiry-data/1456
> >>>>>>>>
> >>>>>>>> 2013-05-06 - Bob Carp
> >>>>>>>> Joleen has a netCDF file that has 2D and 3D data in it.  The 2D data 
> >>>>>>>> displays fine with the Value Plots display type.  When you display 
> >>>>>>>> the 3D data with the Value Plots display, not all of the data is 
> >>>>>>>> displayed, and some of it is displayed twice.  Doing a grid table 
> >>>>>>>> display of the data works fine (data does not repeat, and all data 
> >>>>>>>> is listed.  Here is how the 3D data looks in the .cdl file and in 
> >>>>>>>> the Main Display window.
> >>>>>>>>
> >>>>>>>> In the .cdl file     In the Main Display window
> >>>>>>>> 0 1 2 3 4            0 1 2 3 4
> >>>>>>>> 10 11 12 13 14       20 21 22 24 25
> >>>>>>>> 20 21 22 24 25       30 31 32 33 34
> >>>>>>>> 30 31 32 33 34       30 31 32 33 40
> >>>>>>>> 35 36 37 38 39       40 41 42 43 44
> >>>>>>>> 40 41 42 43 44       40 41 42 43 44
> >>>>>>>> 45 46 47 48 49       50 51 52 53 54
> >>>>>>>> 50 51 52 53 54       50 51 52 53 54
> >>>>>>>> 60 61 62 63 64       70 71 72 73 74
> >>>>>>>> 70 71 72 73 74       70 71 72 73 74
> >>>>>>>>
> >>>>>>>> This has been tested in the 1.3b1 nightly from 05/06, 1.2, 1.01, and 
> >>>>>>>> 1.0.  I also tested this with a recent IDV 4.0u2 nightly.  This was 
> >>>>>>>> tested with both Weighted Average and Nearest Neighbor sampling 
> >>>>>>>> modes.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> ################################################################################
> >>>>>>>>
> >>>>>>>> http://dcdbs.ssec.wisc.edu/inquiry-v/index.php?inquiry=1465
> >>>>>>>>
> >>>>>>>>
> >>>>>>> Ticket Details
> >>>>>>> ===================
> >>>>>>> Ticket ID: JXK-429403
> >>>>>>> Department: Support IDV
> >>>>>>> Priority: Normal
> >>>>>>> Status: Open
> >>>>>>>
> >>>>> Ticket Details
> >>>>> ===================
> >>>>> Ticket ID: JXK-429403
> >>>>> Department: Support IDV
> >>>>> Priority: Normal
> >>>>> Status: Open
> >>> Ticket Details
> >>> ===================
> >>> Ticket ID: JXK-429403
> >>> Department: Support IDV
> >>> Priority: Normal
> >>> Status: Closed
> >>>
> >>
> >>
> > Ticket Details
> > ===================
> > Ticket ID: JXK-429403
> > Department: Support IDV
> > Priority: Normal
> > Status: Open
> >
> 
> 


Ticket Details
===================
Ticket ID: JXK-429403
Department: Support IDV
Priority: Normal
Status: Closed