[netCDFJava #PYP-475421]: Bug in NetCDF-Java 4.3 exposed by this GRIB2 file

> Lansing & John,
> 
> I can live with that, but it's unclear to me how to fix a collection of
> these in NcML, so if you have an example, that would be great.

One could argue this should be done by experts.
I think the right thing to do is to fix them on the TDS server.
There are several problems with CFSR that we are still working on.

> 
> Speaking of which, it would be very cool if there was a place (a wiki?)
> where the community could contribute annotated NcML that solves different
> problems.  On the IOOS Modeling Testbed, we have several complicated NcMLs
> that others have asked for as examples, and if there was a Unidata wiki
> that we could author, perhaps others would find them useful as well.

good idea.


> 
> -Rich
> 
> 
> address@hidden> wrote:
> 
> > Hi Rich:
> >
> > It is an error in the CFSR files (they have lots). We have a workaround in
> > NcML and if served from the TDS. Can you live with that?
> >
> > John
> >
> > > Lansing,
> > >
> > > I was expecting you to say that this issue was a bug in the GRIB
> > > reader, and it would get a ticket to be fixed.   But I think you are
> > > saying that the data was written wrong, and therefore the issue will
> > > not be fixed?
> > >
> > > This is one of those "big deal" datasets that it would be great to
> > > read correctly.
> > > http://rda.ucar.edu/pub/cfsr.html
> > >
> > > It's good to know that NcML can fix the issue for a file, but there
> > > are hundreds of these, so I would be better to fix the reader if you
> > > can.
> > >
> > > Thanks,
> > > -Rich
> > >
> > > On Tue, Jun 12, 2012 at 12:53 PM, Unidata netCDF Java Support
> > > <address@hidden> wrote:
> > > >> Lansing,
> > > >>
> > > >> It looks like you are using ToolsUI 4.2, not ToolsUI 4.3.   In toolsUI
> > > >> 4.2, you see one variable, but the times are wrong.   In 4.3, you see
> > > >> two variables with correct times, but we want one variable with
> > > >> correct time!
> > > >>
> > > >> Try this:
> > > >>
> > http://www.unidata.ucar.edu/software/netcdf-java/v4.3/webstart/netCDFtools.jnlp
> > > >>
> > > >> On Mon, Jun 11, 2012 at 1:10 PM, Unidata netCDF Java Support
> > > >> <address@hidden> wrote:
> > > >> >> NetCDF-Java folks:
> > > >> >>
> > > >> >> There appears to be a bug in the GRIB2 decoding problem using
> > > >> >> NetCDF-Java 4.3 on some of the surface pressure data we got from
> > > >> >> http://rda.ucar.edu/pub/cfsr.html
> > > >> >>
> > > >> >> This data is supposed to have a single variable (pressure) with
> > > >> >> uniform hourly sampling, but when we bring it up in NetCDF 4.3
> > tools,
> > > >> >> we see two variables instead of one, with the 1st having a uniform
> > > >> >> time base of 6 hours (0,6,12), and the 2nd having a non-uniform
> > time
> > > >> >> base that fills in the rest of the hourly data  (1,2,3,4,5
> > > >> >> 7,8,9,10,11  13,14...)
> > > >> >>
> > > >> >> Screenshot: http://screencast.com/t/DWEgKMfy
> > > >> >>
> > > >> >> I put a sample file here:
> > > >> >>
> > http://geoport.whoi.edu/thredds/catalog/usgs/data1/rsignell/test/catalog.html?dataset=usgs/data1/rsignell/test/pressfc.gdas.199801.grb2
> > > >> >>
> > > >> >> Thanks,
> > > >> >> Rich
> > > >> >> --
> > > >> >> Dr. Richard P. Signell   (508) 457-2229
> > > >> >> USGS, 384 Woods Hole Rd.
> > > >> >> Woods Hole, MA 02543-1598
> > > >> >>
> > > >> >>
> > > >> > Hello Rich,
> > > >> >
> > > >> > I'm not sure what's going on, but I tried your file and saw only
> > one variable reported.  I've attached a screenshot from NetCDF-Java
> > ToolsUI.  Maybe I'm missing something obvious here?
> > > >> >
> > > >> > Regards,
> > > >> >  Lansing Madry
> > > >> >  Unidata
> > > >> >  Boulder, Colorado
> > > >> >
> > > >> > Ticket Details
> > > >> > ===================
> > > >> > Ticket ID: PYP-475421
> > > >> > Department: Support netCDF Java
> > > >> > Priority: Normal
> > > >> > Status: Open
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Dr. Richard P. Signell   (508) 457-2229
> > > >> USGS, 384 Woods Hole Rd.
> > > >> Woods Hole, MA 02543-1598
> > > >>
> > > >>
> > > > Rich,
> > > >
> > > > Thanks for educating me on 4.2 vs. 4.3 -- details, details!
> > > >
> > > > I dug a little deeper to see what is going on with the data in the
> > file you sent.  It looks like, at the byte level, there are some
> > differences that 4.2 conveniently ignored and 4.3 notices.  Specifically,
> > if you look at the file in the iosp/GRIB2/GRIB2Collection tab, you'll
> > notice that two variables seem to show up, with separate GDS hash codes.
> >  In that view, if you look down to the Grib2GridDefinitionSection,
> > highlight a row, right-click on the row, and select "Show GDS", you'll see
> > the information for Lo2 differs for the two "different" variables:
> > > >
> > > > Lo2 - longitude of last grid point == 359687000
> > > > Lo2 - longitude of last grid point == 359688000
> > > >
> > > > This is really a defect in how the data was written out - perhaps a
> > rounding error somewhere, or some other loss of precision.
> > > >
> > > > In the meantime, you can "correct" the issue using a bit of NcML to
> > change the GDS hashcode.  Details are here:
> > > >
> > > >
> > http://www.unidata.ucar.edu/software/netcdf-java/formats/GribFiles.html
> > > >
> > > > and I've tried this bit of NcML and found it to work in this case:
> > > >
> > > > <?xml version="1.0" encoding="UTF-8"?>
> > > > <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";
> > location="C:/Users/madry/rsignell/pressfc.gdas.199801.grb2">
> > > >  <iospParam>
> > > >   <gdsHash from="1450192070" to="1450218978"/>
> > > >  </iospParam>
> > > > </netcdf>
> > > >
> > > > However, you will need to delete the associated .ncx file before
> > opening this NcML file in the toolsUI-4.3 viewer.
> > > >
> > > > Let me know if this doesn't work for you, and I'll take another crack
> > at it.
> > > >
> > > > Cheers,
> > > >  Lansing
> > > >
> > > > Ticket Details
> > > > ===================
> > > > Ticket ID: PYP-475421
> > > > Department: Support netCDF Java
> > > > Priority: Normal
> > > > Status: Open
> > > >
> > >
> > >
> > >
> > > --
> > > Dr. Richard P. Signell   (508) 457-2229
> > > USGS, 384 Woods Hole Rd.
> > > Woods Hole, MA 02543-1598
> > >
> > >
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: PYP-475421
> > Department: Support netCDF Java
> > Priority: Normal
> > Status: Open
> >
> >
> 
> 
> --
> Dr. Richard P. Signell   (508) 457-2229
> USGS, 384 Woods Hole Rd.
> Woods Hole, MA 02543-1598
> 
> 


Ticket Details
===================
Ticket ID: PYP-475421
Department: Support netCDF Java
Priority: Normal
Status: Closed