[netCDFJava #SSN-631180]: Re: more grib issues

> On 05/31/2012 09:34 AM, Unidata netCDF Java Support wrote:
> >> I've been getting my feet wet with this stuff, and I'm finally coming
> >> back to this issue.  I've tried to throw away the 6-hour intervals using
> >> the method described in the documentation, but it doesn't seem to work
> >> for me.  Have you tested this on the RFC grib format? If so, could you
> >> possibly send a sample my way?  I'm really trying to tackle this mixed
> >> interval problem (by throwing away non 1-hour intervals) in a complete
> >> way, so any tips you have would be greatly appreciated.
> >>
> >> Also, we are currently downloading these grib files from the NPVU ftp
> >> server to build an updating archive.  Our worry however is that the NPVU
> >> losing funding will cause this server to be shut down.  Would it make
> >> sense for us to pursue getting hooked up to the IDD so we can have
> >> access to this (and possibly other datasets) in a more proper way?  We
> >> are not sure what the implications of hooking up to IDD streams is, but
> >> it seems like we should start considering this as an option.
> > motherlode catalogs are in github.
> > this is what i currently have for RFC:
> >
> >      <dataset name="NWS National Precipitation Verification Unit">
> >
> >        <metadata inherited="true">
> >          <dataFormat>GRIB-1</dataFormat>
> >          <documentation xlink:href="http://www.hpc.ncep.noaa.gov/npvu/";
> >                         xlink:title="National Precipitation Verification 
> > Unit Page"/>
> >          <documentation
> >              
> > xlink:href="http://origin.hpc.ncep.noaa.gov/npvu/confpres/rfcworkshop07/NPVUatRFC_Verification_Workshop_2007.ppt";
> >              xlink:title="NPVU Overview (ppt)"/>
> >          <creator>
> >            <name vocabulary="DIF">DOC/NOAA/NWS/NPVU</name>
> >            <contact url="http://www.ncep.noaa.gov/"; 
> > email="http://www.ncep.noaa.gov/mail_liaison.shtml"/>
> >          </creator>
> >        </metadata>
> >
> >        <featureCollection name="RFC" featureType="GRIB" harvest="true" 
> > path="grib/NPVU/RFC">
> >          <collection spec="/data/ldm/pub/native/grid/NPVU/RFC/**/.*grib1$"
> >                      dateFormatMark="yyyyMMdd#.grib1#"
> >                      olderThan="5 min"/>
> >          <tdm startup="true" rescan="0 0/15 * * * ? *" trigger="allow"/>
> >
> >          <gribConfig>
> >            <gdsName hash='-1960629519' groupName='KTUA:Arkansas-Red River 
> > RFC'/>
> >            <gdsName hash='-1819879011' groupName='KFWR:West Gulf RFC'/>
> >            <gdsName hash='-1571856555' groupName='KORN:Lower Mississippi 
> > RFC'/>
> >            <gdsName hash='-1491065322' groupName='KKRF:Missouri Basin RFC'/>
> >            <gdsName hash='-1017807718' groupName='TSJU:San Juan PR WFO'/>
> >            <gdsName hash='-1003775954' groupName='NCEP-QPE National 
> > Mosaic'/>
> >            <gdsName hash='-529497359' groupName='KRHA:Middle Atlantic RFC'/>
> >            <gdsName hash='289752153' groupName='KRSA:California-Nevada 
> > RFC-6hr'/>
> >            <gdsName hash='424971237' groupName='KRSA:California-Nevada 
> > RFC-1hr'/>
> >            <gdsName hash='511861653' groupName='KTIR:Ohio Basin RFC'/>
> >            <gdsName hash='880498701' groupName='KPTR:Northwest RFC'/>
> >            <gdsName hash='1123818409' groupName='KTAR:Northeast RFC'/>
> >            <gdsName hash='1174418106' groupName='KNES-National Satellite 
> > Analysis'/>
> >            <gdsName hash='1193085709' groupName='KMSR:North Central RFC'/>
> >            <gdsName hash='1464276934' groupName='KSTR:Colorado Basin RFC'/>
> >            <gdsName hash='1815048381' groupName='KALR:Southeast RFC'/>
> >          </gribConfig>
> >        </featureCollection>
> >
> >      </dataset>
> >
> > not currently throwing away the 6-hour intervals. which has that problem?
> I've pretty much copied your configuration on our side after I found it
> on github.  I'm not using the TDM (need to read up on that a bit more).
> What I want is for a center like the northeast, where there is only the
> mixed interval variable, to throw away the one hour stuff in order to
> have it look like consistent hourly data.  Where this is really useful
> for me is writing a long archive that should be consistent across time
> (same variable name, same time intervals).
> 
> So I'm using this ncml:
> 
> <?xml version="1.0" encoding="UTF-8"?>
> <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2";
> location="QPE.20110101.009.158">
> 
> <iospParam>
> <intvFilter intvLength="1">
> <variable id="9-158-2"/>
> </intvFilter>
> </iospParam>
> </netcdf>

1) its possible i dont have this working for GRIB1, let me check.

2)
look at:

http://www.unidata.ucar.edu/software/netcdf-java/formats/GribFiles.html#intvFilter

<iospParam>
   <intvFilter intvLength="3">
     <variable id="0-1-8"/>
     <variable id="0-1-10"/>
   </intvFilter>
 </iospParam>

Only include the 3 hour intervals for parameters 0-1-8 and 0-1-10, defining the 
parameter using discipline-category-number notation.

so

> <intvFilter intvLength="1">
> <variable id="9-158-2"/>
> </intvFilter>

would include, not exclude, length 1 intervals

3) i dont see a parameter 9-158-2 in ZETA_KTAR.*grib1$, send me a sample file

4) you probably have to delete the *.ncx files to make this work, once you 
change the filter.

> 
> I am interested in your opinion of connecting directly to the IDD, what
> would be involved, and whether it makes sense.

probably does, do you just want the RFC data?



Ticket Details
===================
Ticket ID: SSN-631180
Department: Support netCDF Java
Priority: High
Status: Open