[THREDDS #GHR-456072]: nullPointerException in when accessing 4.3.15 GRIB featureCollection's

Hi Dan:

Can you zip up the entire content/thredds/logs files and send them to me?

thanks

John

> John,
> 
> I scraped this month of gfsanl data down to the skeleton
> 279 .grb files... nothing else in the directory tree.
> 
> Shut down tomcat.
> Scraped the cache/GribIndex directory clean.
> Scraped logs directory clean.
> restarted tomcat.
> Waited for grib indexing to finish
> (they were all written to the right place)
> Attempted to access.
> 
> Results were disappointing.
> 
> Upon server startup (and its associated index file creation pass)
> 9 java.lang.NullPointerException's
> Upon attempting to access the entry from a browser:
> (which fails with HTTP:404)
> 23 java.lang.NullPointerException
> 
> from command [ grep -i -n "nullPoint" ./* ]
> in logs directory : <tomcat>/content/thredds/logs
> 
> At this point, I'm finding it harder to believe it may be an obscurity
> in our system and/or file systems.
> 
> I attempted to look at an ncx file with ToolsUI but could not make
> heads or tails out of it.
> 
> ~~~~
> at 
> thredds.inventory.MFileCollectionManager$DateSorter.compare(MFileCollectionManager.java:457)
> at 
> thredds.inventory.MFileCollectionManager$DateSorter.compare(MFileCollectionManager.java:455)
> at java.util.TimSort.countRunAndMakeAscending(TimSort.java:324)
> 
> Are what most of the NPE's started with.   What are these
> methods trying to do?    ~ Might lead us in the right direction.
> 
> -Dan
> NCDC-NOMADS
> 
> 
> 
> 
> On Fri, Mar 8, 2013 at 12:24 PM, Unidata THREDDS Support
> <address@hidden> wrote:
> > Hi Dan:
> >
> > (sorry i missed this message from last week- was just thinking that i hadnt 
> > heard from you).
> >
> > for sure remove the spurious index files from the data directory. make sure 
> > that only the files you want are getting included.
> >
> > you can look at the index files (.ncx) in latest toolsui iosp/cdmIndex tab. 
> > far right info button shown what files are in it.
> >
> > John
> >
> >
> >> I have redone the lists and included the
> >> GribIndex directory 4.3.15 has created:
> >>
> >> http://nomads.ncdc.noaa.gov/data/gfsanl/gribindex.tar.gz
> >> http://nomads.ncdc.noaa.gov/data/gfsanl/findlist.txt.bz2
> >>
> >> The .grb2 are for GFS grid 4, which should not be included
> >> due to the collectionSpec.   I have changed this silghtly since
> >> sending you the configuration, due to Ethans suggestions...
> >> Homing into just one month makes no difference in the
> >> nullPointerException that gets tirggered.
> >>
> >> The index files resident in the file system itself was
> >> not created by this installation -- they ARE going into
> >> GribINdex.. but I'm just not sure if the presense of these
> >> old index files might be causing trouble in some way.  I've
> >> seen stranger things.
> >>
> >> We need to have index files in a separate directory, in some
> >> cases a read only file system is present due to security reasons
> >> and this is actually essential and I'm glad you have developed
> >> this option.
> >>
> >> -Dan
> >>
> >>
> >>
> >>
> >>
> >> On Mon, Feb 25, 2013 at 3:54 PM, Unidata THREDDS Support
> >> <address@hidden> wrote:
> >> > Hi Dan:
> >> >
> >> > 1) can you redo this with "ls -l" to get time, size and ownership?
> >> >    can restrict to *grb*
> >> >
> >> > 2) I see some .grb2 in some directories, not sure what they are:
> >> >
> >> > ./200711/20071130/gfsanl_3_20071130_1800_006.grb
> >> > ./200711/20071130/gfsanl_3_20071130_1800_006.grb.gbx9
> >> > ./200711/20071130/gfsanl_3_20071130_1800_006.grb.ncx
> >> > ./200711/20071130/gfsanl_3_20071130_1800_006.idx
> >> > ./200711/20071130/gfsanl_3_20071130_1800_006.inv
> >> > ./200711/20071130/gfsanl_4_20071130_1200_000.grb2
> >> > ./200711/20071130/gfsanl_4_20071130_1200_003.grb2
> >> > ./200711/20071130/gfsanl_4_20071130_1200_006.grb2
> >> > ./200711/20071130/gfsanl_4_20071130_1800_000.grb2
> >> > ./200711/20071130/gfsanl_4_20071130_1800_003.grb2
> >> > ./200711/20071130/gfsanl_4_20071130_1800_006.grb2
> >> >
> >> > 3) the gbx9 and ncx files are the cdm index files. But you have
> >> >
> >> >         <GribIndex>
> >> >                 <alwaysUse>true</alwaysUse>
> >> >                 
> >> > <dir>/home/tomcat/tomcat-main.4.3.14/content/thredds/cache/GribIndex</dir>
> >> >                 <policy>nestedDirectory</policy>
> >> >                 <scour>48 hours</scour>
> >> >                 <maxAge>90 days</maxAge>
> >> >         </GribIndex>
> >> >
> >> > so these will go into 
> >> > /home/tomcat/tomcat-main.4.3.14/content/thredds/cache/GribIndex/
> >> >
> >> > if you are sure this is what you want to do, for sure delete the gbx9 
> >> > and ncx files in the data dirs.
> >> >
> >> > 4) i need a file listing on 
> >> > /home/tomcat/tomcat-main.4.3.14/content/thredds/cache/GribIndex/
> >> >
> >> > 5) we might want to get this working on just one month first.
> >> >
> >> >
> >> >
> >> >> John,
> >> >> Assuming you want the source GRIB files, you can
> >> >> get them here:
> >> >>
> >> >> wget -r http://nomads.ncdc.noaa.gov/data/gfsanl/201001/20100104/
> >> >>
> >> >> For a complete find file list from the top level (nexus/gfsanl):
> >> >>
> >> >> http://nomads.ncdc.noaa.gov/data/gfsanl/findlist.txt.bz2
> >> >>
> >> >> I immediately noticed : gfsanl_3_20040302_0000_000.grb.ncx
> >> >> and the like... I'm not sure what created these...  may have been an
> >> >> older version of TDS.
> >> >> Should I get rid of them?
> >> >>
> >> >> -rw-rw-r-- 1 nomads nomads 5884 Nov 29 12:11
> >> >> ./200403/20040302/gfsanl_3_20040302_0000_000.grb.ncx
> >> >>
> >> >>
> >> >
> >> > Ticket Details
> >> > ===================
> >> > Ticket ID: GHR-456072
> >> > Department: Support THREDDS
> >> > Priority: High
> >> > Status: Open
> >> >
> >>
> >>
> >>
> >> --
> >> =======================================
> >> Dan Swank
> >> NCDC-NOMADS Project
> >> Team ERT/STG
> >> (NCDC Government Contractor)
> >> Data Access & Applications Branch
> >> Veach-Baley Federal Building
> >> 151 Patton Avenue
> >> Asheville, NC 28801-5001
> >> Email: address@hidden
> >> Phone: 828-271-4007
> >> ============================================
> >> Any opinions expressed in this message are mine personally and
> >> do not represent any position from STG, ERT, NCDC, or NOAA
> >> ============================================
> >>
> >>
> >
> >
> > Ticket Details
> > ===================
> > Ticket ID: GHR-456072
> > Department: Support THREDDS
> > Priority: Critical
> > Status: Open
> >
> 
> 
> 
> --
> =======================================
> Dan Swank
> NCDC-NOMADS Project
> Team ERT/STG
> (NCDC Government Contractor)
> Data Access & Applications Branch
> Veach-Baley Federal Building
> 151 Patton Avenue
> Asheville, NC 28801-5001
> Email: address@hidden
> Phone: 828-271-4007
> ============================================
> Any opinions expressed in this message are mine personally and
> do not represent any position from STG, ERT, NCDC, or NOAA
> ============================================
> 
> 


Ticket Details
===================
Ticket ID: GHR-456072
Department: Support THREDDS
Priority: Critical
Status: Open