[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: New Client Reply - [THREDDS !GHR-456072]: nullPointerException in when accessing 4.3.15 GRIB featureCollection's

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: Re: New Client Reply - [THREDDS !GHR-456072]: nullPointerException in when accessing 4.3.15 GRIB featureCollection's
  • Date: Tue, 28 May 2013 17:29:04 -0600
wow, i didnt envision this scale. It should work if you dont run out of memory, but probably need to think of better algorithms. 

however, once its indexed, shouldnt do it again, if you use

<update startup="nocheck" trigger="allow" />

On 5/28/2013 9:17 AM, Dan Swank - NOAA Affiliate wrote:

Morning John,
Assume your still waiting on logs.. will get thta
to you shortly.   Looks like I stepped into
a 14 day process with indexing the NDFD
4667691 gbx9's created over the weekend and its
still got about 15 million to go.     The spin-up I
mentioned took approximate 15 hours.   So unless
I comment the NDFD away, restarting will cost alot of time.


On Thu, May 23, 2013 at 4:06 PM, Dan Swank - NOAA Affiliate
<address@hidden <mailto:address@hidden>> wrote:

    One quick note
    the ruc130anl FC is actually working from
    the openDap form.   ruc252anl is not.   the
    errors still exist in the ruc130anl logs .. however.
    But it does not seem to be affecting functionality.




    On Thu, May 23, 2013 at 3:37 PM, Dan Swank - NOAA Affiliate
    <address@hidden <mailto:address@hidden>> wrote:

        John,

        After opening it up to the entire NDFD tree.... the scan log
        seems to
        cease here.... wondering if there is a "spin up" period while its
        finding files ... and nothing is written to the log for a period
        of time ?
        (talking scale of millions of files )

        Probably just need to give it more time.

        2013-05-23T15:12:35.655 -0400 INFO  - FeatureCollection added =
        name ='NDFD' type='GRIB'
           spec='/global/nomads/nexus/ndfd/**/......_...._#yyyyMMdd#....$'
           timePartition =1 day
           updateConfig =UpdateConfig{startup=true, recheckAfter='null',
        rescan='null', triggerOk=true, deleteAfter=null}
           protoConfig =ProtoConfig{choice=Penultimate, change='null',
        param='null', outerNcml='null', cacheAll=true}
           hasInnerNcml =false
           gribConfig =GribConfig: datasetTypes=[Best, Files]


        NNR2 :

        After battling a bit with the spec and mark, I got this:
        -- seemingly odd exception.

        ---------

        2013-05-23T15:23:55.682 -0400 INFO  - FeatureCollection added =
        name ='NNR2pgrbft' type='GRIB'

        spec='/global/nomads/nexus/reanalysis-2/6hr/pgb.ft06/pgb.ft06.#yyyyMM'
           timePartition =1 year
           updateConfig =UpdateConfig{startup=true, recheckAfter='null',
        rescan='null', triggerOk=true, deleteAfter=null}
           protoConfig =ProtoConfig{choice=Penultimate, change='null',
        param='null', outerNcml='null', cacheAll=true}
           hasInnerNcml =false
           gribConfig =GribConfig: datasetTypes=[Best, Files]

        2013-05-23T15:27:21.062 -0400 INFO  - NNR2pgrbft : was scanned
        nfiles= 360
        2013-05-23T15:27:21.062 -0400 INFO  - NNR2pgrbft: createIndex
        
/home/tomcat/tomcat-main.4.3.15/content/thredds/cache/GribIndex/global/nomads/nexus/reanalysis-2/6hr/pgb.ft06/NNR2pgrbft.ncx
        2013-05-23T15:27:21.063 -0400 ERROR - Fail to create/update
        collection
        java.lang.RuntimeException: SimpleDateFormat = yyyyMM fails on
        pgb.ft06.198306 ParseException:Unparseable date: "pgb.ft06.198306"
                 at
        
ucar.nc2.units.DateFromString.getDateUsingCompleteDateFormatWithOffset(DateFromString.java:214)
                 at
        
ucar.nc2.units.DateFromString.getDateUsingDemarkatedCount(DateFromString.java:99)
                 at
        
thredds.inventory.DateExtractorFromName.getDate(DateExtractorFromName.java:67)
                 at
        
thredds.inventory.DateExtractorFromName.getCalendarDate(DateExtractorFromName.java:74)
                ~~~~

        ---
        This is probably all from me today,
        See you Tuesday.





        On Thu, May 23, 2013 at 8:56 AM, Dan Swank - NOAA Affiliate
        <address@hidden <mailto:address@hidden>> wrote:

            Okay, I will flush the .ncx for ruc.

            NDFD is purring now for 1 days worth.
            It will take a long time to generate gbx9 for the entire
            online period,
              we will see how far it gets over the extended weekend.
              That will be a great stress test.
            In the meantime,
            I will take a shot at the GDS hashes for NDGD.


            On Wed, May 22, 2013 at 9:04 PM, John Caron
            <address@hidden <mailto:address@hidden>> wrote:

                havent been able to reproduce this one.

                delete all the collection ncx and restart tds. then send
                me the logs.

                im gone starting tommorrow for a long weekend, so we
                will resume next tuesday

                ciao!


                On 5/22/2013 11:57 AM, Dan Swank - NOAA Affiliate wrote:

                    Got the correct version running this morning.

                    Heres some logs, some insight on the fc.ruc252,
                         but not too much
                    
http://nomads.ncdc.noaa.gov/__data/gfsanl/201001-test/__tdsLogs-20130522.tar.bz2
                    
<http://nomads.ncdc.noaa.gov/data/gfsanl/201001-test/tdsLogs-20130522.tar.bz2>




                    On Tue, May 21, 2013 at 5:38 PM, John Caron
                    <address@hidden <mailto:address@hidden>
                    <mailto:address@hidden
                    <mailto:address@hidden>__>> wrote:



                         On 5/21/2013 2:40 PM, Dan Swank - NOAA
                    Affiliate wrote:

                             Running Version 4.3.17-SNAPSHOT - 20130515.1958


                         then it should say:

                         Version 4.3.17-SNAPSHOT - 20130517.1111




                    --
                    ==============================__=========
                    Dan Swank
                    NCDC-NOMADS Project
                    Team ERT/STG
                    (NCDC Government Contractor)
                    Data Access & Applications Branch
                    Veach-Baley Federal Building
                    151 Patton Avenue
                    Asheville, NC 28801-5001
                    Email: address@hidden
                    <mailto:address@hidden>
                    <mailto:address@hidden <mailto:address@hidden>>

                    Phone: 828-271-4007 <tel:828-271-4007>
                    ==============================__==============
                    Any opinions expressed in this message are mine
                    personally and
                    do not represent any position from STG, ERT, NCDC,
                    or NOAA
                    ==============================__==============




            --
            =======================================
            Dan Swank
            NCDC-NOMADS Project
            Team ERT/STG
            (NCDC Government Contractor)
            Data Access & Applications Branch
            Veach-Baley Federal Building
            151 Patton Avenue
            Asheville, NC 28801-5001
            Email: address@hidden <mailto:address@hidden>
            Phone: 828-271-4007 <tel:828-271-4007>
            ============================================
            Any opinions expressed in this message are mine personally and
            do not represent any position from STG, ERT, NCDC, or NOAA
            ============================================




        --
        =======================================
        Dan Swank
        NCDC-NOMADS Project
        Team ERT/STG
        (NCDC Government Contractor)
        Data Access & Applications Branch
        Veach-Baley Federal Building
        151 Patton Avenue
        Asheville, NC 28801-5001
        Email: address@hidden <mailto:address@hidden>
        Phone: 828-271-4007 <tel:828-271-4007>
        ============================================
        Any opinions expressed in this message are mine personally and
        do not represent any position from STG, ERT, NCDC, or NOAA
        ============================================




    --
    =======================================
    Dan Swank
    NCDC-NOMADS Project
    Team ERT/STG
    (NCDC Government Contractor)
    Data Access & Applications Branch
    Veach-Baley Federal Building
    151 Patton Avenue
    Asheville, NC 28801-5001
    Email: address@hidden <mailto:address@hidden>
    Phone: 828-271-4007 <tel:828-271-4007>
    ============================================
    Any opinions expressed in this message are mine personally and
    do not represent any position from STG, ERT, NCDC, or NOAA
    ============================================




--
=======================================
Dan Swank
NCDC-NOMADS Project
Team ERT/STG
(NCDC Government Contractor)
Data Access & Applications Branch
Veach-Baley Federal Building
151 Patton Avenue
Asheville, NC 28801-5001
Email: address@hidden <mailto:address@hidden>
Phone: 828-271-4007
============================================
Any opinions expressed in this message are mine personally and
do not represent any position from STG, ERT, NCDC, or NOAA
============================================