[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Problems with GRIB feature collections
- Subject: Re: Problems with GRIB feature collections
- Date: Wed, 6 Dec 2017 14:19:25 -0700
Hi Roy,
This might take a bit. If we look at the time range on the "Best"
collection, we get:
Start: 2013-02-21T00:00:00Z
End: 2017-12-09T06:00:00Z
However, if we look at the time range on the "latest" (201712), we get:
Start: 2017-12-01T00:00:00Z
End: 2017-12-04T18:00:00Z
It's not clear where anything after 2017-12-04T18:00:00Z is coming
from, but I think I have a way of tracking it down if it isn't a bug
(again, it will just take a bit of time).
Cheers,
Sean
On Wed, Dec 6, 2017 at 12:47 PM, Roy Mendelssohn - NOAA Federal
<address@hidden> wrote:
> Okay the TwoD datasets are showing. I still see the same issue with the last
> time. Can easily send you sample data if that would help.
>
> -ROy
>
>> On Dec 6, 2017, at 11:38 AM, Sean Arms <address@hidden> wrote:
>>
>> Hmmm. Can you try removing the ncx files to force the TDM to regenerate
>> them? Both the TDS and TDM will need to be restarted.
>>
>> Cheers,
>>
>> Sean
>>
>> On Wed, Dec 6, 2017 at 12:17 PM Roy Mendelssohn - NOAA Federal
>> <address@hidden> wrote:
>> Adding that appears to have sent that dataset into never-neverland. Now I
>> get no response. DOn't see an error in the log file either.
>>
>> -Roy
>>
>> > On Dec 6, 2017, at 10:53 AM, Sean Arms <address@hidden> wrote:
>> >
>> > Greetings Roy,
>> >
>> > Sorry for the delay - I've been in a teleconference all morning.
>> >
>> > So the TDS may be picking up on the fact that some of those records
>> > are encoded as forecasts. That's possible, and I'll keep that in mind.
>> >
>> > The xml snippet should look like this (change made to first line to add
>> > TwoD):
>> >
>> > <gribConfig datasetTypes="TwoD Best LatestFile">
>> > <pdsHash>
>> > <useTableVersion>false</useTableVersion>
>> > </pdsHash>
>> > </gribConfig>
>> >
>> > Cheers,
>> >
>> > Sean
>> >
>> > On Wed, Dec 6, 2017 at 11:05 AM, Roy Mendelssohn - NOAA Federal
>> > <address@hidden> wrote:
>> >> BTW - I should add that the 1-degree data is a special run that they do
>> >> for us, and the way I understand it in their processing terms the 00 and
>> >> 12 files are considered real-time, while the 06 and 18 files are
>> >> considered "forecasts". They have different filenames because of that,
>> >> for example:
>> >>
>> >> US058GMET-GR1mdl.0018_0240_00000F0RL2017120400_0102_000000-000000pres
>> >> US058GMET-GR1mdl.0018_0240_00000F0OF2017120418_0102_000000-000000pres
>> >>
>> >> That is why in the regex for the time I have to skip two spaces. In case
>> >> this info helps any:
>> >>
>> >> wgrib -V
>> >> US058GMET-GR1mdl.0018_0240_00000F0RL2017120400_0102_000000-000000pres
>> >> rec 1:21:date 2017120400 PRMSL kpds5=2 kpds6=102 kpds7=0 levels=(0,0)
>> >> grid=240 MSL anl:
>> >> PRMSL=Pressure reduced to MSL [Pa]
>> >> timerange 0 P1 0 P2 0 TimeU 1 nx 360 ny 181 GDS grid 0 num_in_ave 0
>> >> missing 0
>> >> center 58 subcenter 0 process 18 Table 2 scan: WE:SN winds(N/S)
>> >> latlon: lat -90.000000 to 90.000000 by 1.000000 nxny 65160
>> >> long 0.000000 to -1.000000 by 1.000000, (360 x 181) scan 64 mode
>> >> 128 bdsgrid 1
>> >> min/max data 95236 104736 num bits 10 BDS_Ref 9523.6 DecScale -1
>> >> BinScale 0
>> >>
>> >> wgrib -V
>> >> US058GMET-GR1mdl.0018_0240_00000F0OF2017120418_0102_000000-000000pres
>> >> rec 1:0:date 2017120418 PRMSL kpds5=2 kpds6=102 kpds7=0 levels=(0,0)
>> >> grid=240 MSL anl:
>> >> PRMSL=Pressure reduced to MSL [Pa]
>> >> timerange 0 P1 0 P2 0 TimeU 1 nx 360 ny 181 GDS grid 0 num_in_ave 0
>> >> missing 0
>> >> center 58 subcenter 0 process 18 Table 3 scan: WE:SN winds(N/S)
>> >> latlon: lat -90.000000 to 90.000000 by 1.000000 nxny 65160
>> >> long 0.000000 to -1.000000 by 1.000000, (360 x 181) scan 64 mode
>> >> 128 bdsgrid 1
>> >> min/max data 94770 104260 num bits 10 BDS_Ref 9477 DecScale -1
>> >> BinScale 0
>> >>
>> >>
>> >> BTW - Note the different table, which is why I have include in the xml
>> >> the stuff about:
>> >>
>> >> <pdsHash>
>> >> <useTableVersion>false</useTableVersion>
>> >> </pdsHash>
>> >>
>> >>
>> >> Thanks,
>> >>
>> >> -Roy
>> >>
>> >>
>> >>> On Dec 6, 2017, at 6:09 AM, Sean Arms <address@hidden> wrote:
>> >>>
>> >>> For the second question regarding "full" vs "best", this has to do with
>> >>> how the overall collection is aggregated. For the half-degree NAVGEM,
>> >>> you have monthly collections of grids, but none of the forecasts
>> >>> overlap. Therefore, the overall collection aggregation exposes all of
>> >>> the grids. For the one degree, it looks like the TDS thinks there was an
>> >>> overlap in the grids (i.e. two grids with the same valid time), and so
>> >>> one of the overlapped grids was "better" (i.e. closer to a runtime) and
>> >>> the other was dropped to make a "best" collection. However, I don't see
>> >>> that happening in any of the monthly collections listed, so it is
>> >>> probably a bug. In the xml for these two collections, can you add "TwoD"
>> >>> to the datasetTypes attribute of the gribConfig element? That will give
>> >>> us a bit more information.
>> >>>
>> >>
>> >> **********************
>> >> "The contents of this message do not reflect any position of the U.S.
>> >> Government or NOAA."
>> >> **********************
>> >> Roy Mendelssohn
>> >> Supervisory Operations Research Analyst
>> >> NOAA/NMFS
>> >> Environmental Research Division
>> >> Southwest Fisheries Science Center
>> >> ***Note new street address***
>> >> 110 McAllister Way
>> >> Santa Cruz, CA 95060
>> >> Phone: (831)-420-3666
>> >> Fax: (831) 420-3980
>> >> e-mail: address@hidden www: http://www.pfeg.noaa.gov/
>> >>
>> >> "Old age and treachery will overcome youth and skill."
>> >> "From those who have been given much, much will be expected"
>> >> "the arc of the moral universe is long, but it bends toward justice" -MLK
>> >> Jr.
>> >>
>>
>> **********************
>> "The contents of this message do not reflect any position of the U.S.
>> Government or NOAA."
>> **********************
>> Roy Mendelssohn
>> Supervisory Operations Research Analyst
>> NOAA/NMFS
>> Environmental Research Division
>> Southwest Fisheries Science Center
>> ***Note new street address***
>> 110 McAllister Way
>> Santa Cruz, CA 95060
>> Phone: (831)-420-3666
>> Fax: (831) 420-3980
>> e-mail: address@hidden www: http://www.pfeg.noaa.gov/
>>
>> "Old age and treachery will overcome youth and skill."
>> "From those who have been given much, much will be expected"
>> "the arc of the moral universe is long, but it bends toward justice" -MLK Jr.
>>
>
> **********************
> "The contents of this message do not reflect any position of the U.S.
> Government or NOAA."
> **********************
> Roy Mendelssohn
> Supervisory Operations Research Analyst
> NOAA/NMFS
> Environmental Research Division
> Southwest Fisheries Science Center
> ***Note new street address***
> 110 McAllister Way
> Santa Cruz, CA 95060
> Phone: (831)-420-3666
> Fax: (831) 420-3980
> e-mail: address@hidden www: http://www.pfeg.noaa.gov/
>
> "Old age and treachery will overcome youth and skill."
> "From those who have been given much, much will be expected"
> "the arc of the moral universe is long, but it bends toward justice" -MLK Jr.
>