20050204: 20050202: Late 1 degree GFS and late eta's on CONDUIT] (cont.)

Tiziana,

When the 0.5 degree GFS data was added to the CONDUIT data stream,
it created certain performance problems on the machine responsible for
placing the data into the data stream. We have actively been working to
improve the speed at which the data is placed into the data queue,
and have arrived at a point in the last day or so where the delay in
posting of the data to where the system can place it into the LDM queue 
is improved. The peak data occurs when 5 NCEP models are posting output
concurrently, and that generates added load on the host system.
These data are inserted as quickly as possible into the LDM.
Since different users have expressed opposing priorities on
which data sets they need most readily, it is not a simple problem to
prioritize data...or even delete certain data products from delivery. 
At this point, the LDM does not allow weighting of data within the queue,
so we will try to maintain as soon as available delivery.


Steve Chiswell
Unidata User Support



>From: Tiziana Cherubini <address@hidden>
>Organization: UCAR/Unidata
>Keywords: 200502042101.j14L1Rv2002750

>Hi Tom,
>
>thanks again for your nice reply. But the more I think about this
>all story the more doesn't make much sense from the users point
>of view. I mean, what's the point to have more data in the stream
>if then we can't use any of them timely? Wouldn't be better to
>give some priority to some of the data, so that at least the
>users get some in time. Would the GFS 0.5 deg resolution come
>any faster than the 1 deg or are they injected in the stream at
>the same time? Sorry for the bother, but I'm trying to understand
>and see if I can find a way around so that we can continue with
>out forecasts. At the moment we are in the conditions of using
>a GFS cycle 6 hours older than what we would in normal conditions.
>Thanks so much for you attention.
>
>Ciao ciao,
>                       Tiziana.
>
>---------------------------------------------------------------
> Tiziana Cherubini, Ph.D.             Department of Meteorology
> Research Meteorologist               University of Hawaii
> Mauna Kea Weather Center             2525 Correa Rd. HIG 367
 http://mkwc.ifa.hawaii.edu           Honolulu, Hawaii 96822
>
> tel.: (808) 956-4593                 fax : (808) 956-2877
>---------------------------------------------------------------
>
>On Wed, 2 Feb 2005, Tom Yoksas wrote:
>
>> >From:  Tiziana Cherubini <address@hidden>
>> >Organization:  University of Hawaii
>> >Keywords:  200502010005.j1105Fv2005162 IDD CONDUIT
>>
>> Hi Tiziana,
>>
>> >thanks for the informative reply. It looks like we only have to
>> >wait and hope that things get faster soon again.
>>
>> Unfortunately, that is the case.  I looked at the real time stats for
>> your CONDUIT ingest machine, kiloaoloa.soest.hawaii.edu:
>>
>> http://my.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+kiloaoloa.soe
> st.hawaii.edu
>>
>> and see that it is as good as can be expected, so there is no
>> additional latency being added by the IDD relay of the data once it is
>> injected at NCEP.
>>
>> >However it's still not very clear to me how come the data were
>> >coming in more or less timely before january 05. Are there more
>> >data in the stream now that there were in 2004?
>>
>> Yes, the 0.5 degree GFS data was added to the CONDUIT stream, and this
>> pushed the processing load past a critical point on the data injection
>> machine.
>>
>> >Thanks a lot,
>>
>> No worries.
>>
>> Ciao,
>>
>> Tom
>> --
>> +---------------------------------------------------------------------------
> --+
>> * Tom Yoksas                                             UCAR Unidata Progra
> m *
>> * (303) 497-8642 (last resort)                                  P.O. Box 300
> 0 *
>> * address@hidden                                   Boulder, CO 8030
> 7 *
>> * Unidata WWW Service                             http://www.unidata.ucar.ed
> u/*
>> +---------------------------------------------------------------------------
> --+
>>
>
--
NOTE: All email exchanges with Unidata User Support are recorded in the
Unidata inquiry tracking system and then made publicly available
through the web.  If you do not want to have your interactions made
available in this way, you must let us know in each email you send to us.