[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[CONDUIT #EZK-253623]: CONDUIT GFS files not complete

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [CONDUIT #EZK-253623]: CONDUIT GFS files not complete
  • Date: Wed, 27 Sep 2006 11:29:19 -0600
Giovanni.

The GFS files being delivered by the LDM are complete. For example,
the 00Z data gfs.t00z.pgrb2f78 as you reference below:
All fields (37392062 bytes) were received at Unidata:

http://motherlode.ucar.edu/cgi-bin/ldm/statusgen_new.csh?/nccf/com/gfs/prod/gfs.2006092700/gfs.t00z.pgrb2f78
via:
http://motherlode.ucar.edu/cgi-bin/ldm/conduit_reception_new.csh?/nccf/com/gfs

I would need to know the name ofthe LDM host on your end that is doing the data 
processing.

The machine moingabe reports little latency, and so should be receiving all its 
requests:
http://www.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+moingobe.cptec.inpe.br

However, some machines in .br I looked at showed large CONDUIT latencies:
http://www.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?CONDUIT+cruz.inmet.gov.br


There are 2 possibilities on why you are seeing small files even though the 
upstream
is receiving all data:
1) Your LDM host is falling behind, and the data gets scoured out of the queue 
on the upstream
    machine before your LDM is able to receive the data.

2) You receive the data timely, but it gets scoured out of your local queue 
before the pqact process
is able to send the data to the decoder.

Note that in GEMPAK5.9.3, I made a significant speed improvement to dcgrib2 so 
that it would
store the GRIB2 messages without expanding the data and repacking. This is 
quite necessary for
the decoder to be able to keep up with the data volumes, and so that the data 
isn't
scoured out of the local queue before pqact is able to send all the data to the 
decoder.
See: http://www.unidata.ucar.edu/software/gempak/GEMPAK5.9/whats_new.html#5.9.3

If you still have problems, please tell me what LDM host is reeding the data, 
and what GEMPAK
version you are using.

Steve Chiswell
Unidata User SUpport





> Hello,
> 
> I've noted that frequently some files of GFS from CONDUIT feed are not 
> complete.
> for exemple today the 00Z run the file  gfs.t00z.pgrb2f78 has a size ok only 
> 222K.
> What I can't understand is that the same file is complete at ncep ftp server:
> ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/gfs/prod/gfs.2006092700/
> 
> Is this a bug in LDM? Or can it be some problem during the transmission or
> during saving the file to hard disk?
> 
> Thanks,  Giovanni
> 
> 
> --
> Giovanni Dolif
> Meteorologista (Msc.) - CPTEC/INPE
> http://www.cptec.inpe.br
> Tel.: 55 - (12) 3186-8459
> 
> 


Ticket Details
===================
Ticket ID: EZK-253623
Department: Support CONDUIT
Priority: Normal
Status: Closed