[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[CONDUIT #CPL-818261]: CONDUIT data and GFS

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [CONDUIT #CPL-818261]: CONDUIT data and GFS
  • Date: Thu, 06 Aug 2015 10:13:25 -0600
Hi Wesley,

re:
> Seems we haven't been getting all of the usual data files (at least of
> the correct size, the files themselves are coming in) through our
> upstream feed(s). You can see the drop-off in volume:
> http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?CONDUIT+polarmet12.mps.ohio-state.edu

Hmm...  The real time stats that both UW/AOS and UIllnois
show that they appear to be receiving all of the CONDUIT data:

UW/AOS:
http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?CONDUIT+idd.aos.wisc.edu

UIllinois
http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?CONDUIT+flood.atmos.uiuc.edu

The product residency time in the LDM queues at UW and UIllnois are not
as long as we would like, but all of the data is flowing into them.

re:
> And the file sizes:
> [ldm@polarmet12 prod]$ ll 
> gfs.2015080100/gfs.t00z.pgrb2.0p25.f192gfs.2015080600/gfs.t00z.pgrb2.0p25.f192
> -rw-rw-r-- 1 ldm ldm 224861078 Aug  1 00:31 
> gfs.2015080100/gfs.t00z.pgrb2.0p25.f192
> -rw-rw-r-- 1 ldm ldm   3476091 Aug  6 00:53 
> gfs.2015080600/gfs.t00z.pgrb2.0p25.f192

Yes, the file size drop matches the decrease in CONDUIT volume being
reported in your real time stats.

re:
> I'm going to restart our LDM daemon... but wanted to check on this
> upstream, too, in case there is a larger problem.
> 
> Thanks for any feedback you can provide!

Questions:

- are you experiencing difficulties with other IDD datastreams
  that you are ingesting?

  The sharp drop off has a hint that there may be some "packet
  shaping" (artificial bandwidth limiting) going on, or that
  you have actually saturated your network connection.  Comparing
  what is happening with other IDD feeds will help to clarify
  if either of these scenarios reflect what is happening.

- would it be possible for us to get a login as your 'ldm'
  user so we could poke around?

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: CPL-818261
Department: Support CONDUIT
Priority: Normal
Status: Closed