[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[IDD #TAI-164591]: 20190418: splitting feed REQUESTs

This archive contains answers to questions sent to Unidata support through mid-2025. Note that the archive is no longer being updated. We provide the archive for reference; many of the answers presented here remain technically correct, even if somewhat outdated. For the most up-to-date information on the use of NSF Unidata software and data services, please consult the Software Documentation first.

  • Subject: [IDD #TAI-164591]: 20190418: splitting feed REQUESTs
  • Date: Thu, 18 Apr 2019 14:12:23 -0600
Hi Art,

re:
> I originally used the creation seconds to do the split, but then decided that 
> tenths of
> seconds might be better.  Of course, the trick is to try and use something 
> random.

If the products were all more-or-less the same size, then I would agree that
using something random would be best.  Because of the major difference in
product sizes, I think a different approach is called for.

re:
> The
> 2-way below is probably safer because it can never miss data no matter how 
> the feed might
> change.  The second way assumes digits must occur in that 4th place from the 
> end.  I
> should probably tweak it by adding a [^0-9] line or something as well.  
> Here's the
> section in my ldmd.conf:
> 
> 
> # Split feed into two streams assuming products ending like 
> "yyyyjjjhhmmsss.nc".
> # We split based on the tenths of seconds of generation.
> 
> # 2-way split
> #request        SATELLITE "[02468]...$" idd.unidata.ucar.edu
> #request        SATELLITE "[^02468]...$" idd.unidata.ucar.edu
> 
> # 10-way split
> request SATELLITE "0...$" idd.unidata.ucar.edu
> request SATELLITE "1...$" idd.unidata.ucar.edu
> request SATELLITE "2...$" idd.unidata.ucar.edu
> request SATELLITE "3...$" idd.unidata.ucar.edu
> request SATELLITE "4...$" idd.unidata.ucar.edu
> request SATELLITE "5...$" idd.unidata.ucar.edu
> request SATELLITE "6...$" idd.unidata.ucar.edu
> request SATELLITE "7...$" idd.unidata.ucar.edu
> request SATELLITE "8...$" idd.unidata.ucar.edu
> request SATELLITE "9...$" idd.unidata.ucar.edu

Thanks.  The approach taken by ERAU was much different, and it is having
the effect of dropping latencies to near-zero values:

http://rtstats.unidata.ucar.edu/cgi-bin/rtstats/iddstats_nc?DIFAX+hurricane.db.erau.edu

I'll share what I get back from ERAU if your are interested...

Thanks for the quick response!

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: TAI-164591
Department: Support IDD
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.