[Support #FLO-999289]: IDD Access and Participation

Hi John,

re:
> Yet again I must confess the courtesy and promptness of your responses
> leave me smiling. Fortune 500 companies should have such willing and able
> supporters of their products!

Thanks for the kind words.  We take very seriously the job of supporting
our community.

re: I sent a note to the LDM/IDD contact at Texas A&M
 
> Great, We look forward to working with them or another member institution.

I got word back late last night:

 "Yes, I'm interested, and I'll give him a call on Monday.  I've got the
  bandwidth.  I'll keep you in the loop so  we can make sure the community
  can see the benefits."

So, you should receive a phone call tomorrow (Monday) or the day after.
If you don't please let me know and I will follow-up.

re: ground-based truth product for severe weather events

> This project has been on the drawing board for the past 4 years, and we
> finally accepted a programming bid on January 24th. We hope to move it
> forward quickly. This could allow our beta-testing to begin in the mid
> to late spring Storm Season if all goes well (fingers crossed)

Sounds great!

re: system needed to run the LDM

> Our servers are all hosted off-site (out of office). The "heavy hitters" of
> are servers are VPS and are incredible at their process ability.
> 
> Two questions with the type of setup we have. (first sounds silly, but I
> must ask).
> 
> 1) Are their any special issues with using the LDM to capture the IDD at the
> remote server location but doing the processing (in office) on one of our 
> local
> machines by tapping the data on the server? This would take a huge load off 
> of our
> server and keep it from dragging down on decoding.

No, but you would lose the benefit of being able to kick off processing upon
receipt of a particular product of interest.  This event-driven processing
is one of the big benefits of using the LDM.  You could, of course, kick
off processing on a remote machine using RPC or some other event notification.

> 2) Since this is a fairly "unique" software and configuration setup, are
> there people available who could assist in the install of the system. While
> I know Unidata offers no "official" support of the software, are their people
> out there who are familiar enough with the system that could help in the 
> install
> (perhaps for compensation)?

Doing a first-time installation of the LDM should take no more than a half
hour assuming, of course, that the person doing the installation is familiar
with *nix and has 'root' privilege (one part of the configuration of the
LDM requires 'root' to configure use of syslogd; another requires 'root'
privilege to set the setuid bit on two executables created in the build.
Upgrades of the LDM take on the order of 5 minutes.  Configuration of
what to do upon receipt of products is what takes more time depending on
what a user's objectives are.

re:
> We will need to review our bandwidth issues with each individual product
> selection. Is there a place to review (typical) throughput information on
> each data product?

Not on each data product, but there is good information in our real time
statistics pages on how much data is available for every IDD feed.  For
instance, check out:

Unidata HomePage
http://www.unidata.ucar.edu
  IDD
  http://www.unidata.ucar.edu/software/idd
    Real-Time IDD statistics
    http://www.unidata.ucar.edu/software/idd/rtstats/
      Statistics by Host
      http://www.unidata.ucar.edu/cgi-bin/rtstats/siteindex

      idd.unidata.ucar.edu
      http://www.unidata.ucar.edu/cgi-bin/rtstats/siteindex?idd.unidata.ucar.edu

      Cumulative volume summary
      
http://www.unidata.ucar.edu/cgi-bin/rtstats/rtstats_summary_volume?idd.unidata.ucar.edu

This gives a list of the data flowing into the toplevel IDD relay node
that we maintain here in UCAR, idd.unidata.ucar.edu:

Data Volume Summary for idd.unidata.ucar.edu

Maximum hourly volume   7309.435 M bytes/hour
Average hourly volume   3400.194 M bytes/hour

Average products per hour     161787 prods/hour

Feed                           Average             Maximum     Products
                     (M byte/hour)            (M byte/hour)   number/hour
CONDUIT                1706.259    [ 50.181%]     4935.786    50142.348
NGRID                   643.945    [ 18.938%]     1129.108    15504.348
NEXRAD2                 434.530    [ 12.780%]      544.767    30075.261
HDS                     215.287    [  6.332%]      437.078    17998.674
NIMAGE                  149.144    [  4.386%]      227.037      196.391
NEXRAD3                  87.371    [  2.570%]      102.932    18266.652
FNEXRAD                  80.745    [  2.375%]       89.485       68.326
IDS|DDPLUS               31.306    [  0.921%]       38.147    29135.978
EXP                      22.677    [  0.667%]       29.569      339.913
UNIWISC                  22.499    [  0.662%]       27.284       20.826
DIFAX                     4.396    [  0.129%]       21.193        6.065
FSL2                      2.033    [  0.060%]        2.134       22.304
LIGHTNING                 0.001    [  0.000%]        0.004        9.696

As you can see, if one got all un-restricted IDD feeds, one would be getting
up to 7.3 GB/hr of data.

> Right now we use 1TB of data throughput, but if we would be taxing that with
> the total number of data-feeds selected, then we would trim a bit.

1 TB -- one terabyte?  Are you referring to disk storage or Internet
bandwidth?  For reference, UCAR has a 10 Gbps connection to I2, and I
think that Texas A&M does also.
  
re:
> Look forward to sharing our new data with the community later this year if
> we can jump through our own little set of hoops at our end.

Sounds great.  We will be very interested in seeing how this all evolves.

By the way, I won't be at the office on Monday or Tuesday of this week --
'tis skiing time in the Colorado Rockies :-)

Cheers,

Tom
--
****************************************************************************
Unidata User Support                                    UCAR Unidata Program
(303) 497-8642                                                 P.O. Box 3000
address@hidden                                   Boulder, CO 80307
----------------------------------------------------------------------------
Unidata HomePage                       http://www.unidata.ucar.edu
****************************************************************************


Ticket Details
===================
Ticket ID: FLO-999289
Department: Support IDD
Priority: Normal
Status: Closed