Re: NOAAPORT receive system feeding LDM

Hi all,

Having more sites injecting NOAAport data into the IDD is an exciting
prospect, but I suggest we do a few controlled experiments and come up with
an overall strategy before moving ahead with this on the main IDD
distribution topology.   We need to examine how the products are packaged
on the different systems, which feedtypes will be used, and so forth.

Of course, if individual universities want to do some experiments on their
own, that's up to them, but I think it would be best to keep that
datastream separate from the main IDD distribution until we get a better
handle on what's involved.

Thanks,

--Ben
******************************************************
   Ben Domenico                Unidata Deputy Director
   http://www.unidata.ucar.edu           P.O. Box 3000
   address@hidden              Boulder, CO 80307
   (303)497-8631                    FAX: (303)497-8690

--On Tuesday, October 19, 1999, 12:35 PM -0600 Robb Kambic
<address@hidden> wrote:

> On Mon, 18 Oct 1999, Jim Koermer wrote:
> 
>> Mike,
>> 
>> We just had a full Unisys NOAAPORT system installed here at Plymouth
>> State, a little over a week ago. Based on our preliminary experience, I
>> would like to offer a few cautions before feeding into the IDD at this
>> time:
>> 
> Hiya,
> 
> I talked to Dan Vietor about the differences between the unisys,
> alden and SSEC NOAAport systems.  He confirmed my thoughts that all the
> streams( products ) look different to the LDM for the same product.
> Therefore a site receiving all the stream would get 3 times the amount of
> data on NOAAport, this would cause havoc to the downstream nodes and
> the networks. I propose to any site receiving 2-3 of the streams implement
> some type of script to check that the current source is receiving data. If
> the current source is not receiving, fail over to an alternate. The
> ldmfail script could be used as a starting template.  Also, NOAAport has
> products that are currently being filtered out of the IDD, ie the NEXRAD
> products, etc. If these products are passed through then the downstream
> nodes would have to install the filter in the request line or increase
> their LDM queue to accommodate these products.
> 
> filter:
> 
> "^[^S]|^S[^D]|^SD..[^59]|^SD..9[^7]"
> 
> 
> I think it is good that more sites are installing NOAAport systems,
> increases the reliability of the IDD and probably decreases the latencies
> at sites.  My concern is that downstream sites be prepared so the NOAAport
> stream doesn't cause some type of problem. Also, I'm interested in how the
> systems are functioning.
> 
> Robb...
> 
> 
> 
>>      - The NOAAPORT 1 seems to lock up more frequently than I would
>>      like to see. Over the past ten days, I've had to reboot it
>>      on about four occasions. Until this stabilizes, this behavior
>>      would leave downstream sites with frequent periods of no data.
>> 
>>      - You would also need to have downstream users be more specific
>>      with the pqact.conf specifications or they might become
>>      overwhelmed with data. When I turned on the pqing ingest on one
>>      of my machines running LDM, about 2 GB of disk space got used up
>>      in about 10 hours and I wasn't even ingesting NOAAPORT II (GOES-
>>      E) on that machine. Disk storage from the current IDD feed ramps
>>      up much less quickly.
>> 
>> Once these issues are resolved, you could set up the pqing ingest
>> process on the your IDD server and I believe that it would then just
>> pass along the data automatically to downstream sites.
>> 
>>                              Jim
>> -- 
>> James P. Koermer             E-Mail: address@hidden
>> Professor of Meteorology     Office Phone: (603)535-2325
>> Natural Science Department   Office Fax: (603)535-2723
>> Plymouth State College       WWW: http://vortex.plymouth.edu/
>> Plymouth, NH 03264
>> 
>> 
>> 
>> Michael W Dross wrote:
>> > 
>> > We have just installed a NOAAPORT receive system from Unisys that is
>> > has both the NWSTG and GOES EAST channels.  My intention
>> > is to feed the LDM with the output and make the data available over
>> > the IDD to a university feed site (probably NIU) as a another feed
>> > source for the IDD community.  We are connected to the internet via a
>> > T3 with Sprintlink
>> > 
>> > My question is what is the best way to feed the LDM the NOAAPORT data?
>> > How is Unidata doing it?  I have quite a bit of flexibility in
>> > distributing the data from the NP receive machines. FTP, NFS, Socket
>> > communications.  Dan Vietor at Unisys had told me that the LDM
>> > buffers could not keep up with a direct socket feed into the ldm and
>> > suggested that we dump the NP data into 2 minute temporary files then
>> > using `pqing` and ingest them in that way. I am doing something
>> > similar to that now with the FOS data. I would rather feed it directly
>> > somehow.
>> > 
>> > If anyone has any suggestions please let me know as my goal is to
>> > essentially duplicate the NP feed from here that Unidata is send out.
>> > 
>> > Thanks!
>> > 
>> > Mike Dross
>> > address@hidden
>> 
> 
> =========================================================================
> ====== Robb Kambic                               Unidata Program Center
> Software Engineer III                    Univ. Corp for Atmospheric Research
> address@hidden                   WWW: http://www.unidata.ucar.edu/
> =========================================================================
> ======
>